Here's a use case that seems more science fictional to me (as the parent of a 2yo) than warp drive: a robot that can gently restrain an uncooperative human baby while changing its diaper, with everything that entails: identifying and eliminating all traces of waste from all crevices, applying diaper cream as necessary, unfolding and positioning the new diaper correctly and quickly, always using enough but never too much force... not to mention the nightmare of providing any guarantees about safety at mass-market scale. Even one maimed baby, or even just a baby some robot neglects to prevent from falling off the changing table, is game over for that line of robots.
Is there any research program that could claim to tackle this? It's so far beyond folding laundry and doing dishes, which are already quite difficult.
I wouldn't bet my life on this tech _never_ materializing, but I would mistrust anyone who claimed it was feasible with today's tech. It calls for an entirely different kind of robotic perception, feedback, and control.
I don't see why that would be so hard. This is potentially easier than reliably shooting guns at people.
That machine will look like a bean bag couch in rough shape of a giant human hand, with few of cooperative work robotic arms. The couch part hugs and secures all limbs of the baby to into the party escort submission position, then the cobots move in to find the disassembly markers on the diaper to tear it open to remove it. Then a showerhead, then a hair dryer, then baby powder sprayer can be brought out and ran to clean any residues and take care of rashes. Finally, the new diaper can be brought in, baby wrapped, and the double sided tapes on it lightly pressed on to secure it.
The entire machine would probably cost less than 10 million USD per unit if mass produced at reasonable scales, and most technological elements needed in such machines would be readily available.
> It would require an entirely different kind of robotics.
I was 100% with you until suddenly this technical claim pops out. You might feel this way, and might be right, but why? Changing a diaper is crazy hard, I absolutely agree, but you seem to be just declaring from vibes that we 'require an entirely different kind of robotics'. Can you put your finger on why this is true?
Not nitpicking for the fun of it - I'm genuinely interested. Robot person.
Zero failure rates are a very different bar. Software must be treated as actively malicious from a hardware standpoint. So it comes down to designing hardware capable of the task that’s also incapable of causing harm.
Meanwhile it must also be strong enough to move and restrain a range of infants which is a level of force capable of harm without any possibility to fail deadly. Nuclear power plants are an incredibly simple problem by comparison yet zero possibility of harm isn’t the standard because it’s so unrealistic.
The main limitation right now is that robotics are very limited in their sense of touch.
After that, they are limited in their understanding of physics. After that, perhaps understanding of physics and physiology would come into play - but perhaps superhuman perception and reaction time could reduce the need for intuitive understanding physics and physiology.
I think it needs a water gun. If the diaper was a spray on layered rubber, like a sponge then an impermeable layer, and then you sprayed a solvent to clear the old diaper and poop and then spray on a new one.
You'd just need to slot them into styrups briefly or some socks on strings to move the legs into a good position.
But can this be done with baby skin and lung safe chemicals at a reasonable temperature?
Point being humanoid designs for robots that manipulate objects designed for humans are an artificially hard problem we have decided to fail at solving.
Well, Mr Robot person, would you let today's robotics change your clothes right now? If you wouldn't, then why would you allow it any where near a baby? If you would, why? What robot with what tech would you allow?
This is a great one. The manipulation is hard, but we're probably on a trajectory to be able to do it in 1-3 years if you were tolerant of some risk to the baby, but, of course, your tolerance for injuring babies is basically zero. I think 'risk & reliability' is a good potential category: there is the bar of 'got it to do a task reliably enough that we got a video' and the bar of 'got it to do a task reliably enough that I'd risk an infant in its grippers.
> but we're probably on a trajectory to be able to do it in 1-3 years
This is wildly optimistic. I quit working in robotics because I got tired of all the bullshit promises everybody made all the time. I'm not saying robotics isn't advancing or the work is unimportant, but the spokespeople are about as reliable as Musk when it comes to timelines.
I doubt it will happen in 10 years, even with a constrained environment and hardware that costs well into 6 digits.
I think GP was basically talking about doing it on a doll. As in, a robot in 1-3 years might be able to change diapers with occasional success, but half the tries will result in a dismembered diaper user: we'd use dolls in this scenario, since dismembering babies is taboo and generally frowned upon within the robotics community.
> your tolerance for injuring babies is basically zero.
Um, no it's not. Is absolutely zero tolerance. There is not weasel words out of this. If a robot was to cause any pain to the baby, there would be no remorse. There would be no front of mind thoughts to not repeat the same thing the next time. There would be no guilt for causing pain to the baby.
Why you would "basically" this the way you have is disturbing.
Sorry, this is me communicating like an engineer. In a technical sense risk of anything can only approach zero: never actually get there. I meant that there should be essentially zero chance, similar to holding a baby in your arms or putting it in a high chair, and probably less chance of injury than driving in a car with a baby in a car-seat. Basically zero.
I don't think the parent comment advocates for hurting babies. It just, probably correctly, states that cherry picked examples won't be representative of roboty safety with infants in the next years, but that true safety will improve over time as well.
real world treatment of babies is very different from the zero tolerance you've described. From pregnant mothers smoking/drinking to medical care unavailability to doctor errors to various toxin contaminated baby products and the environment (Flint leaded water comes to mind) to babies left in hot cars and other abuse to poor availability of daycare (even less availability of daycare good for mental development) to ...
Granted most of this is unintentional. The same about injuries by robots - we're supposedly talking about unintentional injuries here. So, if robots save money/time/effort (like Flint water switch) i'm not sure that the society would suddenly change its current approach to unintentional baby injuries and implement zero tolerance.
To illustrate - Uber self-driving killed a woman, and another self-driving maimed a woman in SF. Uber case was an obvious criminal gross negligence running with explicitly disabled emergency braking), and the company wiggled out of it in part by having to shut down the self-driving. Where is in SF it was an obvious case of technology limitations and teething issue, so there were no real severe consequences as we're much more tolerant to honest technological accidents (at least when they happen not to us personally).
Why? There's nothing particularly special about this problem. I would bet a year for an alpha version, and production version in 5 years. We are not exactly limited by mechanical engineering here, there's nothing particularly unique about the human hand that can't be replicated. Tele operated surgical robotics have been a thing for decades. Give it a few months for the multimodal robotic VLM/LAMs to catch up. In many ways this particular problem is a lot more well defined than e.g. self driving cars.
> a baby some robot neglects to prevent from falling off the changing table
that is when we think about 2 handed robots. 6 handed robot can easily have 2-3 hands assigned to tightly keeping the baby. Humanoid robots are handicapped by their similarity to humans which is really an artificial constraint. After all we aren't building airplanes using birds as the blueprint.
On the similar note - while not about baby, was just rewatching an early Bing Bang Theory season with this episode where Howard "falls right into the mechanical hand"
On the other hand - the baby will from the beginning develop an instinct to keep track of 6 hands flying around instead of just 2. Will help in future street fights :)
In general, looking at the AI coding agents i think we all either already feel or soon will feel disabled. And honestly i think human race with its perception of itself as the "top of the Creation" is due for a modesty lesson to help speed up the evolution. We're spending tremendous resources unproductively, be it wars or just ineffective economies, etc. We don't feel the urge to develop our civilization and to evolve ourselves in all aspects - from mental and biological to cyber-integration. The Mother Nature doesn't like such relaxed species.
Platinum Medal: Complete a gold medal task in the presence of multiple children between the ages of 3 and 8. The children must remain safe throughout (and after) the entire process.
Interesting post and great reference to [1] about why laundry hits a sweet spot of capability.
Interestingly the repeated critiques in the article are about sensor richness: primarily force feedback and tactility, which indicates lacking hardware. Software only robotics has a long and fraught history, but it really feels to me that current industrial hardware could be driven more intelligently without much change. No doubt the "ideal" robot for any given task requires developments in both.
I'm also curious about safety, since generally capable mechanisms need a multilayered safety stack that includes semantics, and cobot certification is likely not enough anymore. Examples: feeding someone the wrong pill, pouring a glass of water into electronics, cutting vegetables vs fingers.
I'd go for something in the manipulation of ropes or wires.
State of the art seems to be that they can untangle a loosely knotted cord.
Untying a short rope with a tightly pulled overhand knot in the middle seems like it's decades away. You have to be able to grip it well enough, then twist the rope and push (even though every physicist says pushing a rope is impossible).
Interesting. Futurism is super hard, but "decades" too far away to me. I think with strong 2 finger grippers this is probably close to state of the art, especially with a wrist force sensor, like the TRI setup.
You need to manipulate a large sheet, and you probably need to move around, bend down and lean over to reach all the corners. Bonus points for neat hospital corners on a flat sheet.
Putting pillows in pillowcases is another fun one. Usually pretty easy, probably a bronze medal.
Gold medal: put a UK super king size duvet inside a duvet cover. It's huge and awkward, there are buttons, and it's almost but not quite square (why??) so there's a good chance you'll get it round the wrong way and have to rotate it 90 degrees.
Standard evening family home tidy/reset - toys, books, clothes, shoes away in their places. All over the house.
Oh, and load/unload dishwasher. Same with laundry machines. Along with folding laundry, these are the domestic robot equivalents of 'de-mining' and 'search and rescue': the classic motivating use cases for mobile autonomous robots.
Consider examples using building tools like screwing in a drywall screw, or hammering a nail, using a paint roller, caulking a sink, minor plumbing repair with a torch and solder. These differ enough in terms of forces, state changes, and combined dexterity/acuity (two-handed proprioception) from the windex, sandwich and key examples
Maybe careful application of large amounts of force? Opening a jar, peeling garlic, splitting a squash, opening a soda can. This category seems like a good test of "grip" strength + force feedback + sense of touch.
How about something with unpacking items from a shopping bag, i suspect the difference in bags (standard plastic, reusable etc) and certain items can really crank the difficulty.
It can also create a good time of a story - open the door to get the grocery delivery, unpack the delivery etc.
something requiring navigating stairs while holding something full like a laundry basket. bronze - straight stairs, silver - one 90deg turn. gold - spiral.
something requiring co-ordination between 2 robots. think relay race which the olympics has. So say, moving a couch together.
btw love the idea and the silver body suit. good stuff.
Ooh. I like full body manipulation. Humans use hips & elbows to move laundry baskets. Two robot collaboration is good too. I wonder who I can convince to wear another silver suit.... :)
I love your list and it makes me think we are so far away from these things ever being feasible/cost effective compared to just hiring a poor person to do it. And the world is making a lot of poor people right now.
(HN link on Substack points at empty page instead of this one, at least before I made this comment.)
What I think is missing is marathon events. Biathalons and Triathalons.
We all know LLMs have a rather limited context window. Thus seeing robots do longer chains of events would be interesting to see that they're capable than a possibly rigged demo.
Something like: move a stack of boxes from one room to another. The boxes at the end also need to be stacked up. or how about pick up a box, go up some stairs, open a door, and put the box on a shelf on the other side.
Also, the real world is sloppy and messy and dirty and, to be real, kinda janky sometimes. Gold for unlocking a door with a key at a well-maintained office complex, (and opening it, and walking through it) is one thing, because facilities is going to replace the lock before it gets old and needs replacing, and we can assume the door fits in the frame properly so it doesn't need to be shoved or lifted up or yanked in order to be opened is easier than. But the real world is messy and sloppy and you gotta jiggle the key in just the right way in order to get it to work.
Closing the door (assuming the robots weren't raised in a robot barn) is also harder than it looks if the door is shitty and needs a proper slam in order to be fully closed. Also, the robot locking the door behind itself after it comes in.
Scanning a key card and opening a door, but the first try fails.
We're a long way from a general robot that can screw a simple screw together like you would to assemble Ikea furniture.
Object recognition.
Gather only the dishes from a messy coffee table and put them in the dish bin.
Pick up only the clothes from a messy floor and bed, and put them in the hamper.
Dump a hamper of clothes onto a table, and sort out stuff that doesn't want to go into the washing machine.
Terrain traversal.
Just walk 500 ft, but theres increasing levels of obstacles in the way.
We all saw Boston dynamics robot parkour videos, but what I want to see is a robot make it from the front door of Simpsons house to the kitchen in the back, but it's got to go through the living room, but it's hella messy, with Maggie and Bart and Lisa’s crap strewn all over, Homer’s got some beer bottles, some empty, some full, all over the floor and on the table, and all the robot has to do is walk from one side of the room to the far side of the room without stepping on anything, or knocking anything over. (Simpsons merely being a home layout that's familiar to most people. Doesn't need to actually be them.)
Ducking under a low ceiling. Climb over a barrier,
of varying shapes and sizes.
Other loocomotion. how much weight in its arms in front of it, holding a 5-lb briefcase with one hand while walking. Can it carry something on its back? What's the limit? Can it give piggyback rides?
A category for simulated. Let companies show off their robot's kinematics control systems, so have something on the level of CoppeliaSim, so the motors and the gears and the actuators are themselves simulated, vs a simple 3d video game where they are not. Plug their model into the simulated robot and see how well it just walks. If we remember QWOP, it's harder than it looks!
Obviously it's not going to be totally 100% accurate to the real world. The benefit of this is it lets people complete from all over world without having to replicate a very specific setup in the physical world, and compete from wherever they live am not have to fly to your facility to test, opening up a whole new world of contestants because they can now compete because they can afford it now.
At the end of the day, the most important challenge is, can it pick up a battery from the shelf, swap it with one of the two in its chassis, and put the dead one it just pulled out onto the charger?
Can I say something: could you all please stop inserting contaminated tool back into a jar of food? You use a clean tool to take out the amount you eat, and that's it.
You can put it back if the tool had touched nothing but: air, the food in the jar, and your hand at the operating end. Otherwise, that butter knife stays on your dish for the rest of the meal. The exception would be if you cleaned the tool, like bare minimum by wiping with a brand new piece of tissue paper(but that's kind of wasteful).
Is that an outrageous ask? I know it's probably not a huge deal, like free water and such, and my techniques are that of total amateur being never professionally involved in medical and/or bio science fields, but just, how can you stand possible breadcrumbs IN THE JAR!?
> but just, how can you stand possible breadcrumbs IN THE JAR!?
Bacteria can do a number on people... Kept telling the wife to not cross contaminate jars, and other products. But you know, she knows better...
Wife was eating some fish in can, puts spoon back (with lovely saliva bacteria), puts in fridge "i wil eat tomorrow", 2 days later she eats the leftover.
She enjoyed a hour+ of "fun" muscles contracting stomach cramps to the point it was almost hospital time. Learned her lesson, well, ... for fish.
Its always like "but i do not like to keep using fresh utensils". I am always: "the dishwasher does not care if it 20 or 40 utensils". Its the same amount of wash and cost. So stop trying to recycle utensils!
Think about how fast progress is being made now. When I was a kid in the early 2000s we would see some basic robot progress on movements (almost always from Boston Dynamics or sometimes China) and we thought it was incredible. Robot dogs running was amazing and five or so years later a backflip blew our minds. Those robots were specially designed and didn't look humanoid. Now we have bi-pedal humanoid robots and they walk and move fairly capably - even able to get up after falls. Now within the last year I have seen them learn Kung Fu, become really fast at getting up, become resistant to being knocked down by quite a lot of force, and now even doing tasks like those shown here.
Just imagine 2050 if the progress continues at this rate. I am both excited and really scared.
Almost all progress at doing tasks reliably has been made with 2 massive caveats:
1. Force. Walking, running, fighting, doing backflips, etc. all allow for large amounts of force, without a lot of dynamic precision required. Many common tasks require precise and dynamic force. E.g. for washing a window, pushing too hard breaks the glass while pushing too softly will leave streaks.
2. Environment interaction. Most reliable humanoid robots do minimal environment interaction beyond self-balancing. They walk/run/jump in environments that are largely open, with usually convex blocky obstacles. The real world has lots of tasks that require processing beyond low-resolution maps of solid/open space. E.g. I'd want to see a robot that can walk through a forest: jumping/stepping over thin branches that are hard to see, ducking under fallen logs, pushing though bendy branches without breaking them, avoiding ground that is muddy, and seeing through the current obstacle to determine if the obstacle beyond is traversable.
Just to reiterate, I don't see fast progress being made on doing these tasks reliably. It's easy to show 1/N success rate, and much much harder to show ~N/N success rate on these dynamic tasks.
Great post! Now somebody with the connections just needs to make it happen. For event five, slippery when wet, you should definitely include drying your hands on a towel, as it serves an important hygienic function.
I think my shirts just automatically get inside-outted
in my washer/dryer? I certainly don’t put them in that way, and it seems like I spend a lot of righting them when putting away laundry.
If it's vinyl applied with heat (numbers on a jersey as an example), they recommend turning inside out. That way, if it gets too hot in the dryer, it only sticks to itself instead of to other garments. Losing the one garment is better than multiple.
I've seen suggestions that having your clothes inside out in the wash helps them get cleaner. And if it keeps the AIs under control, you know, benefits.
Here's a use case that seems more science fictional to me (as the parent of a 2yo) than warp drive: a robot that can gently restrain an uncooperative human baby while changing its diaper, with everything that entails: identifying and eliminating all traces of waste from all crevices, applying diaper cream as necessary, unfolding and positioning the new diaper correctly and quickly, always using enough but never too much force... not to mention the nightmare of providing any guarantees about safety at mass-market scale. Even one maimed baby, or even just a baby some robot neglects to prevent from falling off the changing table, is game over for that line of robots.
Is there any research program that could claim to tackle this? It's so far beyond folding laundry and doing dishes, which are already quite difficult.
I wouldn't bet my life on this tech _never_ materializing, but I would mistrust anyone who claimed it was feasible with today's tech. It calls for an entirely different kind of robotic perception, feedback, and control.
I don't see why that would be so hard. This is potentially easier than reliably shooting guns at people.
That machine will look like a bean bag couch in rough shape of a giant human hand, with few of cooperative work robotic arms. The couch part hugs and secures all limbs of the baby to into the party escort submission position, then the cobots move in to find the disassembly markers on the diaper to tear it open to remove it. Then a showerhead, then a hair dryer, then baby powder sprayer can be brought out and ran to clean any residues and take care of rashes. Finally, the new diaper can be brought in, baby wrapped, and the double sided tapes on it lightly pressed on to secure it.
The entire machine would probably cost less than 10 million USD per unit if mass produced at reasonable scales, and most technological elements needed in such machines would be readily available.
> It would require an entirely different kind of robotics.
I was 100% with you until suddenly this technical claim pops out. You might feel this way, and might be right, but why? Changing a diaper is crazy hard, I absolutely agree, but you seem to be just declaring from vibes that we 'require an entirely different kind of robotics'. Can you put your finger on why this is true?
Not nitpicking for the fun of it - I'm genuinely interested. Robot person.
Zero failure rates are a very different bar. Software must be treated as actively malicious from a hardware standpoint. So it comes down to designing hardware capable of the task that’s also incapable of causing harm.
Meanwhile it must also be strong enough to move and restrain a range of infants which is a level of force capable of harm without any possibility to fail deadly. Nuclear power plants are an incredibly simple problem by comparison yet zero possibility of harm isn’t the standard because it’s so unrealistic.
The main limitation right now is that robotics are very limited in their sense of touch.
After that, they are limited in their understanding of physics. After that, perhaps understanding of physics and physiology would come into play - but perhaps superhuman perception and reaction time could reduce the need for intuitive understanding physics and physiology.
I think it needs a water gun. If the diaper was a spray on layered rubber, like a sponge then an impermeable layer, and then you sprayed a solvent to clear the old diaper and poop and then spray on a new one. You'd just need to slot them into styrups briefly or some socks on strings to move the legs into a good position.
But can this be done with baby skin and lung safe chemicals at a reasonable temperature?
Point being humanoid designs for robots that manipulate objects designed for humans are an artificially hard problem we have decided to fail at solving.
Well, Mr Robot person, would you let today's robotics change your clothes right now? If you wouldn't, then why would you allow it any where near a baby? If you would, why? What robot with what tech would you allow?
This is a great one. The manipulation is hard, but we're probably on a trajectory to be able to do it in 1-3 years if you were tolerant of some risk to the baby, but, of course, your tolerance for injuring babies is basically zero. I think 'risk & reliability' is a good potential category: there is the bar of 'got it to do a task reliably enough that we got a video' and the bar of 'got it to do a task reliably enough that I'd risk an infant in its grippers.
> but we're probably on a trajectory to be able to do it in 1-3 years
This is wildly optimistic. I quit working in robotics because I got tired of all the bullshit promises everybody made all the time. I'm not saying robotics isn't advancing or the work is unimportant, but the spokespeople are about as reliable as Musk when it comes to timelines.
I doubt it will happen in 10 years, even with a constrained environment and hardware that costs well into 6 digits.
I think GP was basically talking about doing it on a doll. As in, a robot in 1-3 years might be able to change diapers with occasional success, but half the tries will result in a dismembered diaper user: we'd use dolls in this scenario, since dismembering babies is taboo and generally frowned upon within the robotics community.
> your tolerance for injuring babies is basically zero.
Um, no it's not. Is absolutely zero tolerance. There is not weasel words out of this. If a robot was to cause any pain to the baby, there would be no remorse. There would be no front of mind thoughts to not repeat the same thing the next time. There would be no guilt for causing pain to the baby.
Why you would "basically" this the way you have is disturbing.
Sorry, this is me communicating like an engineer. In a technical sense risk of anything can only approach zero: never actually get there. I meant that there should be essentially zero chance, similar to holding a baby in your arms or putting it in a high chair, and probably less chance of injury than driving in a car with a baby in a car-seat. Basically zero.
I don't think the parent comment advocates for hurting babies. It just, probably correctly, states that cherry picked examples won't be representative of roboty safety with infants in the next years, but that true safety will improve over time as well.
real world treatment of babies is very different from the zero tolerance you've described. From pregnant mothers smoking/drinking to medical care unavailability to doctor errors to various toxin contaminated baby products and the environment (Flint leaded water comes to mind) to babies left in hot cars and other abuse to poor availability of daycare (even less availability of daycare good for mental development) to ...
Granted most of this is unintentional. The same about injuries by robots - we're supposedly talking about unintentional injuries here. So, if robots save money/time/effort (like Flint water switch) i'm not sure that the society would suddenly change its current approach to unintentional baby injuries and implement zero tolerance.
To illustrate - Uber self-driving killed a woman, and another self-driving maimed a woman in SF. Uber case was an obvious criminal gross negligence running with explicitly disabled emergency braking), and the company wiggled out of it in part by having to shut down the self-driving. Where is in SF it was an obvious case of technology limitations and teething issue, so there were no real severe consequences as we're much more tolerant to honest technological accidents (at least when they happen not to us personally).
Why? There's nothing particularly special about this problem. I would bet a year for an alpha version, and production version in 5 years. We are not exactly limited by mechanical engineering here, there's nothing particularly unique about the human hand that can't be replicated. Tele operated surgical robotics have been a thing for decades. Give it a few months for the multimodal robotic VLM/LAMs to catch up. In many ways this particular problem is a lot more well defined than e.g. self driving cars.
> there's nothing particularly unique about the human hand that can't be replicated
Humanity is far from replicating / matching performance of human hand.
> a baby some robot neglects to prevent from falling off the changing table
that is when we think about 2 handed robots. 6 handed robot can easily have 2-3 hands assigned to tightly keeping the baby. Humanoid robots are handicapped by their similarity to humans which is really an artificial constraint. After all we aren't building airplanes using birds as the blueprint.
On the similar note - while not about baby, was just rewatching an early Bing Bang Theory season with this episode where Howard "falls right into the mechanical hand"
won't the baby feel dis-abled by only having two arms?
On the other hand - the baby will from the beginning develop an instinct to keep track of 6 hands flying around instead of just 2. Will help in future street fights :)
In general, looking at the AI coding agents i think we all either already feel or soon will feel disabled. And honestly i think human race with its perception of itself as the "top of the Creation" is due for a modesty lesson to help speed up the evolution. We're spending tremendous resources unproductively, be it wars or just ineffective economies, etc. We don't feel the urge to develop our civilization and to evolve ourselves in all aspects - from mental and biological to cyber-integration. The Mother Nature doesn't like such relaxed species.
Platinum Medal: Complete a gold medal task in the presence of multiple children between the ages of 3 and 8. The children must remain safe throughout (and after) the entire process.
Interesting post and great reference to [1] about why laundry hits a sweet spot of capability.
Interestingly the repeated critiques in the article are about sensor richness: primarily force feedback and tactility, which indicates lacking hardware. Software only robotics has a long and fraught history, but it really feels to me that current industrial hardware could be driven more intelligently without much change. No doubt the "ideal" robot for any given task requires developments in both.
I'm also curious about safety, since generally capable mechanisms need a multilayered safety stack that includes semantics, and cobot certification is likely not enough anymore. Examples: feeding someone the wrong pill, pouring a glass of water into electronics, cutting vegetables vs fingers.
[1] https://substack.com/redirect/82d94852-76b6-4b0d-8595-86e46a...
Hey Hacker News. Curious if folks feel like I'm missing categories of challenge.
I'd go for something in the manipulation of ropes or wires.
State of the art seems to be that they can untangle a loosely knotted cord.
Untying a short rope with a tightly pulled overhand knot in the middle seems like it's decades away. You have to be able to grip it well enough, then twist the rope and push (even though every physicist says pushing a rope is impossible).
Interesting. Futurism is super hard, but "decades" too far away to me. I think with strong 2 finger grippers this is probably close to state of the art, especially with a wrist force sensor, like the TRI setup.
Belaying a climber would be a hilarious (and fraught) gold medal challenge.
There are already robots that do that (autobelayers)...
Not really.
Auto belay device can't hold the climber up on the wall, like human belayer can, so that climber can rest and try again.
Auto belay device can't do lead belay.
Making a bed.
You need to manipulate a large sheet, and you probably need to move around, bend down and lean over to reach all the corners. Bonus points for neat hospital corners on a flat sheet.
Putting pillows in pillowcases is another fun one. Usually pretty easy, probably a bronze medal.
Gold medal: put a UK super king size duvet inside a duvet cover. It's huge and awkward, there are buttons, and it's almost but not quite square (why??) so there's a good chance you'll get it round the wrong way and have to rotate it 90 degrees.
Something I still struggle with as a human: Cracking an egg into a bowl. No making a mess and no egg shell in the bowl.
Gold Medal: Separate the egg white from the yolk!
Standard evening family home tidy/reset - toys, books, clothes, shoes away in their places. All over the house.
Oh, and load/unload dishwasher. Same with laundry machines. Along with folding laundry, these are the domestic robot equivalents of 'de-mining' and 'search and rescue': the classic motivating use cases for mobile autonomous robots.
I like these benchmarks and the videos are funny!
Consider examples using building tools like screwing in a drywall screw, or hammering a nail, using a paint roller, caulking a sink, minor plumbing repair with a torch and solder. These differ enough in terms of forces, state changes, and combined dexterity/acuity (two-handed proprioception) from the windex, sandwich and key examples
Ikea product assembly for gold medal.
Maybe careful application of large amounts of force? Opening a jar, peeling garlic, splitting a squash, opening a soda can. This category seems like a good test of "grip" strength + force feedback + sense of touch.
How about something with unpacking items from a shopping bag, i suspect the difference in bags (standard plastic, reusable etc) and certain items can really crank the difficulty.
It can also create a good time of a story - open the door to get the grocery delivery, unpack the delivery etc.
replacing an inner tube on bicycle wheel - strong forces / both arms / level use / dexterity / problem solving
This one is really hard for me...
something requiring navigating stairs while holding something full like a laundry basket. bronze - straight stairs, silver - one 90deg turn. gold - spiral.
something requiring co-ordination between 2 robots. think relay race which the olympics has. So say, moving a couch together.
btw love the idea and the silver body suit. good stuff.
Ooh. I like full body manipulation. Humans use hips & elbows to move laundry baskets. Two robot collaboration is good too. I wonder who I can convince to wear another silver suit.... :)
Robots currently do things easy and faster to do for robot but difficult to do for man.
I love your list and it makes me think we are so far away from these things ever being feasible/cost effective compared to just hiring a poor person to do it. And the world is making a lot of poor people right now.
threading a needle (different thread thickness and hole sizes)
Carving a wooden spoon in hand.
(HN link on Substack points at empty page instead of this one, at least before I made this comment.)
What I think is missing is marathon events. Biathalons and Triathalons.
We all know LLMs have a rather limited context window. Thus seeing robots do longer chains of events would be interesting to see that they're capable than a possibly rigged demo.
Something like: move a stack of boxes from one room to another. The boxes at the end also need to be stacked up. or how about pick up a box, go up some stairs, open a door, and put the box on a shelf on the other side.
Also, the real world is sloppy and messy and dirty and, to be real, kinda janky sometimes. Gold for unlocking a door with a key at a well-maintained office complex, (and opening it, and walking through it) is one thing, because facilities is going to replace the lock before it gets old and needs replacing, and we can assume the door fits in the frame properly so it doesn't need to be shoved or lifted up or yanked in order to be opened is easier than. But the real world is messy and sloppy and you gotta jiggle the key in just the right way in order to get it to work.
Closing the door (assuming the robots weren't raised in a robot barn) is also harder than it looks if the door is shitty and needs a proper slam in order to be fully closed. Also, the robot locking the door behind itself after it comes in. Scanning a key card and opening a door, but the first try fails.
We're a long way from a general robot that can screw a simple screw together like you would to assemble Ikea furniture.
Object recognition.
Gather only the dishes from a messy coffee table and put them in the dish bin.
Pick up only the clothes from a messy floor and bed, and put them in the hamper.
Dump a hamper of clothes onto a table, and sort out stuff that doesn't want to go into the washing machine.
Terrain traversal.
Just walk 500 ft, but theres increasing levels of obstacles in the way.
We all saw Boston dynamics robot parkour videos, but what I want to see is a robot make it from the front door of Simpsons house to the kitchen in the back, but it's got to go through the living room, but it's hella messy, with Maggie and Bart and Lisa’s crap strewn all over, Homer’s got some beer bottles, some empty, some full, all over the floor and on the table, and all the robot has to do is walk from one side of the room to the far side of the room without stepping on anything, or knocking anything over. (Simpsons merely being a home layout that's familiar to most people. Doesn't need to actually be them.)
Ducking under a low ceiling. Climb over a barrier, of varying shapes and sizes.
Other loocomotion. how much weight in its arms in front of it, holding a 5-lb briefcase with one hand while walking. Can it carry something on its back? What's the limit? Can it give piggyback rides?
A category for simulated. Let companies show off their robot's kinematics control systems, so have something on the level of CoppeliaSim, so the motors and the gears and the actuators are themselves simulated, vs a simple 3d video game where they are not. Plug their model into the simulated robot and see how well it just walks. If we remember QWOP, it's harder than it looks!
Obviously it's not going to be totally 100% accurate to the real world. The benefit of this is it lets people complete from all over world without having to replicate a very specific setup in the physical world, and compete from wherever they live am not have to fly to your facility to test, opening up a whole new world of contestants because they can now compete because they can afford it now.
At the end of the day, the most important challenge is, can it pick up a battery from the shelf, swap it with one of the two in its chassis, and put the dead one it just pulled out onto the charger?
Can I say something: could you all please stop inserting contaminated tool back into a jar of food? You use a clean tool to take out the amount you eat, and that's it.
You can put it back if the tool had touched nothing but: air, the food in the jar, and your hand at the operating end. Otherwise, that butter knife stays on your dish for the rest of the meal. The exception would be if you cleaned the tool, like bare minimum by wiping with a brand new piece of tissue paper(but that's kind of wasteful).
Is that an outrageous ask? I know it's probably not a huge deal, like free water and such, and my techniques are that of total amateur being never professionally involved in medical and/or bio science fields, but just, how can you stand possible breadcrumbs IN THE JAR!?
> but just, how can you stand possible breadcrumbs IN THE JAR!?
Bacteria can do a number on people... Kept telling the wife to not cross contaminate jars, and other products. But you know, she knows better...
Wife was eating some fish in can, puts spoon back (with lovely saliva bacteria), puts in fridge "i wil eat tomorrow", 2 days later she eats the leftover.
She enjoyed a hour+ of "fun" muscles contracting stomach cramps to the point it was almost hospital time. Learned her lesson, well, ... for fish.
Its always like "but i do not like to keep using fresh utensils". I am always: "the dishwasher does not care if it 20 or 40 utensils". Its the same amount of wash and cost. So stop trying to recycle utensils!
Think about how fast progress is being made now. When I was a kid in the early 2000s we would see some basic robot progress on movements (almost always from Boston Dynamics or sometimes China) and we thought it was incredible. Robot dogs running was amazing and five or so years later a backflip blew our minds. Those robots were specially designed and didn't look humanoid. Now we have bi-pedal humanoid robots and they walk and move fairly capably - even able to get up after falls. Now within the last year I have seen them learn Kung Fu, become really fast at getting up, become resistant to being knocked down by quite a lot of force, and now even doing tasks like those shown here.
Just imagine 2050 if the progress continues at this rate. I am both excited and really scared.
Almost all progress at doing tasks reliably has been made with 2 massive caveats:
1. Force. Walking, running, fighting, doing backflips, etc. all allow for large amounts of force, without a lot of dynamic precision required. Many common tasks require precise and dynamic force. E.g. for washing a window, pushing too hard breaks the glass while pushing too softly will leave streaks.
2. Environment interaction. Most reliable humanoid robots do minimal environment interaction beyond self-balancing. They walk/run/jump in environments that are largely open, with usually convex blocky obstacles. The real world has lots of tasks that require processing beyond low-resolution maps of solid/open space. E.g. I'd want to see a robot that can walk through a forest: jumping/stepping over thin branches that are hard to see, ducking under fallen logs, pushing though bendy branches without breaking them, avoiding ground that is muddy, and seeing through the current obstacle to determine if the obstacle beyond is traversable.
Just to reiterate, I don't see fast progress being made on doing these tasks reliably. It's easy to show 1/N success rate, and much much harder to show ~N/N success rate on these dynamic tasks.
I don't know, for me it looks like the demo robot already does it quite well.
Great post! Now somebody with the connections just needs to make it happen. For event five, slippery when wet, you should definitely include drying your hands on a towel, as it serves an important hygienic function.
This is amazing!!
Rather than teaching your robot to fold inside-out clothes, you should teach it to attack people who put their clothes in the hamper inside out.
I think my shirts just automatically get inside-outted in my washer/dryer? I certainly don’t put them in that way, and it seems like I spend a lot of righting them when putting away laundry.
Try putting some in inside-out and seeing if they get turned outside-out.
I’m also curious which device does it!
We should work this out, and also we will need a lot of robots to go after the manufacturer.
I agree, but in some cases it makes sense to protect the print. Although I habe no idea if it actually helps.
If it's vinyl applied with heat (numbers on a jersey as an example), they recommend turning inside out. That way, if it gets too hot in the dryer, it only sticks to itself instead of to other garments. Losing the one garment is better than multiple.
Clothe instruction care sometimes requires washing clothes inside out.
I've seen suggestions that having your clothes inside out in the wash helps them get cleaner. And if it keeps the AIs under control, you know, benefits.
What are your thoughts on this? Where do we stand?
https://substack.com/@isitpropaganda/note/c-167073531?utm_so...