At 2:54, it struggles to pick up the cloth for 10 seconds (100 seconds real-time).
This may just be a software fix, but I wonder about the idea of exchanging tools for different tasks. In this case some kind of pincher-vacuum or roller-grip might have done the job better.
Picking up cloth with a robot remains firmly in the “unsolved hard problems” bucket. Use that to gauge the believability of industry heads predicting the timeline of “robots in every home”.
I’m not even particular skilled at laundry but I can easily manipulate clothes in complex ways at speed. I can use a sudden flick to turn things inside out, or flat-fold a mattress cover.
I suspect we’re at least five years away from those rather ordinary capabilities in robots.
> I suspect we’re at least five years away from those rather ordinary capabilities in robots.
Maybe. Here's a robot at Berkeley folding towels in 2010.[1] A Willow Garage robot folding jeans in 2012.[2]
Foldimate in 2017.[3] Even boring old Chicago Dryer had this working by 2021.[4]
They're all really slow. That's because they have no understanding of dynamics. The item has to come to a full stop between operations. Chicago Dryer got past that problem with a sequence of steps at different stations, each station taking about one second. That yields a useful commercial machine for large laundries.
It's just a demo problem, though. The approach is interesting. They're trying to use LLM technology on a completely different kind of problem. For that, you need a lot of training data.
If you're going to do things in this way, you need data from the inside of doing it. That's hard to acquire, but not impossible. They claim to use "robotic training data". Not sure if this is from robots being operated by humans as teleoperators. Others have tried that. There's a Stanford project that looks very similar.
Something like this has been used to train quadrotor drone controllers.
There's no obvious cheap way to acquire lots of data of this type, though. You have to run your own experimental setup and log.
Motion tracking from vision on a squirrel colony would be interesting as a data source. Squirrels are very agile and easy to observe. Then run the skeleton movement data back through a simulator and try to extract the forces the muscles are exerting. Now you have something usable for training an agile robot. Maybe sports videos could be used for training.
I agree, five years is a decent estimate. But think how crazy that is! We're talking about capabilities that were firmly in the realm of far-off sci-fi three years ago. Capabilities that could revolutionize the market for all physical labor. Only five years away?
I wonder at the long term vision for humanity. We have AI replacing a lot of art, writing, coding, etc. We have a bunch of robotics companies racing to replace physical human labor. Waymo and Tesla replacing drivers.
What role do the majority of people realistically play in this world?
There is a lot of undone labor in the world. In developing countries the middle class has drivers, cooks, housekeepers. That’s only possible due to inequality. With automation we can all get that.
These people with tons of help by and large live fulfilled lives. You find fulfillment in family, friendships, and non necessary creation (art, research, etc); whatever makes you happy.
But most of all, the Industrial Revolution made people think we’d all be idle and nothing can be further from those predictions. Many more people, and many more jobs, and most of the world still lives in relative poverty and various forms of insecurity and unmet material and labor needs.
Finally there are a lot of problems we have (thousands of health conditions, the environment, autocrats) that will prob take centuries to tackle even with ai, robotics, and being freed up from menial labor.
As optimistic as your comment is, the fact that there are lots of problems does not mean that we will tackle them. In my opinion, if we don't aim at doing anything about it, the gap between the rich and the poor will widen. Both between societies, and within one society. I'm now in Canada, but in my childhood country, most of the recent "smart" (meaning connected) devices and the recent AI models are not available. This is starts a viscious cycle that makes things worse and worse. For the less connected high teck devices, the ratio of the price (That's set based on supply/demand in the richest countries) to income (that's damanged by sanctions and general government stupidity) is getting so high that it's really hard to get high-end devices.
As the labour required to produce goods and services is automated, one possible scenario is that fewer and fewer people will stay "relevant", while the rest will sink and become invisible.
Things can be avoided, but looking at countries that have been unable/unwilling to ensure housing (as one of the 3 most fundamental material needs of the human: food, housing, clothing) stays affordable, does not raise hope. In my opinion, the housing problem is extremely easy to solve when looking at the problem as a technical one, and impossible when you include the way economic incentives are working at it.
I hope I'm wrong, but when I project the current path into future, it's not bright.
Is our goal to reduce the gap between the rich and the poor, or to improve the lives of the poor? Because the gap itself is actually irrelevant to poor people's lives. Rich people's improvement outpacing poor people's is not necessarily an issue.
Second, a financial gap does not necessarily translate to a material gap. Someone with 1,000,000x someone else's net worth still buys the same iPhone and drinks the same coca cola. Many important wellbeing factors are not actually blocked by finance, like healthcare and education. Even if you take all the rich people's money and repurpose to education not much will change. Maybe an iPad for every kid, but what good does that do?
Housing is actually a great example. Real estate has a way of sucking up entire GDPs worth of money. As a country you can't pay your way out of housing problems. Look at something like China which has been consistently overbuilding housing for decades now. They still have housing issues.
People in the fashion industry do the same thing, basically, that architects and car body designers and other product designers do: they design the clothes that people wear. Apparently, people don't want to just keep wearing the same designs that were popular in 1950 or 1850 or even 10 years ago, so clothing designers create new designs. Many of them go nowhere, but some are popular are sell well. Over time, this results in fashion trends changing, which is why you can look at photographs of people in the 1980s or 2000s and see that their clothing styles are noticeably different from now. Of course, this generally seems to be more noticeable to women than men as they generally have a greater interest in being fashionable (though plenty of men do too, though probably far, far less among the crowd that frequents HN).
Basic universal income makes sense to me. I imagine a society where everyone is free to create art or relax in hammocks all day. A basic universal income would not be enough to fund world travel, your fav consumer items, or ambitious projects, so I don't foresee it causing an intellectual meltdown in society as some fear-monger (As an aside: I speculate people afraid of this may likely be the actual lazy members of our current society :-).
If everybody had their basic needs covered, that should actually lead to more prosperity and reduced crime, leading to more people being able to produce superior knowledge, art and enterprises of all sorts. To make science or art or whatnot, you first need to be able eat!
The question of whether Silicon Valley's "AI luminaries" are genuinely pursuing this utopia or have a more selfish hidden agenda is another matter entirely.
There are many jobs where people prefer other people to fill those roles, irrespective of the ability of machines. Most people don't want to watch computers play chess, or spend money on computer-generated art, or go to a robot therapist.
Ah, you say: But such jobs don't employ many people! Most people do things that nobody cares if they're automated away. Surely we can't all be chess players or artists?
To which I say: The job market will adapt, and people will move into those jobs where customers prefer to have a human. We have no real idea what those jobs are today, but some of them might be the things you wish you had more of, but are too expensive for most of us to hire someone for. (Interior decorator? Personal chef? ...)
Imagine these kind of robots in the home of of 2 or more kids. Roomba doing vacuuming, this robot doing laundry and folding.
Parents would be spending quality time with her kids, helping them with their homework or helping with their practice - sports or music, instead of getting frustrated looking pile of laundry and kids don't have nothing to wear.
Kids now have more questions due to quality engagement. So they would visit library or if they are into sports, parents spend more time with them.
Automation has always been there. We just pick up things we didn't get a chance to pick up. We travelled on cars when horses were no more needed. We built bigger and better things, when we don't have to make our own hammer. Also, we created more problems from these and needed more innovation to fix them.
We always worked around 40 hrs a week since time immemorial. So, we will continue to work 40hrs.
People who want to spend time with their kids make time to spend with their kids. Those who don't, don't. I don't think household chores are the blocker here.
This is a bad example I think: Roomba seems to be dying, as other competitors are making vacuum robots that look similar yet are technically far superior. It's almost like talking about spreadsheets and giving Lotus as the canonical example.
I was down voted before for asking a similar question, I have no idea what the plan is but I struggle to understand what the future looks like when we literally have nothing to do. Why would I even bother with a hobby when a robot can do everything 10x better?
Maybe just "enjoy nature" would be the best bet if we survive the robot wars.
> Why would I even bother with a hobby when a robot can do everything 10x better?
Because you enjoy doing it. It is about the journey, not the destination. It always was and will be.
> what the future looks like when we literally have nothing to do
Human life is about finding meaning. Go to a book club, learn sailing, dance at a beach, practice blacksmithing, learn to draw the best circle you can freehand, give a trully world class massage. Just ideas from the top of my head. I’m sure you can come up with even better ones.
What do you do now when there is nothing you have to do? Are all hobbyists doing it to be "better" than someone else?
If I get to choose between the status quo, where I have to work for 30+ years to have a chance at an uncertain retirement, or spending the rest of my life exploring the question of what to do with my time, I know which I'd pick.
Society is imposing the 30+ year working requirement on us. It's not really required nor as it been for basically ever. We don't need advanced robots to stop that madness.
I don't get this feeling... wouldn't you want to play soccer like Messi? Or play guitar like Hendrix? (etc..).
There can be pleasure in being a spectator, but being a performer, at least to me, is 1000 times more fulfilling. I don't care if someone else can do it, even a robot... I want to do that myself! A society with more space for personal ambitions and less need to hustle for food sounds great to me.
I know all this, it's basic common sense, but when you REALLY think about what you're saying, it's still strange.
Likely a humanoid robot will be able to play guitar 1000x better than hendrix, then what's the point in you becoming as good as Hendrix, so you can play guitar for yourself in your basement?
The difference between today and the future is, today there are still things humans can do better than machines and robots, when there isn't, it will be weird.
I do, but I also enjoy the fact I'm making something, what is the point of making something when I can have something else make it. It's similar to deciding to dig up my front yard by hand to build a new garden rather than use a machine. You can do it, but you know it's pointless. Now imagine when EVERYTHING is like that?
Inbetween the current world full of labor scarcity, and the philosophical dilemma "what do I even do" post-scarcity utopia, is a world similar to our current one with much less labor scarcity and much more quality of life. That's what we're aiming for right now. What comes afterwards we can worry about then.
Hopefully we transition into a post-work society. Socialist countries will stick the landing, while the bottom of the American society plunges more into poverty. It's never too late to stop voting for people who despise you, and think that "temporary hardships" are necessary as they plan to cut government spending.
I'm 99% sure that old ideas of eugenics will crop up massively (together with a new strain of pro-colonial history-denialism in the "truth-spouting" right), and a new age of genocidal wars with robots will take place for taking over material resources.
We under estimate how much of "Western morality" has nothing to do with the "goodness of our hearts" (just see the propaganda for wars over the years). Very dark times ahead.
I saw your foundation model is trained on data from several different robots. Is the plan to eventually train a foundation model that can control any robot zero shot? That is, the effect of actuations on video/sensor input is collected and understood in-context and actuations are corrected to yield intended behavior. All in-context. Is this feasible?
More specifically, has your model already exhibited this type of capability, in principle?
Nearly 2 years ago I bet a roboticist $10 that we’d have “sci-fi” robots in 2 years.
Now, we didn’t set good criteria for the bet (it was late at night). However, my personal criteria for “scifi” are twofold:
1. Robots that are able to make peanut butter sandwiches without explicit training
2. Robots able to walk on sand (eg Tatooine)
Based on your current understanding, who won the bet? Also, what kind of physical benchmarks do you associate with “sci-fi robots”?
How does the post-training step work? In the case of t-shirt folding, does a supervisor perform the folding first, many times? Or is the learning interactive, where a supervisor corrects the robot if it does something wrong?
May actually be much more important than LLM products in the long run. I can see how these smart hands operate a car building procedure for me in the backyard, or even print some MCUs with sensors. This is huge, indeed.
At 1:50, the guy gives the robot a glass to pick up and then immediately nopes out of there. Wonder if previous demos resulted in a broken glass haha.
Also at 2:08 the upside-down container gets flipped quickly. I wonder if that was a known limitation of the robot at the time or if the person just had a desire to flip it right-side up (to be polite? haha).
I'm commenting on these tiny details and laughing a lot because I'm not sure I can handle a more serious approach to this. Doesn't it seem like in < 10 years there will be dozens of autonomous, affordable home-robots? Everything is going to change.
One last note, they call this generalist, but each of the examples is quite specific from a macro perspective. Yes the robot can fold maybe any pile of crumpled laundry now and that is generalist compared to previous efforts, but seems like we shouldn't be trying to train bots how to do billions of tasks in specific detail; rather they should learn to learn and take on new tasks they weren't trained for.
> Doesn't it seem like in < 10 years there will be dozens of autonomous, affordable home-robots?
If you buy the hype, sure. I know many startups that have already gone bust working on this. I've also seen lots of similar attempts in laboratories around the world going back well over a decade.
> One last note, they call this generalist, but each of the examples is quite specific from a macro perspective. Yes the robot can fold maybe any pile of crumpled laundry now and that is generalist compared to previous efforts, but it does seem like we shouldn't try to train bots how to do billions of tasks in specific detail; rather they should learn to learn and take on new tasks they weren't trained for.
You are starting to see how difficult the problem is and how limited the solutions are. You're basically saying "let's just give the robots general AI and everything will be so much easier!"
I'm not saying there isn't progress. I'm saying progress is slow relative to the work that needs to be done. I've also worked at enough robotics companies to be skeptical of anything they publish because there is a strong tendency to cherry-pick results. The disconnect between the research papers being published and the reality of the robots at one company I worked at was pretty egregious.
Robots are super cool. Just be skeptical of the hype.
Conversely it's hard until it's not. Quadcopters were hard until now they're a disposable item purchased in bulk.
The point of a model like this is targeting that very notion: that with the right software, and enough computer power, you should be able to learn a pretty wide range of available capability (i.e. humans can do this anyway - we drive, we fly planes, we operate heavy machinery - that's us being the software but it's not clear that you need the whole human to get the effect).
I think it would be super awesome. I hate doing laundry so if someone sold a robot that washed + dry + folded all my laundry, I would spend money on it.
I'm talking about I want to throw my dirty clothes into a basket and it takes care of the rest.
At 2:54, it struggles to pick up the cloth for 10 seconds (100 seconds real-time).
This may just be a software fix, but I wonder about the idea of exchanging tools for different tasks. In this case some kind of pincher-vacuum or roller-grip might have done the job better.
Picking up cloth with a robot remains firmly in the “unsolved hard problems” bucket. Use that to gauge the believability of industry heads predicting the timeline of “robots in every home”.
I’m not even particular skilled at laundry but I can easily manipulate clothes in complex ways at speed. I can use a sudden flick to turn things inside out, or flat-fold a mattress cover.
I suspect we’re at least five years away from those rather ordinary capabilities in robots.
The Tesla Optimus robot can already fold clothes...
With hands, not some purpose-designed tool.
It's slow but the trajectory of improvements is clear and rapid.
> I suspect we’re at least five years away from those rather ordinary capabilities in robots.
Maybe. Here's a robot at Berkeley folding towels in 2010.[1] A Willow Garage robot folding jeans in 2012.[2] Foldimate in 2017.[3] Even boring old Chicago Dryer had this working by 2021.[4]
They're all really slow. That's because they have no understanding of dynamics. The item has to come to a full stop between operations. Chicago Dryer got past that problem with a sequence of steps at different stations, each station taking about one second. That yields a useful commercial machine for large laundries.
It's just a demo problem, though. The approach is interesting. They're trying to use LLM technology on a completely different kind of problem. For that, you need a lot of training data.
If you're going to do things in this way, you need data from the inside of doing it. That's hard to acquire, but not impossible. They claim to use "robotic training data". Not sure if this is from robots being operated by humans as teleoperators. Others have tried that. There's a Stanford project that looks very similar. Something like this has been used to train quadrotor drone controllers. There's no obvious cheap way to acquire lots of data of this type, though. You have to run your own experimental setup and log.
Motion tracking from vision on a squirrel colony would be interesting as a data source. Squirrels are very agile and easy to observe. Then run the skeleton movement data back through a simulator and try to extract the forces the muscles are exerting. Now you have something usable for training an agile robot. Maybe sports videos could be used for training.
[1] https://people.eecs.berkeley.edu/~pabbeel/papers/Maitin-Shep...
[2] https://www.youtube.com/watch?v=MOtbcYE4Z4o
[3] https://www.youtube.com/watch?v=C76osXtpLeM
[4] https://www.youtube.com/watch?v=YpTuwKu5fY0
I agree, five years is a decent estimate. But think how crazy that is! We're talking about capabilities that were firmly in the realm of far-off sci-fi three years ago. Capabilities that could revolutionize the market for all physical labor. Only five years away?
I wonder at the long term vision for humanity. We have AI replacing a lot of art, writing, coding, etc. We have a bunch of robotics companies racing to replace physical human labor. Waymo and Tesla replacing drivers.
What role do the majority of people realistically play in this world?
My thoughts are a few:
There is a lot of undone labor in the world. In developing countries the middle class has drivers, cooks, housekeepers. That’s only possible due to inequality. With automation we can all get that.
These people with tons of help by and large live fulfilled lives. You find fulfillment in family, friendships, and non necessary creation (art, research, etc); whatever makes you happy.
But most of all, the Industrial Revolution made people think we’d all be idle and nothing can be further from those predictions. Many more people, and many more jobs, and most of the world still lives in relative poverty and various forms of insecurity and unmet material and labor needs.
Finally there are a lot of problems we have (thousands of health conditions, the environment, autocrats) that will prob take centuries to tackle even with ai, robotics, and being freed up from menial labor.
As optimistic as your comment is, the fact that there are lots of problems does not mean that we will tackle them. In my opinion, if we don't aim at doing anything about it, the gap between the rich and the poor will widen. Both between societies, and within one society. I'm now in Canada, but in my childhood country, most of the recent "smart" (meaning connected) devices and the recent AI models are not available. This is starts a viscious cycle that makes things worse and worse. For the less connected high teck devices, the ratio of the price (That's set based on supply/demand in the richest countries) to income (that's damanged by sanctions and general government stupidity) is getting so high that it's really hard to get high-end devices.
As the labour required to produce goods and services is automated, one possible scenario is that fewer and fewer people will stay "relevant", while the rest will sink and become invisible.
Things can be avoided, but looking at countries that have been unable/unwilling to ensure housing (as one of the 3 most fundamental material needs of the human: food, housing, clothing) stays affordable, does not raise hope. In my opinion, the housing problem is extremely easy to solve when looking at the problem as a technical one, and impossible when you include the way economic incentives are working at it.
I hope I'm wrong, but when I project the current path into future, it's not bright.
Is our goal to reduce the gap between the rich and the poor, or to improve the lives of the poor? Because the gap itself is actually irrelevant to poor people's lives. Rich people's improvement outpacing poor people's is not necessarily an issue.
Second, a financial gap does not necessarily translate to a material gap. Someone with 1,000,000x someone else's net worth still buys the same iPhone and drinks the same coca cola. Many important wellbeing factors are not actually blocked by finance, like healthcare and education. Even if you take all the rich people's money and repurpose to education not much will change. Maybe an iPad for every kid, but what good does that do?
Housing is actually a great example. Real estate has a way of sucking up entire GDPs worth of money. As a country you can't pay your way out of housing problems. Look at something like China which has been consistently overbuilding housing for decades now. They still have housing issues.
I share all your concerns. I considered mentioning them but figured I should keep my reply focused on idea of freeing people up isn’t inherently bad.
In particular zero sum resources like land ownership will be an increased challenge.
And our governments have been slow to respond to things like climate change and we could be slow to respond here.
No health problems if there's no people to have them.
What roles did monks praying play in this World?
What role do people in the Fashion industry(for ex.) play in this World?
It's all a bunch of made up stories.
We'll make up other stories.
People in the fashion industry do the same thing, basically, that architects and car body designers and other product designers do: they design the clothes that people wear. Apparently, people don't want to just keep wearing the same designs that were popular in 1950 or 1850 or even 10 years ago, so clothing designers create new designs. Many of them go nowhere, but some are popular are sell well. Over time, this results in fashion trends changing, which is why you can look at photographs of people in the 1980s or 2000s and see that their clothing styles are noticeably different from now. Of course, this generally seems to be more noticeable to women than men as they generally have a greater interest in being fashionable (though plenty of men do too, though probably far, far less among the crowd that frequents HN).
Basic universal income makes sense to me. I imagine a society where everyone is free to create art or relax in hammocks all day. A basic universal income would not be enough to fund world travel, your fav consumer items, or ambitious projects, so I don't foresee it causing an intellectual meltdown in society as some fear-monger (As an aside: I speculate people afraid of this may likely be the actual lazy members of our current society :-).
If everybody had their basic needs covered, that should actually lead to more prosperity and reduced crime, leading to more people being able to produce superior knowledge, art and enterprises of all sorts. To make science or art or whatnot, you first need to be able eat!
The question of whether Silicon Valley's "AI luminaries" are genuinely pursuing this utopia or have a more selfish hidden agenda is another matter entirely.
There are many jobs where people prefer other people to fill those roles, irrespective of the ability of machines. Most people don't want to watch computers play chess, or spend money on computer-generated art, or go to a robot therapist.
Ah, you say: But such jobs don't employ many people! Most people do things that nobody cares if they're automated away. Surely we can't all be chess players or artists?
To which I say: The job market will adapt, and people will move into those jobs where customers prefer to have a human. We have no real idea what those jobs are today, but some of them might be the things you wish you had more of, but are too expensive for most of us to hire someone for. (Interior decorator? Personal chef? ...)
Imagine these kind of robots in the home of of 2 or more kids. Roomba doing vacuuming, this robot doing laundry and folding.
Parents would be spending quality time with her kids, helping them with their homework or helping with their practice - sports or music, instead of getting frustrated looking pile of laundry and kids don't have nothing to wear.
Kids now have more questions due to quality engagement. So they would visit library or if they are into sports, parents spend more time with them.
Automation has always been there. We just pick up things we didn't get a chance to pick up. We travelled on cars when horses were no more needed. We built bigger and better things, when we don't have to make our own hammer. Also, we created more problems from these and needed more innovation to fix them.
We always worked around 40 hrs a week since time immemorial. So, we will continue to work 40hrs.
People who want to spend time with their kids make time to spend with their kids. Those who don't, don't. I don't think household chores are the blocker here.
I find household chores to be a welcome break from paying constant attention to our children. They are exhausting. We deserve some "me time".
>Roomba doing vacuuming
This is a bad example I think: Roomba seems to be dying, as other competitors are making vacuum robots that look similar yet are technically far superior. It's almost like talking about spreadsheets and giving Lotus as the canonical example.
More likely, parents are going to be working while robots pick kids up from school, read them some books, assist with homework....
In countries where labour is cheap enough relative to the professional class (eg Singapore) humans are currently hired to perform these tasks.
I was down voted before for asking a similar question, I have no idea what the plan is but I struggle to understand what the future looks like when we literally have nothing to do. Why would I even bother with a hobby when a robot can do everything 10x better?
Maybe just "enjoy nature" would be the best bet if we survive the robot wars.
> Why would I even bother with a hobby when a robot can do everything 10x better?
Because you enjoy doing it. It is about the journey, not the destination. It always was and will be.
> what the future looks like when we literally have nothing to do
Human life is about finding meaning. Go to a book club, learn sailing, dance at a beach, practice blacksmithing, learn to draw the best circle you can freehand, give a trully world class massage. Just ideas from the top of my head. I’m sure you can come up with even better ones.
What do you do now when there is nothing you have to do? Are all hobbyists doing it to be "better" than someone else?
If I get to choose between the status quo, where I have to work for 30+ years to have a chance at an uncertain retirement, or spending the rest of my life exploring the question of what to do with my time, I know which I'd pick.
Society is imposing the 30+ year working requirement on us. It's not really required nor as it been for basically ever. We don't need advanced robots to stop that madness.
I don't get this feeling... wouldn't you want to play soccer like Messi? Or play guitar like Hendrix? (etc..).
There can be pleasure in being a spectator, but being a performer, at least to me, is 1000 times more fulfilling. I don't care if someone else can do it, even a robot... I want to do that myself! A society with more space for personal ambitions and less need to hustle for food sounds great to me.
I know all this, it's basic common sense, but when you REALLY think about what you're saying, it's still strange.
Likely a humanoid robot will be able to play guitar 1000x better than hendrix, then what's the point in you becoming as good as Hendrix, so you can play guitar for yourself in your basement?
The difference between today and the future is, today there are still things humans can do better than machines and robots, when there isn't, it will be weird.
Can't you just enjoy things you like doing? Not everything needs to be a grind or a side hustle.
I do, but I also enjoy the fact I'm making something, what is the point of making something when I can have something else make it. It's similar to deciding to dig up my front yard by hand to build a new garden rather than use a machine. You can do it, but you know it's pointless. Now imagine when EVERYTHING is like that?
It will be strange.
Inbetween the current world full of labor scarcity, and the philosophical dilemma "what do I even do" post-scarcity utopia, is a world similar to our current one with much less labor scarcity and much more quality of life. That's what we're aiming for right now. What comes afterwards we can worry about then.
You can pick up any hobby now and there'll be someone who's 10x better then you at it.
Hopefully we transition into a post-work society. Socialist countries will stick the landing, while the bottom of the American society plunges more into poverty. It's never too late to stop voting for people who despise you, and think that "temporary hardships" are necessary as they plan to cut government spending.
Nothing.
I'm 99% sure that old ideas of eugenics will crop up massively (together with a new strain of pro-colonial history-denialism in the "truth-spouting" right), and a new age of genocidal wars with robots will take place for taking over material resources.
We under estimate how much of "Western morality" has nothing to do with the "goodness of our hearts" (just see the propaganda for wars over the years). Very dark times ahead.
Nothing has changed. We'll still be doing the same things we've been doing for thousands of years. This is nothing new.
(I work at π.)
Happy to answer any questions on the model, hardware, etc
I saw your foundation model is trained on data from several different robots. Is the plan to eventually train a foundation model that can control any robot zero shot? That is, the effect of actuations on video/sensor input is collected and understood in-context and actuations are corrected to yield intended behavior. All in-context. Is this feasible?
More specifically, has your model already exhibited this type of capability, in principle?
Nearly 2 years ago I bet a roboticist $10 that we’d have “sci-fi” robots in 2 years.
Now, we didn’t set good criteria for the bet (it was late at night). However, my personal criteria for “scifi” are twofold: 1. Robots that are able to make peanut butter sandwiches without explicit training 2. Robots able to walk on sand (eg Tatooine)
Based on your current understanding, who won the bet? Also, what kind of physical benchmarks do you associate with “sci-fi robots”?
Coincidentally just saw robots walking on sand today: https://www.youtube.com/watch?v=KRVR0E7AN0A
Is there a web page where we can see bloopers? I want to see the problems you had to solve.
Also, could you please consider adding googly eyes [1] to the robot(s) in future videos?
[1] https://en.wikipedia.org/wiki/Googly_eyes
How does the post-training step work? In the case of t-shirt folding, does a supervisor perform the folding first, many times? Or is the learning interactive, where a supervisor corrects the robot if it does something wrong?
First of all - incredible work. Do you guys plan to integrate frameworks like ROS to help manage this robot?
As a committed AI skeptic, this demo is very impressive. Bravo
This is actually promising. I hope these guys continue to iterate for how much ever time they need to
May actually be much more important than LLM products in the long run. I can see how these smart hands operate a car building procedure for me in the backyard, or even print some MCUs with sensors. This is huge, indeed.
This is a duplicate thread. Can some mod merge them oO? I don't know how this works on HN.
At 1:50, the guy gives the robot a glass to pick up and then immediately nopes out of there. Wonder if previous demos resulted in a broken glass haha.
Also at 2:08 the upside-down container gets flipped quickly. I wonder if that was a known limitation of the robot at the time or if the person just had a desire to flip it right-side up (to be polite? haha).
I'm commenting on these tiny details and laughing a lot because I'm not sure I can handle a more serious approach to this. Doesn't it seem like in < 10 years there will be dozens of autonomous, affordable home-robots? Everything is going to change.
One last note, they call this generalist, but each of the examples is quite specific from a macro perspective. Yes the robot can fold maybe any pile of crumpled laundry now and that is generalist compared to previous efforts, but seems like we shouldn't be trying to train bots how to do billions of tasks in specific detail; rather they should learn to learn and take on new tasks they weren't trained for.
> Doesn't it seem like in < 10 years there will be dozens of autonomous, affordable home-robots?
If you buy the hype, sure. I know many startups that have already gone bust working on this. I've also seen lots of similar attempts in laboratories around the world going back well over a decade.
> One last note, they call this generalist, but each of the examples is quite specific from a macro perspective. Yes the robot can fold maybe any pile of crumpled laundry now and that is generalist compared to previous efforts, but it does seem like we shouldn't try to train bots how to do billions of tasks in specific detail; rather they should learn to learn and take on new tasks they weren't trained for.
You are starting to see how difficult the problem is and how limited the solutions are. You're basically saying "let's just give the robots general AI and everything will be so much easier!"
Idk this is really promising, how many robot foundation models have you seen before that also work very well? I believe this is all quite recent.
I'm not saying there isn't progress. I'm saying progress is slow relative to the work that needs to be done. I've also worked at enough robotics companies to be skeptical of anything they publish because there is a strong tendency to cherry-pick results. The disconnect between the research papers being published and the reality of the robots at one company I worked at was pretty egregious.
Robots are super cool. Just be skeptical of the hype.
Conversely it's hard until it's not. Quadcopters were hard until now they're a disposable item purchased in bulk.
The point of a model like this is targeting that very notion: that with the right software, and enough computer power, you should be able to learn a pretty wide range of available capability (i.e. humans can do this anyway - we drive, we fly planes, we operate heavy machinery - that's us being the software but it's not clear that you need the whole human to get the effect).
I think it would be super awesome. I hate doing laundry so if someone sold a robot that washed + dry + folded all my laundry, I would spend money on it.
I'm talking about I want to throw my dirty clothes into a basket and it takes care of the rest.
The demo from the video gives me hope!
> Doesn't it seem like in < 10 years there will be dozens of autonomous, affordable home-robots?
Yes, and maybe we can even put them in the driver seat of a car ;)