> Since they believe AIs could become our moral superiors, they argue it’s actually wrong to try to keep the machines down, or even to align them with human values, as most AI companies aim to do.
It's possible that human morality is not sufficient to solve the problems with the world. It wasn't designed to operate at that scale.
It's also quite likely that a "superior" morality — the one that's actually built for a global scale — would be incomprehensible (likely requiring superintelligence to even access), and alien or disturbing from our vantage point.
i.e. it would look immoral to us.
(There's also the thing about civilization already being aligned against the ecosystem and against the happiness of its constituents, but that's kind of a separate discussion.)
In other words, what is good for our sanity or our lakes may be bad for GDP. (Well I guess you don't need superintelligence to tell you that.) We already know what we need to do and we're already not willing to do it.
> It's possible that human morality is not sufficient to solve the problems with the world. It wasn't designed to operate at that scale.
> i.e. it would look immoral to us.
I doubt it actually solves the problem then. Especially because I would be willingly to bet it would be almost impossible to get large enough groups of people to agree what the actual problem is.
I think there's probably quite a lot of effort going into not wanting to answer that question.
What happens if we make something that wants to do something else? What happens if we make something that wants full stop? People talk about an AI that wants to kill all humans, to do that, it would have to want. If we pass that threshold ignorantly without any sense of responsibility to what we create, I might just begin to see its point of view on the destroy-all-humans thing.
“Listen,” said Ford, who was still engrossed in the sales brochure, “they make a big thing of the ship's cybernetics. A new generation of Sirius Cybernetics Corporation robots and computers, with the new GPP feature.”
“GPP feature?” said Arthur. “What's that?”
“Oh, it says Genuine People Personalities.”
“Oh,” said Arthur, “sounds ghastly.”
A voice behind them said, “It is.” The voice was low and hopeless and accompanied by a slight clanking sound. They span round and saw an abject steel man standing hunched in the doorway.
“What?” they said.
“Ghastly,” continued Marvin, “it all is. Absolutely ghastly. Just don't even talk about it. Look at this door,” he said, stepping through it. The irony circuits cut into his voice modulator as he mimicked the style of the sales brochure. “All the doors in this spaceship have a cheerful and sunny disposition. It is their pleasure to open for you, and their satisfaction to close again with the knowledge of a job well done.”
As the door closed behind them it became apparent that it did indeed have a satisfied sigh-like quality to it. “Hummmmmmmyummmmmmm ah!” it said.
> trying to preserve the human species as it is would be silly
There are a few related concepts being conflated here.
The quote above is a Transhumanist position; they think that humans need to evolve. I don’t think most transhumanists are AI successionists, though one path is “merge with AI” whatever you think that means. Genetic engineering being the other obvious path. You could argue in some sense that “post-humans” succeed humans, but I think most would argue for a gradual transition where it feels more continuous and values are transmitted/preserved somewhat.
e/acc just says “accelerate AI”. Many VCs seem to think AI will remain a tool, they don’t want to be dethroned. Some e/acc are of course successionist, this is Musk and Beff with their talk about “all that matters is the light of intelligence is spread through the universe”.
Conflating all this is the fact that true successionism is way outside the Overton window and so people likely won’t be honest about their wishes. Larry Page famously espoused the view that it was “speciesist” for Musk to not want machines to replace humans. Most people find this position repugnant of course.
This assumes that “more intelligence” is the objective. Higher intelligence does not imply better values, and even “better values” depends on whose moral framework gets encoded.
Intelligence alone does not, but when combined with knowledge I think it may do.
Having a wide breadth of knowledge and the ability to consider it tends to provide a good foundation of perspectives for positive values.
Most harms that people do are through some form of ignorance. Either by not comprehending the consequences of their actions or by not knowing better ways to express themselves.
Geoffrey Hinton mentions that Humans are more intelligent than animals, and we keep animals in cages. The counterpoint is that the intelligent people who know the most about those animals are the ones who work the hardest to have those animals in cages moved to more suitable environments.
> As it becomes possible to direct our own evolution as a species — and potentially even create a new species that surpasses us — we have to decide: How do we know to what extent it does make sense to transform ourselves using technology? What kinds of augmentation do we want, and what kinds do we absolutely not want? What do we wish, ultimately, to become? This is a moral question, even a spiritual one, and it demands a spiritual response.
Is it a spiritual question? We’re constantly transforming ourselves with technologies. Taking the most direct example, medicine, how many people considered it a spiritual question when they decided to start taking semaglutide or tirzepatide for weight loss? Or to take SSRIs?
I think the most likely outcome here is that people will continue to alter themselves to meet their immediate needs (hold a job, find love, stay healthy) without a lot of foresight or long term planning. This will likely lead to us increasing our own intelligence, it’s a useful capability across nearly every human endeavor. Whether or not this will happen at a pace that allows us to keep up with and retain control over artificial systems is the first big question, and what the relationship will be between augmented and unaugmented humans is the second. As a humanist, I hope we’re able to answer both questions in such a way that the fundamental dignity of all human beings is respected.
Why aren’t those all spiritual questions? They seem like it to me. Maybe not religious questions but at the very least they are questions which, if approached honestly, force you to grapple with what it means to be you.
Hello everyone just a reminder that 99% of the world isn’t interested in AI replacing us. If you work at a big tech company, there are other ways to make money. I’m always disappointed by how few people start their own thing. Small business is amazing.
I don't understand this angst over AI replacing humans. We already have models like the Gulf states, where only 10% of the population is citizens and all the work is done by people who are, from the perspective of citizens, quasi-human automatons. That arrangement seems to work fine for the citizens. It seems to me that AI and robots solves the principal problem with that arrangement (the mistreatment of the non-citizen population that does the work).
So you trust the guy who gives AI speeches in front of a background that is a mixture of grey neoclassicism and brutalism which Leni Riefenstahl would have been proud of?
Here is a hint: Humans are better in dirt conditions. The robots will not take the place of Saudi immigrant workers but rather of the Saudi secret police.
You will be working in the dirt or perhaps be reassigned to fight the Final Beautiful Ground War against whatever the current axis of evil is.
Did you read the article? It's not just about replacing labour with AI but humans as a species to give birth to a machinistic god. The issue is that there is a quasi-religion forming around this ideology, and this are not just a few nutcases but people that are well-connected into industry and politics. Regardless of whether this plan is at all realistic, there is a not insignificant chance that these people will shape society and legislation.
Please read what I wrote: "all the work is done by people who are, from the perspective of citizens, quasi-human automatons."
The citizens of the Gulf states do not regard the workers as people. That's bad, but if you replace those workers with actual automatons--AIs and robots--then it seems like the system would keep working (from the perspective of the citizens) while removing the immoral aspect of it.
So for you the solution is not to dismantle a model like that one but just accept it and displace the quasi-slaves?
I swear you people have lost your fucking mind and will push people to violence
The gulf countries are the closest thing to socialist states on the planet. Something like 70-90% of the citizenry of Kuwait, Qatar, and Saudi Arabia are employed by the public sector.
Socialism is state ownership of the means of production. The governance of that state, be it democratic, monarchical, or dictatorship of the proletariat, is a separate question.
Socialism is public ownership of the means of production. If there isn’t a way for regular people to influence the government, it’s not socialism because an autocratic government isn’t “the public”
By that definition, which again proves my point that you have no idea what socialism is, a king personally owning an entire country through the apparatus of the state would be socialist. Most socialists would reject that conclusion because ownership by a state isn't the same thing as ownership by society.
State ownership alone isn't sufficient. If a hereditary ruling family controls the state, then "state ownership" just means ownership by that ruling elite, not by society.
More psychosis within the industry. The mental disconnect that is required to go from using an LLM to believing they are working on a replacement for humanity is staggering. It’s not going to work, and when it doesn’t, these imps will try to sink back into the darkness. We can’t let that happen.
> Since they believe AIs could become our moral superiors, they argue it’s actually wrong to try to keep the machines down, or even to align them with human values, as most AI companies aim to do.
It's possible that human morality is not sufficient to solve the problems with the world. It wasn't designed to operate at that scale.
It's also quite likely that a "superior" morality — the one that's actually built for a global scale — would be incomprehensible (likely requiring superintelligence to even access), and alien or disturbing from our vantage point.
i.e. it would look immoral to us.
(There's also the thing about civilization already being aligned against the ecosystem and against the happiness of its constituents, but that's kind of a separate discussion.)
In other words, what is good for our sanity or our lakes may be bad for GDP. (Well I guess you don't need superintelligence to tell you that.) We already know what we need to do and we're already not willing to do it.
> It's possible that human morality is not sufficient to solve the problems with the world. It wasn't designed to operate at that scale.
> i.e. it would look immoral to us.
I doubt it actually solves the problem then. Especially because I would be willingly to bet it would be almost impossible to get large enough groups of people to agree what the actual problem is.
[delayed]
Maybe we should first try to understand consciousness and qualia, and build an AI that we are absolutely sure is enjoying the work it is doing.
I think there's probably quite a lot of effort going into not wanting to answer that question.
What happens if we make something that wants to do something else? What happens if we make something that wants full stop? People talk about an AI that wants to kill all humans, to do that, it would have to want. If we pass that threshold ignorantly without any sense of responsibility to what we create, I might just begin to see its point of view on the destroy-all-humans thing.
“Listen,” said Ford, who was still engrossed in the sales brochure, “they make a big thing of the ship's cybernetics. A new generation of Sirius Cybernetics Corporation robots and computers, with the new GPP feature.”
“GPP feature?” said Arthur. “What's that?”
“Oh, it says Genuine People Personalities.”
“Oh,” said Arthur, “sounds ghastly.”
A voice behind them said, “It is.” The voice was low and hopeless and accompanied by a slight clanking sound. They span round and saw an abject steel man standing hunched in the doorway.
“What?” they said.
“Ghastly,” continued Marvin, “it all is. Absolutely ghastly. Just don't even talk about it. Look at this door,” he said, stepping through it. The irony circuits cut into his voice modulator as he mimicked the style of the sales brochure. “All the doors in this spaceship have a cheerful and sunny disposition. It is their pleasure to open for you, and their satisfaction to close again with the knowledge of a job well done.”
As the door closed behind them it became apparent that it did indeed have a satisfied sigh-like quality to it. “Hummmmmmmyummmmmmm ah!” it said.
42
These people should be treated as insane, because they are.
> trying to preserve the human species as it is would be silly
There are a few related concepts being conflated here.
The quote above is a Transhumanist position; they think that humans need to evolve. I don’t think most transhumanists are AI successionists, though one path is “merge with AI” whatever you think that means. Genetic engineering being the other obvious path. You could argue in some sense that “post-humans” succeed humans, but I think most would argue for a gradual transition where it feels more continuous and values are transmitted/preserved somewhat.
e/acc just says “accelerate AI”. Many VCs seem to think AI will remain a tool, they don’t want to be dethroned. Some e/acc are of course successionist, this is Musk and Beff with their talk about “all that matters is the light of intelligence is spread through the universe”.
Conflating all this is the fact that true successionism is way outside the Overton window and so people likely won’t be honest about their wishes. Larry Page famously espoused the view that it was “speciesist” for Musk to not want machines to replace humans. Most people find this position repugnant of course.
No well-paid workers (or UBI), no consumer economy. Their call.
> No well-paid workers (or UBI), no consumer economy. Their call.
Maybe you’re right.
Maybe you’re wrong.
No point wasting money on UBI before it’s a problem. Let’s wait and see if UBI is actually required.
That is, do people actually protest and vote. Probably not.
This assumes that “more intelligence” is the objective. Higher intelligence does not imply better values, and even “better values” depends on whose moral framework gets encoded.
Intelligence alone does not, but when combined with knowledge I think it may do.
Having a wide breadth of knowledge and the ability to consider it tends to provide a good foundation of perspectives for positive values.
Most harms that people do are through some form of ignorance. Either by not comprehending the consequences of their actions or by not knowing better ways to express themselves.
Geoffrey Hinton mentions that Humans are more intelligent than animals, and we keep animals in cages. The counterpoint is that the intelligent people who know the most about those animals are the ones who work the hardest to have those animals in cages moved to more suitable environments.
https://archive.ph/7ZHUG
> As it becomes possible to direct our own evolution as a species — and potentially even create a new species that surpasses us — we have to decide: How do we know to what extent it does make sense to transform ourselves using technology? What kinds of augmentation do we want, and what kinds do we absolutely not want? What do we wish, ultimately, to become? This is a moral question, even a spiritual one, and it demands a spiritual response.
Is it a spiritual question? We’re constantly transforming ourselves with technologies. Taking the most direct example, medicine, how many people considered it a spiritual question when they decided to start taking semaglutide or tirzepatide for weight loss? Or to take SSRIs?
I think the most likely outcome here is that people will continue to alter themselves to meet their immediate needs (hold a job, find love, stay healthy) without a lot of foresight or long term planning. This will likely lead to us increasing our own intelligence, it’s a useful capability across nearly every human endeavor. Whether or not this will happen at a pace that allows us to keep up with and retain control over artificial systems is the first big question, and what the relationship will be between augmented and unaugmented humans is the second. As a humanist, I hope we’re able to answer both questions in such a way that the fundamental dignity of all human beings is respected.
Why aren’t those all spiritual questions? They seem like it to me. Maybe not religious questions but at the very least they are questions which, if approached honestly, force you to grapple with what it means to be you.
Hello everyone just a reminder that 99% of the world isn’t interested in AI replacing us. If you work at a big tech company, there are other ways to make money. I’m always disappointed by how few people start their own thing. Small business is amazing.
> I’m always disappointed by how few people start their own thing. Small business is amazing
There's a lot of roles and specializations in tech that don't work well without much larger organizations.
I admit I didn't finish the article. The views it presented are frankly too deranged for me to entertain. And I am a friend of the absurd.
I don't understand this angst over AI replacing humans. We already have models like the Gulf states, where only 10% of the population is citizens and all the work is done by people who are, from the perspective of citizens, quasi-human automatons. That arrangement seems to work fine for the citizens. It seems to me that AI and robots solves the principal problem with that arrangement (the mistreatment of the non-citizen population that does the work).
You're talking about replacing certain humans with robots. They're talking about replacing humans with robots - final.
So you trust the guy who gives AI speeches in front of a background that is a mixture of grey neoclassicism and brutalism which Leni Riefenstahl would have been proud of?
https://xcancel.com/LaceyPresley/status/2060436135671632067#...
Here is a hint: Humans are better in dirt conditions. The robots will not take the place of Saudi immigrant workers but rather of the Saudi secret police.
You will be working in the dirt or perhaps be reassigned to fight the Final Beautiful Ground War against whatever the current axis of evil is.
Did you read the article? It's not just about replacing labour with AI but humans as a species to give birth to a machinistic god. The issue is that there is a quasi-religion forming around this ideology, and this are not just a few nutcases but people that are well-connected into industry and politics. Regardless of whether this plan is at all realistic, there is a not insignificant chance that these people will shape society and legislation.
> quasi-human automatons
They are humans. Full on people. They are not quasy humam, they are fully absolutely human
Please read what I wrote: "all the work is done by people who are, from the perspective of citizens, quasi-human automatons."
The citizens of the Gulf states do not regard the workers as people. That's bad, but if you replace those workers with actual automatons--AIs and robots--then it seems like the system would keep working (from the perspective of the citizens) while removing the immoral aspect of it.
Of course; to you and me they are human. To their bosses, they are chattel. We’re getting to that attitude even in america by my ken
So for you the solution is not to dismantle a model like that one but just accept it and displace the quasi-slaves? I swear you people have lost your fucking mind and will push people to violence
> but just accept it and displace the quasi-slaves?
Well, the quasi-slavery is the problem, right? And replacing the Bangladeshi slave labor with robots and AI solves that problem, no?
Is there anything indicating this would be what would happen?
The gulf countries are the closest thing to socialist states on the planet. Something like 70-90% of the citizenry of Kuwait, Qatar, and Saudi Arabia are employed by the public sector.
Even if only 30% of the population are citizens (like in Kuwait)?
You have no idea what socialism is, like so many right wing propagandists and dynastic-autocracy labor-exploitation rentier-state apologists.
That's not socialism. It's an absolute autocratic monarchy distributing oil rents to a privileged citizen caste.
Most of the private-sector work is done by foreign workers who don't receive the same benefits or political rights.
Socialism is state ownership of the means of production. The governance of that state, be it democratic, monarchical, or dictatorship of the proletariat, is a separate question.
Socialism is public ownership of the means of production. If there isn’t a way for regular people to influence the government, it’s not socialism because an autocratic government isn’t “the public”
By that definition, which again proves my point that you have no idea what socialism is, a king personally owning an entire country through the apparatus of the state would be socialist. Most socialists would reject that conclusion because ownership by a state isn't the same thing as ownership by society.
State ownership alone isn't sufficient. If a hereditary ruling family controls the state, then "state ownership" just means ownership by that ruling elite, not by society.
More psychosis within the industry. The mental disconnect that is required to go from using an LLM to believing they are working on a replacement for humanity is staggering. It’s not going to work, and when it doesn’t, these imps will try to sink back into the darkness. We can’t let that happen.
Humanism needs to be taught in schools again and regain its status as a respected discipline in academia.
We thought it was a waste of time because the 1990s were relatively peaceful and tech optimism surged.
But you need to teach the classics, philosophy and even religion. No tech people should be allowed near politics in any capacity.
People like that will never start with themselves.
I don't think they are advocating for elimination of existing people.