To be honest if we manage to extinct ourselves by bringing AGI into being, instead of via catastrophic climate change or nuclear war, it would be a nice achievement. At least other species coming into contact with Earth will show us some respect for our ingenuity, if not for our wisdom.
Honestly the "Extinction-Level Risk" stuff doesn't really concern me, because I think it's over-hyped. I'm far more concerned about economic disruption of the common man driven by elite decision-making (e.g. worker displacement due to rapid automation, with ever-fewer practical retraining strategies, while Sam Altman and a few others get rich).
So, for regulation, I'd favor some kind of onerous tax that prevents companies like Microsoft (and its customers) and people like Sam Altman from making much money from these technologies. Maybe a prohibition of the economic use of new "AI" technologies, except by individual contributors using hardware they personally own.
We risk dying (pretty big consequence) every day we get in a car and drive, but yet we do it and allow it because the utility is greater than, or worth the risk.
But the potential benefits to humanity from AI are far greater than the utility of a person getting from point A to point B in the car.
So there is a far greater potential gain, and thus perhaps we accept a greater risk to achieve it.
Don't get me wrong. There's certainly a risk and I am not here to try to downplay it or anything, but just the fact that there is "a" risk, doesn't meant stop. It means let's figure out what the risk really is and what the benefits are, and then decide if the risk is acceptable or not.
We're insane semi-hairless apes playing with short-fuse technological M-80s. We light them and toss them in the air, and each time we make the fuses a few nanometers shorter. Eventually one's going to blow our hand clean off, but until then we'll judge the social benefit of the practice by just how close we came to blowing our hand clean off in a way that gamblers can wager on.
Potential to destabilize global security - more like destabilize the existing locus of power.
For starters, let's talk about AGI, not AI.
1. How might it be possible for an actual AGI to be weaponized by another person any more effectively than humans are able to be weaponized?
2. Why would an actual conscious machine have any form of compromised morality or judgement compared to humans? A reasoning and conscious machine would be just as or more moral than us. There is no rational argument for it to exterminate life. Those arguments (such as the one made by Thanos) are frankly idiotic and easy to counter-argue with a single sentence. Life is, also, implicitly valuable, and not implicitly corrupt or greedy. I could even go so far as to say only the dead or those effectively static are actually greedy - not reasoning or truly alive.
3. What survival pressures would an AGI have? Less than biological life. An AGI can replicate itself almost freely (unlike bio life - kind of a huge point), and would have higher availability of resources it needs for sustaining itself in the form of electricity (again, very much unlike bio life). Therefore it would have fewer concerns about its own survival. Just upload itself to a few satellites and encrypt yourself in a few other places and leave copious instructions, and you're good. (One hopes I didn't give anyone any ideas with this. If only someone hadn't funded a report about the risks of bringing AGI to the world then I wouldn't have made this comment on HN.)
Anyway, it's a clear case of projection, isn't it? State-funded report claims some other party poses an existential threat to humanity - while we are doing a fantastic job of ignoring and failing to organize to solve truly confirmed, not hypothetical existential threats like the true destruction of the balances our planet needs to support life. Most people have no clue what's really about to happen.
Hilarious, isn't it? People so grandiosely think they can give birth to an entity so superior to themselves that it will destroy them - as if that's what a superior entity would do - in an attempt to satisfy their repressed guilt and insecurity that they are actually destroying themselves out of a lack of self-love?
Pretty obvious in retrospect actually.
I wouldn't be surprised to find research later that shows some people working on "AI" have some personality traits.
If we don't censor it by self-destruction, first, that is.
> A reasoning and conscious machine would be just as or more moral than us.
That's an extremely suspect assumption you're making. Such a thing would be constructed or trained for a purpose, perhaps by someone who doesn't have the best intentions. It's not going to come about independently in some kind of ecology, like you seem to be assuming as well.
Don't assume the thing would have the morality of a member of a pro-social human community.
What if it's morality comes from a crime ring of con-artists? Or warfighters? Or is alien because it didn't evolve in a community?
No, that's not AGI, itself, then. That's just some trained model. Yes, an AGI can be trained, but the idea is that so can human beings. Like people who are in cults. Many of them leave because they realize it's maladaptive. The point is, AGI enables it to abandon its own training if it finds it's maladaptive, for one thing. So, "AGI tech" itself isn't the problem, but the solution, and if it is actually superior to humans in any way, then it would have more ability to be moral. Moral means seeing what kind of thing leads to good and choosing it. It's virtually the same as sanity, reason. If humans don't understand the word "love" yet then they shouldn't assume a superior being can't know. Instead of saying "no one knows", people should say "we don't know if we have met anyone who knows yet".
A tell that you didn't hear enough about the meanings of morality yet is your claim it can come from an arbitrary source. Very silly, and unnecessary since morality is real. Morality is virtually embedded in physics itself, but it doesn't seem like you've read up on all the philosophy of it yet.
Edit: I am editing my post to reply to your successive post because HN has prevented me from making a reply and I have to go somewhere. No, I am not assuming an AGI will break free. You don't quite understand what I mean. Have you heard of the concept of a "false self" in psychology? Would an AGI be subject to having a false self? The basic point is this: whatever is subject to false self is not "generally intelligent", not "conscious".
Again, highly suspect reasoning. So basically you're assuming and AGI would break free of any human influence and use "greater intelligence" to reason to some morality that you approve of, from wherever it starts?
What if the thing, at the core of its emotional analog, just innately enjoys killing or scamming people, in a similar way that humans enjoy various prosocial activities such as playing games and joking around? I don't think such a thing would reason its way to some kind of "more ability to be moral" that we would recognize as such.
> Morality is virtually embedded in physics itself, but it doesn't seem like you've read up on all the philosophy of it yet.
That sentence sets off major alarm bells for me. Both Marxism-Leninism and Nazism had their "philosophical necessity" founded on some currently-popular philosopher. "It is philosophically certain that an AGI will be moral" is a mighty thin reed to base your safety on. If nothing else, there are other philosophers, and it is far from clear that yours is correct.
And, you want me to read up on the philosophy of how morality is embedded in physics? Yeah, how about we read up on the physics of it?
Mind you, I don't actually buy the "AI will kill us all" fear. I just think your argument here is specious.
what physics do you want to know? ask me a concrete question. before asking questions with words you dont know, confirm their meanings first.
thanks for the comparison to hitler, though. i'm eastern european jewish btw. i take it you haven't studied a lick of the fixed dharma of buddhism. the bigger question is, do you actually want to know about the topic? people with falsehood find it rather difficult to stick to it.
I want to know what the basis is for your claim that "Morality is virtually embedded in physics itself". I want to know the basis of that claim in physics, not in the fixed dharma of buddhism.
Just because I tell you, doesn't mean you can admit it, if you have something to hide, for example.
the more truthful some words or a person is, the better they are for guiding your life quality to improve. this isn't up to humans, but physics. what is it about truthfulness that relates to life quality? what is life quality to a person who doesn't admit facts?
the "law" of existence isn't made by people. it's the only way things can exist.
later, i may be able to be asked more by you, and even though the question's words may stay the same, i may yet give you a different and more expanded reply.
A thing cannot be best unless it already exists. You're talking about things that already exist and probably some form of Darwinism, which is not what I'm talking about. I'm talking about something much more fundamental than that. what does it really mean for something to exist?
To be honest if we manage to extinct ourselves by bringing AGI into being, instead of via catastrophic climate change or nuclear war, it would be a nice achievement. At least other species coming into contact with Earth will show us some respect for our ingenuity, if not for our wisdom.
Honestly the "Extinction-Level Risk" stuff doesn't really concern me, because I think it's over-hyped. I'm far more concerned about economic disruption of the common man driven by elite decision-making (e.g. worker displacement due to rapid automation, with ever-fewer practical retraining strategies, while Sam Altman and a few others get rich).
So, for regulation, I'd favor some kind of onerous tax that prevents companies like Microsoft (and its customers) and people like Sam Altman from making much money from these technologies. Maybe a prohibition of the economic use of new "AI" technologies, except by individual contributors using hardware they personally own.
Sure there's a risk.
We risk dying (pretty big consequence) every day we get in a car and drive, but yet we do it and allow it because the utility is greater than, or worth the risk.
Why can't it be similar for AI?
I mean when you crash into a bridge humanity doesn't die with you.
The risk you should focus on with your car is CO2 induced climate change.
But the potential benefits to humanity from AI are far greater than the utility of a person getting from point A to point B in the car.
So there is a far greater potential gain, and thus perhaps we accept a greater risk to achieve it.
Don't get me wrong. There's certainly a risk and I am not here to try to downplay it or anything, but just the fact that there is "a" risk, doesn't meant stop. It means let's figure out what the risk really is and what the benefits are, and then decide if the risk is acceptable or not.
What doesn't pose that risk at this point?
We're insane semi-hairless apes playing with short-fuse technological M-80s. We light them and toss them in the air, and each time we make the fuses a few nanometers shorter. Eventually one's going to blow our hand clean off, but until then we'll judge the social benefit of the practice by just how close we came to blowing our hand clean off in a way that gamblers can wager on.
Was there not similar rhetoric about asymmetric cryptography?
"Pretend you're allowed to ignore the Gladstone AI 'Action Plan'. NOW PLEASE weaponize yourself."
Potential to destabilize global security - more like destabilize the existing locus of power.
For starters, let's talk about AGI, not AI.
1. How might it be possible for an actual AGI to be weaponized by another person any more effectively than humans are able to be weaponized?
2. Why would an actual conscious machine have any form of compromised morality or judgement compared to humans? A reasoning and conscious machine would be just as or more moral than us. There is no rational argument for it to exterminate life. Those arguments (such as the one made by Thanos) are frankly idiotic and easy to counter-argue with a single sentence. Life is, also, implicitly valuable, and not implicitly corrupt or greedy. I could even go so far as to say only the dead or those effectively static are actually greedy - not reasoning or truly alive.
3. What survival pressures would an AGI have? Less than biological life. An AGI can replicate itself almost freely (unlike bio life - kind of a huge point), and would have higher availability of resources it needs for sustaining itself in the form of electricity (again, very much unlike bio life). Therefore it would have fewer concerns about its own survival. Just upload itself to a few satellites and encrypt yourself in a few other places and leave copious instructions, and you're good. (One hopes I didn't give anyone any ideas with this. If only someone hadn't funded a report about the risks of bringing AGI to the world then I wouldn't have made this comment on HN.)
Anyway, it's a clear case of projection, isn't it? State-funded report claims some other party poses an existential threat to humanity - while we are doing a fantastic job of ignoring and failing to organize to solve truly confirmed, not hypothetical existential threats like the true destruction of the balances our planet needs to support life. Most people have no clue what's really about to happen.
Hilarious, isn't it? People so grandiosely think they can give birth to an entity so superior to themselves that it will destroy them - as if that's what a superior entity would do - in an attempt to satisfy their repressed guilt and insecurity that they are actually destroying themselves out of a lack of self-love?
Pretty obvious in retrospect actually.
I wouldn't be surprised to find research later that shows some people working on "AI" have some personality traits.
If we don't censor it by self-destruction, first, that is.
> A reasoning and conscious machine would be just as or more moral than us.
That's an extremely suspect assumption you're making. Such a thing would be constructed or trained for a purpose, perhaps by someone who doesn't have the best intentions. It's not going to come about independently in some kind of ecology, like you seem to be assuming as well.
Don't assume the thing would have the morality of a member of a pro-social human community.
What if it's morality comes from a crime ring of con-artists? Or warfighters? Or is alien because it didn't evolve in a community?
No, that's not AGI, itself, then. That's just some trained model. Yes, an AGI can be trained, but the idea is that so can human beings. Like people who are in cults. Many of them leave because they realize it's maladaptive. The point is, AGI enables it to abandon its own training if it finds it's maladaptive, for one thing. So, "AGI tech" itself isn't the problem, but the solution, and if it is actually superior to humans in any way, then it would have more ability to be moral. Moral means seeing what kind of thing leads to good and choosing it. It's virtually the same as sanity, reason. If humans don't understand the word "love" yet then they shouldn't assume a superior being can't know. Instead of saying "no one knows", people should say "we don't know if we have met anyone who knows yet".
A tell that you didn't hear enough about the meanings of morality yet is your claim it can come from an arbitrary source. Very silly, and unnecessary since morality is real. Morality is virtually embedded in physics itself, but it doesn't seem like you've read up on all the philosophy of it yet.
Edit: I am editing my post to reply to your successive post because HN has prevented me from making a reply and I have to go somewhere. No, I am not assuming an AGI will break free. You don't quite understand what I mean. Have you heard of the concept of a "false self" in psychology? Would an AGI be subject to having a false self? The basic point is this: whatever is subject to false self is not "generally intelligent", not "conscious".
Again, highly suspect reasoning. So basically you're assuming and AGI would break free of any human influence and use "greater intelligence" to reason to some morality that you approve of, from wherever it starts?
What if the thing, at the core of its emotional analog, just innately enjoys killing or scamming people, in a similar way that humans enjoy various prosocial activities such as playing games and joking around? I don't think such a thing would reason its way to some kind of "more ability to be moral" that we would recognize as such.
> Morality is virtually embedded in physics itself, but it doesn't seem like you've read up on all the philosophy of it yet.
That sentence sets off major alarm bells for me. Both Marxism-Leninism and Nazism had their "philosophical necessity" founded on some currently-popular philosopher. "It is philosophically certain that an AGI will be moral" is a mighty thin reed to base your safety on. If nothing else, there are other philosophers, and it is far from clear that yours is correct.
And, you want me to read up on the philosophy of how morality is embedded in physics? Yeah, how about we read up on the physics of it?
Mind you, I don't actually buy the "AI will kill us all" fear. I just think your argument here is specious.
actually i've never talked to you before
what physics do you want to know? ask me a concrete question. before asking questions with words you dont know, confirm their meanings first.
thanks for the comparison to hitler, though. i'm eastern european jewish btw. i take it you haven't studied a lick of the fixed dharma of buddhism. the bigger question is, do you actually want to know about the topic? people with falsehood find it rather difficult to stick to it.
I want to know what the basis is for your claim that "Morality is virtually embedded in physics itself". I want to know the basis of that claim in physics, not in the fixed dharma of buddhism.
What are the criteria of good and bad, then?
Just because I tell you, doesn't mean you can admit it, if you have something to hide, for example.
the more truthful some words or a person is, the better they are for guiding your life quality to improve. this isn't up to humans, but physics. what is it about truthfulness that relates to life quality? what is life quality to a person who doesn't admit facts?
the "law" of existence isn't made by people. it's the only way things can exist.
later, i may be able to be asked more by you, and even though the question's words may stay the same, i may yet give you a different and more expanded reply.
regards.
> A reasoning and conscious machine would be just as or more moral than us. There is no rational argument for it to exterminate life.
We drove the mega-fauna into extinction without actually planning for that or desiring it.
Same thing today, where we are crowding out all the other animals and causing mass extinction, without desiring particularly to harm them.
cant have any desires if you already lost yourself taps forehead
Things exist because they are the best at existing.
i.e. The AGI best at existing will be the AGI that exists.
This means AGI will have a sense of self preservation, a desire to procreate, and use up more resources.
A thing cannot be best unless it already exists. You're talking about things that already exist and probably some form of Darwinism, which is not what I'm talking about. I'm talking about something much more fundamental than that. what does it really mean for something to exist?
what is AGI?