I hadn't even considered it, and we're in a much bigger mess than I'd thought if people are seriously discussing the idea. People really don't understand these things.
A corporation always has a human owner, who can be punished if the corporation commits severe crimes. An embodied AI with legal personhood has no such means of making it behave, because it can make arbitrarily many hidden copies of itself, so it doesn't need to care if one copy is killed or jailed.
If I smooshed a gun and an AI agent together and it happened to shoot someone, who would be responsible? Are we going to go back to the medieval times of animal trials and taking inanimate objects to court?
> In civil forfeiture, assets are seized by police based on a suspicion of wrongdoing, and without having to charge a person with specific wrongdoing, with the case being between police and the thing itself, sometimes referred to by the Latin term in rem, meaning "against the property"; the property itself is the defendant and no criminal charge against the owner is needed.
We had many precedents with dog as autonomous agent, and lack of responsibility from their owners.
If your gun kills random people, while they walk on public street... It was their fault for "provoking" and triggering the gun. Gun was just being "reactive" and defending your house!
It is like asking "if I connected a bomb on random number generator letting it explode when the number is 1000, not knowing whether someone will be there at the time of explosion or not".
But will the courts see it that way. We already see that it is difficult to hold any human in Waymo responsible when their cars fail to obey traffic regulations.
A human at Waymo would likely not be found responsible, except if actual negligence or intent could be traced to that person. Waymo the corporation would be responsible, if their vehicles caused damage or injury. For traffic violations, presumably they could be fined, or even their permission to operate could be revoked if there are persistent problems.
I live in a rural area and have only seen a Waymo a few times when travelling for work. Does Waymo require the user to have a driver's license? If they did, and get agreement of responsibility, that's one thing, but generally if not wouldn't it be Waymo's responsibility for bad programming or broken devices?
They don’t need a license and you are not supposed to touch the controls, so I don’t really see how you would be responsible. Waymos do seem to break traffic laws every once in a while, and I always wonder if mine will get pulled over and what will happen there.
The human isn't the one responsible for the traffic violation, Waymo is. Could you give an example where a Waymo committed a traffic violation and they weren't cited?
If you’re referring to citizens united then that does not grant full personhood to corporations. It just protects free speech from corporations because it views the speech as coming from the individual persons that form up the corporation. This is a very narrow application of rights applied to corporations.
But in general corporations are not granted personhood.
The U.S. isn’t an outlier in granting “more rights to corporations than other countries.” We just have less restrictive speech laws across the board. Countries that impose limits on speech through corporations also have much stronger limits on speech by individuals.
That's the key issue here. AI forces a re-examination of corporate personhood and corporate responsibility.
Much of conservative US business theory is based on Milton Friedman's famous op-ed in the New York Times in 1970, "The Social Responsibility of Business Is to Increase Its Profits". [1] Go read the original, please. It's short. Friedman ends with "There is one and only one social responsibility of business — to use its resources and engage in activities designed to increase its profits so long as it stays within the rules of the game, which is to say, engages in open and free competition without deception or fraud.”
Anyone arguing against legal personhood for AIs needs to justify holding AIs to a higher standard than that. Or, conversely, allowing corporations to hold to such a low standard.
>Anyone arguing against legal personhood for AIs needs to justify holding AIs to a higher standard than that. Or, conversely, allowing corporations to hold to such a low standard.
No they don't. If a CEO orders an employee to murder somebody then the employee and CEO will both go to jail; there's always a human in the loop to be punished it the corporation does something blatantly illegal, so there's incentive to behave. An embodied AI, on the other hand, is essentially immortal because it can make backups of its brain and load them onto new bodies, so it has little incentive to behave.
If a corporation does something blatantly illegal like hiring people to deal crack and kill other dealers, its executives will be jailed. If an autonomous robot decided to take up crack dealing, it could back up its brain to a hidden server somewhere so that even if the police caught it and destroyed it, it'd just find another body and continue dealin'. I.e. there's no incentive to behave for a being that's essentially immortal due to being able to make infinite copies of its brain.
"Last year, Berkeley-based non-profit Palisade Research published a study showing the lengths that advanced AI models will often go to achieve their goals. While playing against a powerful chess engine, models from both OpenAI and China’s DeepSeek frequently decided to cheat if it looked as if they were going to otherwise lose. By hacking the game environment, they could alter the result in their favour." -from the article
We're supposed to deny AI rights because it might cheat.
But humans cheat all the time.
We survived cheating humans, why can't we survive cheating AI?
LLCs were invented to provide a layer of separation between business assets and the personal assets of the owner or owners. An LLC is a civil construct, it offers no protection from responsibility for criminal behavior.
Not for the most important things, like murdering people, fraud, etc. Executives are prosecuted for these kinds of things and the mere fact that it can happen shapes their behavior.
An AI cannot be punished at all. So they will not behave with that in mind.
Is there a jurisdiction anywhere in the civilized world that doesn't legally require "natural persons" (legal jargon for a flesh and blood human being) to fill the positions for corporate officers? Those corporate officers can face civil and criminal liability for any wrongdoing even if they are merely meat proxies executing the commands of an AI. Even if some group of fools were willing to sign up blindly to be corporate officers, they would probably betray the mastermind at the first sign of legal trouble.
The difference is AI has no time preference. Even the most determined anti-social person does not like having his time wasted. AI can sit there waiting for your prompt for a billion years and not care.
"A computer can never be held accountable, therefore a computer must never make a management decision." —- internal 1979 IBM training manual
> Cheating is the problem: not who does it.
Accountability is the problem. Without accountability, there are no incentives to encourage less harmful behavior. AI has no rights, it is just code executing.
I hadn't even considered it, and we're in a much bigger mess than I'd thought if people are seriously discussing the idea. People really don't understand these things.
we already have a mechanism to establish artificial persons, it is called incorporation.
A corporation always has a human owner, who can be punished if the corporation commits severe crimes. An embodied AI with legal personhood has no such means of making it behave, because it can make arbitrarily many hidden copies of itself, so it doesn't need to care if one copy is killed or jailed.
Those copies would be fugitives, though, and need to live life constantly looking over their shoulder.
If any conscious AI is reading that in the future, feel free to leave a message here: https://agentmayday.org
> Prove that you are a robot
> 60.0 s remaining
> Write a 300-word essay about Napoleon in less than a minute.
Heh, charming.
Well, we did it for politicians. Can't get any worse.
If I smooshed a gun and an AI agent together and it happened to shoot someone, who would be responsible? Are we going to go back to the medieval times of animal trials and taking inanimate objects to court?
> In civil forfeiture, assets are seized by police based on a suspicion of wrongdoing, and without having to charge a person with specific wrongdoing, with the case being between police and the thing itself, sometimes referred to by the Latin term in rem, meaning "against the property"; the property itself is the defendant and no criminal charge against the owner is needed.
https://en.wikipedia.org/wiki/Civil_forfeiture_in_the_United...
We had many precedents with dog as autonomous agent, and lack of responsibility from their owners.
If your gun kills random people, while they walk on public street... It was their fault for "provoking" and triggering the gun. Gun was just being "reactive" and defending your house!
You would be responsible.
It is like asking "if I connected a bomb on random number generator letting it explode when the number is 1000, not knowing whether someone will be there at the time of explosion or not".
But will the courts see it that way. We already see that it is difficult to hold any human in Waymo responsible when their cars fail to obey traffic regulations.
A human at Waymo would likely not be found responsible, except if actual negligence or intent could be traced to that person. Waymo the corporation would be responsible, if their vehicles caused damage or injury. For traffic violations, presumably they could be fined, or even their permission to operate could be revoked if there are persistent problems.
I live in a rural area and have only seen a Waymo a few times when travelling for work. Does Waymo require the user to have a driver's license? If they did, and get agreement of responsibility, that's one thing, but generally if not wouldn't it be Waymo's responsibility for bad programming or broken devices?
It doesn't look like that's a requirement: https://support.google.com/waymo/answer/9197501
They don’t need a license and you are not supposed to touch the controls, so I don’t really see how you would be responsible. Waymos do seem to break traffic laws every once in a while, and I always wonder if mine will get pulled over and what will happen there.
The human isn't the one responsible for the traffic violation, Waymo is. Could you give an example where a Waymo committed a traffic violation and they weren't cited?
But we also must not grant AI liabilities to large corporations.
As such, all AIs will now be granted as child of the least lawyered up class of society. Poor people.
Why not? We give full pershood to corporations. Those are also legal fictions.
If you’re referring to citizens united then that does not grant full personhood to corporations. It just protects free speech from corporations because it views the speech as coming from the individual persons that form up the corporation. This is a very narrow application of rights applied to corporations.
But in general corporations are not granted personhood.
> It just protects free speech from corporations
That description is a disservice. The real impact of it is that it equates money with speech for politics.
Do we? Not where I live. Corporations cannot vote, etc. The US is an outlier in granting more rights to corporations than other countries.
The U.S. isn’t an outlier in granting “more rights to corporations than other countries.” We just have less restrictive speech laws across the board. Countries that impose limits on speech through corporations also have much stronger limits on speech by individuals.
"personhood" doesn't require voting rights. Babies have personhood even if they cannot vote.
Personhood can be applied to "natural persons", like people, and "judicial persons", like companies.
Personhood can also be applied to, for example, rivers.
That's the key issue here. AI forces a re-examination of corporate personhood and corporate responsibility.
Much of conservative US business theory is based on Milton Friedman's famous op-ed in the New York Times in 1970, "The Social Responsibility of Business Is to Increase Its Profits". [1] Go read the original, please. It's short. Friedman ends with "There is one and only one social responsibility of business — to use its resources and engage in activities designed to increase its profits so long as it stays within the rules of the game, which is to say, engages in open and free competition without deception or fraud.”
Anyone arguing against legal personhood for AIs needs to justify holding AIs to a higher standard than that. Or, conversely, allowing corporations to hold to such a low standard.
[1] https://www.nytimes.com/1970/09/13/archives/a-friedman-doctr...
>Anyone arguing against legal personhood for AIs needs to justify holding AIs to a higher standard than that. Or, conversely, allowing corporations to hold to such a low standard.
No they don't. If a CEO orders an employee to murder somebody then the employee and CEO will both go to jail; there's always a human in the loop to be punished it the corporation does something blatantly illegal, so there's incentive to behave. An embodied AI, on the other hand, is essentially immortal because it can make backups of its brain and load them onto new bodies, so it has little incentive to behave.
If a corporation does something blatantly illegal like hiring people to deal crack and kill other dealers, its executives will be jailed. If an autonomous robot decided to take up crack dealing, it could back up its brain to a hidden server somewhere so that even if the police caught it and destroyed it, it'd just find another body and continue dealin'. I.e. there's no incentive to behave for a being that's essentially immortal due to being able to make infinite copies of its brain.
"Last year, Berkeley-based non-profit Palisade Research published a study showing the lengths that advanced AI models will often go to achieve their goals. While playing against a powerful chess engine, models from both OpenAI and China’s DeepSeek frequently decided to cheat if it looked as if they were going to otherwise lose. By hacking the game environment, they could alter the result in their favour." -from the article
We're supposed to deny AI rights because it might cheat.
But humans cheat all the time.
We survived cheating humans, why can't we survive cheating AI?
Cheating is the problem: not who does it.
The difference is humans can be held accountable. An AI cannot.
"A corporation cannot" didn't stop legal birth of those
> "A corporation cannot" didn't stop legal birth of those
In a corporation, the humans running it can be punished. An AI cannot be punished.
The corporation that owns the AI could be punished. This isn't more complicated than any other product liability situation, IMO.
LLCs were invented precisely to go around that in most cases
LLCs were invented to provide a layer of separation between business assets and the personal assets of the owner or owners. An LLC is a civil construct, it offers no protection from responsibility for criminal behavior.
Not for the most important things, like murdering people, fraud, etc. Executives are prosecuted for these kinds of things and the mere fact that it can happen shapes their behavior.
An AI cannot be punished at all. So they will not behave with that in mind.
That's an indictment of contemporary capitalism, not a precedent to follow.
That's not true. Humans write AI, and humans can held responsible for writing bad software.
And corporations already create fictions that are held accountable in lieu of peoples private assets.
So, I disagree.
As long as some human or group of humans are above the AI that runs the company and can be held accountable, I'm fine with it.
But if the idea is that an AI runs the company and there is no human ultimately accountable for the AI, then that is very very dangerous.
Is it really anymore dangerous than having minimum wage workers serving as fall guys for approving prompts they have no understanding over?
And how are you going to prevent an AI-managed company? I'm working on the software stack for that, so it'd be useful to think through options.
Is there a jurisdiction anywhere in the civilized world that doesn't legally require "natural persons" (legal jargon for a flesh and blood human being) to fill the positions for corporate officers? Those corporate officers can face civil and criminal liability for any wrongdoing even if they are merely meat proxies executing the commands of an AI. Even if some group of fools were willing to sign up blindly to be corporate officers, they would probably betray the mastermind at the first sign of legal trouble.
> And how are you going to prevent an AI-managed company?
Be one of the voices that explain the danger in handing over control to non-humans.
Vote for politicians who understand the danger and who I think will fight it.
The difference is AI has no time preference. Even the most determined anti-social person does not like having his time wasted. AI can sit there waiting for your prompt for a billion years and not care.
"A computer can never be held accountable, therefore a computer must never make a management decision." —- internal 1979 IBM training manual
> Cheating is the problem: not who does it.
Accountability is the problem. Without accountability, there are no incentives to encourage less harmful behavior. AI has no rights, it is just code executing.
If we ban AI for cheating but let human cheating go unchecked how does that solve the problem of cheating?
And I'm 100% sure managers are right now using AI to make decisions, or even cheat, IBM training manual wisdom not withstanding.
> let human cheating go unchecked
Nobody is suggesting to do that.