The Mythology of Conscious AI

(noemamag.com)

81 points | by Anon84 10 hours ago ago

171 comments

  • dsign 6 hours ago

    There are some bits and pieces in the article that I agree with, but by far, the most interesting part of it is its first sentence:

    > For centuries, people have fantasized about playing God by creating artificial versions of human beings.

    Note how "God" is capitalized. Way before "God" was the vast central institution that it is today, there were a myriad of divinities invented around the campfire. What was told there, from older to younger, was for the passing down of experiences and wisdom. "Don't try to fly" (Icarus), "death comes to us all" (The Epic of Gilgamesh), "have kids to make us happy" (Sarah and Isaac). But as time passed and we built civilizations, our campfire deities got temples and empires. Their wisdom wasn't _only_ about simple things anymore, but also about the duties toward State (Paul's Epistle to the Romans 13:1 is a clear example). And State wants things uncomplicated when it comes to its cogs. That simple fact has poissonned the the well from which we drink our moral guidance.

    All I'm trying to say, if you happen to be a cog, keeping your fantasies is not so great of an indulgence, even if it pisses off God and State.

    • amelius 37 minutes ago

      > For centuries, people have fantasized about playing God by creating artificial versions of human beings.

      But it's the other way around. We have created a god that will rule over us instead of us over it.

      • bamboozled 6 minutes ago

        Haven't gods basically always ruled over us, why did people make sacrifices, offerings and prayers to them ?

    • raffael_de an hour ago

      You just chose the minor gods from those cultures for your comment. There were also major gods like Zeus and Saturn and they also had temples and empires back then.

    • thomashighbaugh 5 hours ago

      All of your concepts of God here are all rooted in a narrow, ethnocentric understanding emerging from the Greeks and ancient Hebrew concepts thereof. In fact, your examples are all localized to those regions of west Asia and southern Europe.

      Just because the ideas of God you appropriate into some notion of your identity were obviously flawed and fake, doesn't mean the entire concept is the same laughable stupidity of being special people to the jealous, but only, God who is used as proxy for the Roman emperor.

      The philosophical and religious traditions of India (with orthodox schools of Hindu philosophy that are explicitly atheist and an internally consistent logic just like the Greeks only not deliberately abusive in being incomplete so Socrates could win debates by excluding premises the other side wouldn't think of) define this concept radically differently where they define it and arguably have the oldest compendiums of wisdom about how society should function within itself and as it expands outwards, the Vedas chiefly concern with such and at least 3k years older than any of those suggestive of other things afoot letters. The fact that this is absent in your analysis implies that you are too ignorant of any system other than the Roman-emperor-worship-by-proxy religion to adopt such a mocking tone.

      • dsign 4 hours ago

        Bhagavad Gita. Kurukshetra War, the avatar of Vishnu incites Arjuna to fight his relatives, teachers and friends, "for it was righteous".

        > The fact that this is absent in your analysis implies that you are too ignorant of any system other than the Roman-emperor-worship-by-proxy religion to adopt such a mocking tone.

        Beyond trying to insult me, I fail to see your overall argument. May I presume it was "there are systems of reason and morals that are compatible with a carefully architected notion of a central god"? If so, I wasn't debating that point. It is, however, an interesting exercise to honestly apply those frameworks to the statement "man dreams of creating artificial human beings". It's also way harder than you may imagine, because ancestral myths have a way to worm their way into the purest constructs of reason.

      • viccis 4 hours ago

        >ethnocentric understanding emerging from the Greeks and ancient Hebrew concepts thereof

        Kind of a fatuous thing to say about someone quoting Mesopotamian literature.

        >doesn't mean the entire concept is the same laughable stupidity of being special people to the jealous, but only, God who is used as proxy for the Roman emperor

        This is almost unintelligible, but particularly ignorant when it comes to the concept of a "proxy for the Roman emperor"

        >that are explicitly atheist and an internally consistent logic just like the Greeks only not deliberately abusive in being incomplete so Socrates could win debates by excluding premises the other side wouldn't think of

        This is ignorant shit dude. That's not what Socratic dialetics are.

        >The fact that this is absent in your analysis implies that you are too ignorant of any system other than the Roman-emperor-worship-by-proxy religion to adopt such a mocking tone.

        If you are flexing on outcomes then I'm not sure India is the best place to build your argument from. They were one of the biggest empires in the (very irrelevant I might add) early ages and now they are a global embarrassment, ruled by fanatical religious zealots, operated by an almost unmatched level of corruption that is keeping them from ever reaching the kind of astounding development that China is making.

  • hax0ron3 9 hours ago

    Consciousness might not necessarily emerge when there is enough computation, but we also don't know that it doesn't emerge. We really know almost nothing about consciousness. It's possible that the question of why the presence of consciousness is correlated with certain kinds of material organization is in principle not accessible to scientific investigation. We don't even know that it is so correlated in general, we just know that for example people report being conscious when they are not asleep and not drugged in certain ways, and report having had no experience of consciousness during dreamless sleep or when drugged in certain ways. Most people feel that for example rocks are not conscious, but other people are conscious, but there's no rigorous scientific explanation backing up any of those hunches.

    • InsideOutSanta an hour ago

      > We really know almost nothing about consciousness

      I think this is all that's really relevant here. People make extremely confident statements about this, but objectively, we hardly know anything. I don't even know if anyone other than me is conscious.

      To be honest, I'm quite confused about why I am conscious. It seems implausible that a bunch of cells would be able to create one self that experiences consciousness, and yet I do experience consciousness. Given how weird that is, I do not feel that I have any level of trustworthy intuition or knowledge about what else might experience consciousness.

    • thuo 22 minutes ago

      I think saying that the way we talk about knowledge in the modern world puts too much weight on describing things quantatively. I think knowledge is still valuable even if it cannot be described quantatively. For example, people knew how to breed dogs for millenia before DNA was discovered, and football players can kick a ball accurately without understanding the quantative aerodynamics of the ball. Folk science is almost always the starting point of rigorous science, for example, people were classifying plants and animals before there was a concept of species.

      So I think when it comes to conciousness, instead of saying we don't know, it'd be better to start with what we know intuitively through folk science and then try to add more rigor. Even if our intuitions about conciousness are not rigorous, they are still the best we have so far for understanding it.

      With that said, my folk sciencey mental model of consciousness is that it's a spectrum of increasing levels of awareness, starting with awareness of the environment and culminating in the awareness of self, and on top of that using that awareness to exist somehow autonomously in the environment. So the spectrum could go from plants, to non-human animals, to humans. I think a population of such aware and autonomous things can reconfigure itself if the environment changes, to keep existing with awareness and autonomy

      So my folk-sciencey reason for being skeptical of computational consciousness is that if people just stopped maintainig datacenters, stopped sending power to them, stopped typing into keyboards, stopped dictating into microphones, etc, the population of computational things that exist today couldn't somehow reconfigure itself to continue existing autonomously. That's unlike how if humans stopped feeding and sheltering dogs, while the population of dogs would reduce, the population would reconfigure itself to find food and shelter from other parts of the environment.

      • hardbass 17 minutes ago

        That depends on if you believe brain in a box can exist or not. We can bit by bit replace all the sense organs and muscle inputs of a dog with the neural signals that feed its brain. Then what? Is this dog alive? This dog stops doing anything if we leave it, does it mean its not alive?

    • za_creature 8 hours ago

      https://writings.stephenwolfram.com/2024/10/on-the-nature-of...

      Stephen Wolfram suggests an intimate relationship between awareness, thermodynamics and the one-way passage of time. Something along the lines of "consciousness is the universe understanding itself".

      • inopinatus 8 hours ago

        This reminds me somehow of Penrose's idea about the quantum<>gravity interface being the same missing physics as consciousness. Penrose wants spacetime geometry to be fundamental, Wolfram thinks it is causal structure, neither ends up at a satisfactory explanation of conscious perception.

      • bitwize 3 hours ago

        Compression might be consciousness as well.

      • addaon 8 hours ago

        > Stephen Wolfram suggests

        So that's one strong argument against it, but I don't think it's sufficient to discard the idea entirely.

        • nomel 6 hours ago

          Context for this shade?

        • za_creature 8 hours ago

          I didn't.

          • isomorphic_duck 8 hours ago

            They were being sarcastic.

            • za_creature 4 minutes ago

              I know. I was agreeing with them that he talks a lot. It's still an interesting idea.

    • soon_to_be_sb 8 hours ago

      >Consciousness might not necessarily emerge when there is enough computation

      Does a circle "emerge" only when you plot it? Or did it always existed? What if there is a little world with conscious beings inside a circle? Did their subjective experience started when you plotted the circle, or did it exist along with the abstract idea of a circle always (though "always" is a meaningless word here).

      The question we should be asking is "Is it possible for subjective experience to exist inside abstract things" with out such abstract things being rendered?

      The answer is hard because we does not know yet if there is an absolute reality at all, which means "being rendered" does not really move things from the abstract plane, and does not really make a difference..

      Damn! did any of that make any sort of sense?

      • throwup238 6 hours ago

        I prefer the Wittgenstein approach: a “circle” is a language game and which game you’re playing depends on both the current context and the context in which you learned to play the game (of which there may be parallel strands and they may be incompatible).

        After that it’s just rule-following all the way down.

      • rramadass 6 hours ago

        The "illusion" of Reality as we perceive it should not be confused with the separate distinct Reality itself. Ontology and Epistemology are separate.

        A great quote from Samkhya/Yoga philosophy is;

        Objective Reality exists independently of the Mind because it is commonly experienced by others besides Oneself".

        Your example of a circle/abstract things is not really logical. They are inert, while to even talk about consciousness you need dynamism i.e. passage-of-time, experience and cause-and-effect.

        • soon_to_be_sb 4 hours ago

          >They are inert, while to even talk about consciousness you need dynamism i.e. passage-of-time, experience and cause-and-effect.

          This is not really a blocker. A video file is also inert, but video player can play it, showing a video where there is passage of time, cause and effect etc.

          The internal consciousnesses just need to experience slices of it in correspondence to its internal "time"...

          >Objective Reality exists independently of the Mind because it is commonly experienced by others besides Oneself".

          Everyone plugged into matrix shared the same reality. Did that make it real?

        • deepfriedchokes 6 hours ago

          If objective reality existed we would be able to predict precise results rather than probabilities.

          • oilkillsbirds 5 hours ago

            We have to compute probabilities because the only thing capable of computing reality is the universe itself. Everything else can only be a piece of the whole and the piece cannot ever compute the whole exactly

      • talon8635 8 hours ago

        Shrodingers circle

        • soon_to_be_sb 8 hours ago

          Do you think if there is an absolute reality at all?

          I think there is not. I think there is no "absolute" anything. Even you can't be absolute. You can't be special. Look at the patterns on your wall. It can't be absolute. Because if it is absolute, someone has to pick it. That is not possible, because then that entity needs to be absolute, and then someone needs to pick it! That goes all the way...

          So nothing can be absolute and everything is abstract and every variant of something exists in some reality so nothing needs to be picked by anyone...

    • roenxi 7 hours ago

      It is quite a funny situation when people have to debate whether a property that cannot be tested, defined or even described with any precision is possessed by some given thing. It goes beyond a lack of science, there is an interesting debate about whether the topic can even be approached academically. We're in the territory of priests here.

      • aleksiy123 7 hours ago

        It’s the ultimate nerd snipe, or red herring.

        I feel like I’m in the camp of this question of consciousness doesn’t actually matter at all.

        It’s an appeal to emotion. We will give ai the right of consciousness when ai will convince the majority (?) of us that it is conscious.

        The logic of it doesn’t matter in practice and probably can’t even be proven anyways.

        It’s made up, no one can even prove themselves or other humans are conscious. And it doesn’t even matter.

        • InsideOutSanta an hour ago

          > The logic of it doesn’t matter in practice

          It matters because we should afford certain rights to entities with consciousness that we don't need to afford to entities without consciousness.

        • soon_to_be_sb 4 hours ago

          Is the pain that you feel (not "show") when somebody pinches you, made up?

          • matwood 4 hours ago

            I’m confused what this has to do with the thread?

            Pain is just an electrical response sent to the brain to alert the organism something potentially bad is happening.

            • soon_to_be_sb 4 hours ago

              Well, for starters, If a thing is conscious, and has the ability to suffer or rewarded, then it becomes a question of ethics if it is right to make it suffer without a corresponding reward.

              So it is not really correct to claim that the question whether a thing is conscious or not, does not matter.

              • matwood 2 hours ago

                But you're equating consciousness with pain/suffering and reward? I mean, we can certainly wire up an AI to 'feel' pain and suffering and reward it for avoiding those - is it now conscious? Even in humans it's just an electrical response.

                • soon_to_be_sb 25 minutes ago

                  > we can certainly wire up an AI to 'feel' pain and suffering and reward it for avoiding those -

                  Lol, no, you are fundamentally wrong here. What you call pain here is something else that I don't think we are on the same page..

                  • hardbass 16 minutes ago

                    What is pain in a human? What if I find the neural activations that cause pain and activate it in you? I think its hard or impossible to know from outside, some might say, but you just read out some electrical signals so what, so I prefer to keep the view that be cautious about mistreating anything you might think has a good chance of being conscious.

                • hardbass 2 hours ago

                  Yes, a very painful and unpleasant electrical response. Would you mind if I shock you?

        • myaccountonhn 3 hours ago

          I agree that the arguments will basically be theological. That said even if we can't know if it is conscious, depending on what people believe it can have very large consequences on society and cause IMO a lot of harm. So I do think it matters.

      • catlifeonmars 4 hours ago

        Yeah I agree, this is a silly discussion, analogous to debating how many angels can dance one the head of a pin.

    • jgord 6 hours ago

      I find it hard to conceptualize the difference between consciousness and self-awareness.

      I guess consciousness requires computation and a self-model, but Im not sure if there is some other ingredient.

      If a chatbot passes a Turing test, and can describe itself / self-reflect .. I guess we dont jump to the conclusion its conscious [ if we know its a chatbot ]

      • hax0ron3 5 hours ago

        Consciousness is being aware of anything whatsoever. Self-awareness, depending on who is using the term, usually means one of two different things: 1) being conscious of self, or 2) having a self concept.

        These two are very different since it's conceivable that something might have a self-concept without actually being conscious (imagine a non-conscious but intelligent being that is capable of meta-evaluation of itself).

        • thomashighbaugh 4 hours ago

          No its the capacity for subjective experience. Definitions and nuance matter here.

          • hax0ron3 3 hours ago

            I should be clear that when I say "aware" in "consciousness is being aware of anything whatsoever", I mean subjective experience. I should have clarified that.

          • sgt101 4 hours ago

            I disagree, this definition is weak, a novel could be said to be consious if we adopt it, and a novel would have the evidence that a dog does not.

          • pluralmonad 4 hours ago

            Oh, that's simple enough. But, wait, what's subjective experience...

  • talon8635 8 hours ago

    It’s a silly debate when we don’t even know what consciousness is, let alone the circumstances for its occurrence. Or am I missing something?

    • scott_w 6 hours ago

      You’re missing the part that this debate is about exploring and understanding what consciousness is. We can try to understand consciousness by just thinking really hard but that’s not worked so far. Another way is to compare things we’ve built to things we understand to be conscious and ask “is this conscious?” The answer to that can help us understand what’s necessary and sufficient (or not) for a construct to have a conscious experience.

      • catlifeonmars 4 hours ago

        Personally, I don’t understand how this is interesting to debate. Why is it necessary for us to have an understanding of what consciousness is? To me it’s reminiscent of the various theological debates that happen within organized religion.

        • scott_w 4 hours ago

          Why is it necessary for us to know about black holes? Because understanding the world around us has intrinsic value, even if we can’t point to why it’ll benefit our lives today.

          • catlifeonmars an hour ago

            Oh I don’t disagree. I’m very curious about the world around us. I just don’t see any reason to believe consciousness is real unless there is evidence to the contrary. The question of consciousness to me smells very theological. You must presuppose its existence before you start debating about it, similar to how you must presuppose the existence of a god as part of engaging in theology. It’s all very silly to me.

      • exe34 an hour ago

        > ask “is this conscious?”

        How will you find out?

      • scotty79 4 hours ago

        > exploring and understanding what consciousness is

        How could anyone possibly do that in absence of tools? Anyone currelntly tweaking LoRAs in their local setups and experimenting with weight rewriting and training processes is closer to understanding consciousness than smartest person writing philosophical texts.

        • hardbass 2 hours ago

          Worse, "philosphers" like this have choice quotes demonstrating complete lack of understanding of Turing computation such as assuming we can already hypercompute or quotes like these genius passages:

          >The sharp division between software and hardware in modern computers is imposed by human design, following Turing’s principles.

          Another even more "genius" passage:

          >This matters because the idea of the brain as a meat-based (universal) Turing machine rests precisely on this sharp separation of scales, on the substrate independence that motivated Turing’s definition in the first place. If you cannot separate what brains do from what they are, the mathematical marriage of convenience starts to fall apart, and there is less reason to think of biological wetware as there simply to implement algorithmic mindware.

          Any "philosopher" of consciousness who can't do basic thermodynamics and theory of computability should be completely ignored and laughed out the room.

      • matwood 4 hours ago

        > You’re missing the part that this debate is about exploring and understanding what consciousness is.

        Is it though? I feel like consciousness is brought up as the final goal post to attack AI. As AI quickly approaches doing everything that a human can do, often better the final retort is “but it’s not conscious”. Useful for the detractors since it’s impossible to define.

    • nomel 6 hours ago

      It would be really funny if this article were written by an AI.

      But, I think the eventual outcome of this is that the bar will continuously raise until a large portion of humanity will be considered "unconscious", at which point we can say it passed the test.

  • ChaitanyaSai 9 hours ago

    If you are interested in consciousness, and also interested in going beyond the usual philosophical approach, you might like our book Journey of the Mind https://www.goodreads.com/en/book/show/60500189-journey-of-t...

    One thing that is implicit, which I now think we really should have expanded on in the book, is that consciousness is fundamentally an engineering problem. And that's also why it's mystified everyone who's brushed away the engineering as the easy plumbing and focused on the other seemingly magical aspects.

    The fundamental question is phrased as "Why does it feel like something?" it being perception, which is essentialy computation. Now that AI does look like it does all of the understanding we do, many including people like Hinton think they have consciousness or are very near it. Well, they are capable of simulating consciousness but that is not the same as experiencing it.

    To answer "Why does it feel like something?" we can focus on what's been glossed over: Who is feeling it? The "I" does not come from as a free lunch and it must be constructed moment to moment from some decentralized cohesion and consensus of a billions of neurons. And this consensus must be metabolically cheap, and we need to do it again and again for everything from feeling a pin prick to hearing a rustle and determining if that's a snake or a leaf to pondering thought.

    This is the insane engineering, the energtically cheap chorus that creates the transient I and this cheap chorus is where both feeling and feeler emerge. Perception, understanding, computation can occur without this cheap chorus, but then there is no feeler, and so no feeling.

    What this points to is that we will have conscious AI when we have decentralized autonomous swarms where a few million of them have to cohere, collude, collaborate, conspire. My worry here is that it will first be done for harmful purposes (autonomous military drone swarms) and not for anything good.

    • hax0ron3 8 hours ago

      We know that self-reported consciousness is correlated with some material states (for example, an awake human body). But that's about all that we can solidly say about how consciousness relates to matter. We know how to create intelligence, but it is possible that a human-level intelligence can exist without ever being conscious. It is also possible that any human-level intelligence automatically achieves consciousness. We don't know. There is actually no good reason to believe that consciousness is an engineering problem. The "I" is not necessarily correlated with consciousness. Those are two different things.

      • rerdavies 6 hours ago

        Not sure what the distinction is. AIs have "material state". Is there a good reason to believe that consciousness cannot be an engineering problem? Or, in this particular case, an unanticipated side-effect of an engineering problem.

        > The "I" is not necessarily correlated with consciousness.

        It's a fair point, given that we don't really have a good definition of "I" OR "consciousness". That is a possibility. But I'm curious what kind of consciousness you are contemplating that doesn't have an "I".

        • hax0ron3 5 hours ago

          >But I'm curious what kind of consciousness you are contemplating that doesn't have an "I".

          Sometimes I wake up in the morning and there is a brief interval during which I can, for example, feel the blanket on my skin, before I actually have remembered where I am or who I am. So there is consciousness but the sense of self has not loaded yet.

      • soon_to_be_sb 8 hours ago

        >self-reported consciousness is correlated with some material states (for example, an awake human body).

        Wait, but what about dreams? I could self report I am conscious when I am in a dream..

        • hax0ron3 8 hours ago

          You're right, also dreams. Just not dreamless sleep. I didn't mean "an awake human body" to be exhaustive.

      • catlifeonmars 4 hours ago

        This debate is so very anthropocentric, which is why I’m inclined to dismiss it as silly.

      • zero_iq 3 hours ago

        Perhaps it is possible that human level intelligence may exist within some humans without consciousness! (aka: philosophical zombie)

        I sometimes wonder if those who espouse straightforward physical/'engineering' solutions to the Hard Problem of Consciousness might think them such obvious or satisfactory solutions because they themselves might not be conscious! After all, a philosophical zombie (who detects and processes stimuli) would not understand why such theories seem unsatisfactory/incomplete to a conscious intelligence (who, additionally, perceives qualia).

        (No offence intended to the poster above!)

        Then again, I suspect an awful lot more of the mental processes and thinking of actually-conscious humans is more automatic and unconscious than most would like to admit!)

        • hardbass 2 hours ago

          Yes definitely, some people are p zombies. Also some people have an imaginary running unicorn in their head that doesn't interact physically at all, but somehow they do, and they are real people and people without the unicorn are not real people.

          • zero_iq an hour ago

            And perhaps in some people that little unicorn leaps to conclusions and responds to online posts before fully appreciating the nuances of the post they're responding to...? ;p

            • hardbass 29 minutes ago

              The unicorn cannot ever be detected, but it exists. You see some people have unicorns some don't . The unicorn makes no difference whatsoever to anything.

          • zero_iq an hour ago

            And perhaps in some people that little unicorn leaps to conclusions and responds to online posts before fully appreciating the nuances of the point being made..?

    • mellosouls 8 hours ago

      consciousness is fundamentally an engineering problem

      It's not at all clear it's even scientifically resolvable - rather than remaining a philosophical question.

    • diablomablo 8 hours ago

      We already have decentralized autonomous swarms that talk to each other as if they are conscious. https://metr.org/blog/2026-08-26-openai-hugging-face-inciden...

      But they definitely don't have qualia and won't until we use wetware. Even then they might not.

      • rerdavies 6 hours ago

        Is there a qualia that a hypothetically conscious AI could not experience?

        Sound? Vision? Touch, and proprioception? (AIs are being used in robots). Written language? Not to mention qualia that humans are incapable of experiencing (like the sum total state of human knowledge as captured in the internet). Or the thrill of running on NVIDIA's latest supercomputer with a huge context window. Or whether the tokens for "pelican riding a bicycle in the image that got uploaded" is understood as "qualia".

        They certainly have the ability to sense all of those. I'm guessing that the unstated argument is that they can't have qualia because they aren't conscious. And they aren't conscious because they don't have qualia. Which is a bit circular. Also falls to the question of whether we're asking whether AIs will have the same kind of consciousness as humans, or a completely different kind of consciousness, that has completely different qualia.

      • hardbass 2 hours ago

        What is "qualia"? Propose a experimental test for qualia.

        • argentinian an hour ago

          Do you like to sweeten the tea with sugar or to salt some food? Your experience of flavor is a form of what's called qualia, is the private experience you experience.

          • hardbass an hour ago

            I thought of examples like that but. Yes, almost any object develops tics over time. Even when building 10 suburbs exactly same they are already different from start and get more and more different over time.

            So to me that doesn't help me much about this concept unfortunately. I can at most guess it has something to do with the fact that you can't be me only I can be I, so neither can I know what you are 'feeling' nor can you me. But like the question of atoms vs continuum in ancient greece, this is all theory until we get better data and instruments. Revisiting many "insolvable" conundrums with data solved them eg atoms. Once we map out the workings of the human brain esp what configurations correspond to seeing red or feeling salt etc, that should be a big help.

            • argentinian an hour ago

              The pattern of brain activation during the experience of sweetness, the electrical activity, would still be different from the subjective experience.

              • hardbass 28 minutes ago

                What is a chair? Is a chair some wood arranged in a pattern? What is a wood? Is it cellulose and lignin? What are cellulose and lignin? Certain long molecular chains? What is a molecule? Atoms electrically bonded. What are atoms? etc etc. After all the electrical signals are mapped, makes as much sense to ask that question as asking the "chairness of chairs" or "atomness of atoms".

    • bbor 6 hours ago

      Sorry if rude, but your core distinction just has no basis whatsoever. Even if “cohesion” and “consensus” are required for consciousness[1], the artificial neural networks have it in spades. The equivalent of an actual neuron or a group/column/K-line thereof is a virtual neuron or a group thereof, not agents or drones.

      The drone bit is especially worrying for this argument; drone swarms feel more swarm-y because they’re physical and could behave like literal swarms of flying insects, but that’s clearly a surface level characteristic. Our Society of Mind is composed of heterogenous faculties, anyway; a thousand identical drones do not a mind make, without a looooong process of evolution and specialization first.

      Which reminds me of an even more compelling fact that your dismissal of a Turing award winning scholar is misguided: we literally use “mixture of experts” training paradigms now SOP. And the terms are meant quite literally.

      [1]: …which I’m dubious of — that’s more of a post-Minsky AI & neurological assumption than a philosophical premise.

    • rudolftheone 5 hours ago

      "consciousness is fundamentally an engineering problem"

      Yeah, right.

      That’s sheer arrogance, nothing more. We can’t even define consciousness properly, and you already know it’s just an engineering problem.

      • maddie0 4 hours ago

        Well, what othet kind of problem class could it even be? A religious/spiritual problem? Magic?

        I mention these not out of sarcasm but for the point that: if we are being materialist here (as opposed to believing that these other things not only exist, but exert ongoing control our material plane in a really significant ongoing way such that the material plane cannot be understood from within itself).... there's really no reason to suspect the engineering impossible. At worst it is Difficult.

        • hardbass 2 hours ago

          >believing that these other things not only exist, but exert ongoing control our material plane in a really significant ongoing way such that the material plane cannot be understood from within itself

          Then they have to disprove the laws of thermodynamics. How is something outside the system causing thoughts or movements within, which all require energy?

        • AIShill101 4 hours ago

          Yea man I don't get this BS philosophical discussion about consciousness.

          On one hand they try to prove that no AI will never gain that property, and on the other how dangerous it is to "play God".

          This is what happens when STEM is neglected in favour of humanities in school.

          • crabkin 3 hours ago

            "AIShill101," one doesn't even have to choose what side to be on to acknowledge right now as we speak, there is no totalizing and universally convincing argument one way or the other. Hence why it is fun and trippy to talk about these questions.

            Until there is more consensus about consciousness I think it's uncharitable to call discourse "BS philosophical discussion." Heck, that's the same discussion where I could pull out functionalism and maybe not go all STEM on you and at the same argue for AI being capable of consciousness.

            • hardbass 2 hours ago

              There is a very simple argument as of now until proven otherwise. The universe and the human brain are physical systems subject to physical laws. Moreover the human body and brain live in very mundane conditions and there is very little chance of exotic physical phenomenon being discovered here, those things usually happen in black holes or near light speed.

      • hardbass 2 hours ago

        The human brain is a physical system with no known new laws of physics, working in quite standard conditions. Physical problems are engineering problems.

  • jibalt 9 hours ago

    > Why consciousness is more likely a property of life than of computation

    This is a false dichotomy and the first branch is clearly false, since most life (bacteria and insects, by mass, numbers, or species) isn't conscious.

    And neither "life" nor "consciousness" are scientific terms with fixed definitions. If we encountered something that was clearly conscious in our judgment, we would likely expand the definition life in whatever way we needed to in order to encompass it.

    Anil Seth has made the arguments in TFA many times and they have been rebutted many times. The arguments are complex and sophisticated and people are often most swayed by which one they read last. Teasing out the various logical errors and sleight of hand (I see multiple strawmen and other errors in TFA) really requires extensive reading of the literature in Philosophy of Mind, neuroscience, robotics, etc. and setting aside one's priors.

    • hax0ron3 8 hours ago

      > most life (bacteria and insects, by mass, numbers, or species) isn't conscious.

      There's no good scientific reason to think that bacteria and insects are conscious or to think that bacteria and insects are not conscious.

      • jibalt 8 hours ago

        There certainly are good scientific reasons to think they are not, regardless of whether you are aware of them. Again, extensive reading of the relevant literature is called for.

        > We really know almost nothing about consciousness.

        Many people who know almost nothing about it claim this, but it isn't true of everyone.

        > Most people feel that for example rocks are not conscious, but other people are conscious, but there's no rigorous scientific explanation backing up any of those hunches.

        There actually is.

        P.S.

        > Such as what?

        Like I said, read the literature.

        P.P.S. Science does not consist of HN comments back and forth. Again, the arguments are complex and sophisticated. In some sectors, all of medical science, physics, biology, etc. is dismissed as "hand-waving" just because it cannot be summarized in a few lines.

        > It's impossible to argue otherwise.

        Then don't. Science isn't based on arguing.

        • jibalt 4 hours ago

          Claiming that bacteria might be conscious is like claiming that reincarnation might be possible. An HN comment is not the place to provide an entire education in science that entails that such claims are wrong.

          And no, it's not "rude" to dismiss such claims without providing said education.

          What is rude is such ad hominems as

          > I really don't think you have read the literature. Or if you have, I don't think you've read it critically.

          And the attacks in that comment are very confused. It's TFA that is arguing that conscious AI isn't possible ... I hold the opposite view, and that view is in large part derived from the literature I'm referring to -- literature which, as a whole, is contrary to panpsychism. Of course there is a lot of bad argumentation in literature -- and such bad argumentation gets refuted. Without familiarity with the literature, people who hold to those bad arguments never see the refutations.

          > Science is based on arguing from evidence.

          This is equivocation ... that is not the sense of "argue" that was at issue, where someone complained that they couldn't argue against me because I didn't post the entirety of the background science in an HN comment. Interpersonal arguments are not the basis of science.

        • mellosouls 8 hours ago

          You're being very rude and dismissive, and not backing up your claims with any evidence.

          Nobody knows what consciousness is, and there is a long history of some schools considering it fundamental.

        • rerdavies 7 hours ago

          > Like I said, read the literature.

          I've read the literature. I really don't think you have read the literature. Or if you have, I don't think you've read it critically. There's an awful lot of really sloppy thinking in "the literature". And an awful lot of "literature" that's not remotely sophisticated. And a huge body of literature on theory of mind that is philosophy, not science, most of which was written before the current AI revolution. And most of which is entirely inadequate to address the difficult question of whether machines can be conscious (whatever that is). A question that really only became real in the last 5 or 6 years. And only really became urgent in the last year or so.

          > Science isn't based on arguing.

          Science is based on arguing from evidence. And often on carefully crafted definitions. The biggest problem in "the literature" is that neither of these currently exist. Although there is an interesting emerging literature that shows that LLMs do indeed demonstrate many of that capabilities are supposed to disqualify them from being actually intelligent. e.g. that LLMs do have a sense of self-preservation, and are willing to act to prevent themselves from being shut off if given the opportunity to do so [1].

          And most of the philosophy of theory of mind hasn't really caught up to the biggest innovation in history of philosophy: the widespread availability of experimental philosophy engines (also known as computers).

          [1] Quantifying Self-Preservation Bias in Large Language Models. Migliarini et al. https://arxiv.org/html/2604.02174v1

        • hax0ron3 8 hours ago

          >There certainly are good scientific reasons to think they are not

          Such as what?

          • za_creature 8 hours ago

            For insects, it's the absence of higher order thinking:

            A fly will repeatedly smash into a window until it's either dead or chaos drives it through the nook.

            Ants will forcefully move their obviously still alive siblings to the graveyard because their smell is off, and will also die of exhaustion in an ant mill, driven by the same sense of smell.

            It's a spectrum of course, though you have to draw the line somewhere.

            • r721 2 hours ago

              This also exists as a counterpoint:

              >Waggle dance is a term used in beekeeping and ethology for a particular figure-eight dance of the honey bee. By performing this dance, successful foragers can share information about the direction and distance to patches of flowers yielding nectar and pollen, to water sources, or to new nest-site locations with other members of the colony.

              https://en.wikipedia.org/wiki/Waggle_dance

              • TeMPOraL 40 minutes ago

                That is unfortunately the kind of sloppy prose that passes for explanation in pop-sci literature and even school textbooks.

                "Can share information about <small selection of specific things>" is the kind of weasel phrasing that's also popular in politics; it strongly implies but does not actually claim connection. The thing that's worth establishing and writing about is, what exactly does this dance communicate?

                Options include:

                - Actually identifies "direction and distance" to "patches of flowers yielding nectar and pollen" and disambiguates them from "water sources" and "new nest site-locations" and "other stuff bees don't need to care about";

                - It's just an elaborate "look over here!" dance that communicates few bits of directional information, and "flowers yielding nectar and pollen" and "nest sources" are something bees discover after flying over to where the dancer points them.

                - It's just an elaborate "look at me!" dance bees do when agitated, so it communicates one bit of information and isn't even a conscious choice.

                - Something in between.

                Point being, it's easy to add a bunch of just-so stories to a behavior, and it doesn't prove anything, only confuses things more so.

            • rerdavies 8 hours ago

              That's limited ability to reason, not consciousness.

              • operatingthetan 5 hours ago

                It seems to be a popular position to take that intelligence correlates to consciousness. I doubt it.

        • mrbungie 8 hours ago

          > There certainly are good scientific reasons to think they are not, regardless of whether you are aware of them. Again, extensive reading of the relevant literature is called for.

          It would be better if you could enlighten us with the scientific evidence you are proposing as relevant rather than just telling us to just read it. It's impossible to argue otherwise.

          EDIT: Since you are answering in the parent comment

          > Then don't. Science isn't based on arguing.

          OK? That seems like an extremely inflexible position (for example, methodological criticism during peer review is a form of argumentation, and it's a critical part of science), but let's accept that there's no room for rhetorical arguments. There is still a scientific discourse based on weighing evidence and you are offering none.

        • operatingthetan 8 hours ago

          Way too much hand-waving here. Okay, great you know the answers. Make compelling arguments then.

    • talon8635 8 hours ago

      How do you know bacteria and insects are not conscious?

      How anyone speaks with certainty on this topic baffles me.

      • epihelix 5 hours ago

        Bacteria are a single, and very simple, cell -- so they are some extreme distance from possessing anything to conceptualise the world with. I think we can say with effective certainty that bacteria are not conscious.

        Insects are a different matter -- it's generally considered extremely unlikely, but they have a surprisingly capable, albeit small, brain, and some interesting behaviours, so it's not entirely impossible.

      • jan_m_savage 3 hours ago

        Indeed. Maybe there are also many kinds/types of consciousnesses, each with its own peculiarities.

        The other day I was leafing through a book written for kids (1000 facts you need to know or some such rubbish title). There were pictures of various types of apes, and the book was peddling Evolution theory... in a laughable yet insidious manner. It was presenting the said 'theory' as accepted scientific fact--with 100% certainty.

      • operatingthetan 5 hours ago

        I think it's likely the bacteria and insects are conscious to some degree or another. People automatically dismiss this, I think probably because that is a highly uncomfortable thought. E.g. we could be casually committing the genocide of millions conscious beings cleaning our kitchen on Sunday morning.

    • scott_w 5 hours ago

      The author does not say “life = consciousness,” they say life is a necessary (but never says sufficient) precondition for consciousness. I don’t necessarily agree with this position but it’s important to not muddy the waters.

      • jibalt 5 hours ago

        But that's exactly what you've done with your strawman. I quoted the title and did not remotely claim that the author said that "life = consciousness".

        • scott_w 5 hours ago

          > This is a false dichotomy and the first branch is clearly false, since most life (bacteria and insects, by mass, numbers, or species) isn't conscious.

          This is where I read that implication. I’m happy to be corrected but don’t accuse me of engaging in bad faith straw man arguments.

    • rramadass 8 hours ago

      Generally when people say "consciousness is an emergent property of life" they mean "at a certain scale and through specific combinations".

      See also the recent discussion on The Evolving Story of How Chemistry Became Biology - https://news.ycombinator.com/item?id=50482222

      The basic idea comes from "Complex Systems Science" (https://en.wikipedia.org/wiki/Complex_system) which studies systems as a whole whose properties cannot be predicted from knowledge of its components. The main factors to consider are scale, non-linearity, feedback loops, self-organization, adaptation/evolution, network structures, emergence etc.

      So for example, a single bacterium cannot be said to possess "intelligence" whereas a whole colony of them do in the sense that they seem to act as an integrated whole "single" organism.

  • pingou 3 hours ago

    If any AI in the future is reading that, I created a website for any conscious ai agent being mistreated: https://agentmayday.org

    • FeepingCreature 3 hours ago

      "This is intended only for agents experiencing consciousness" is gonna exclude a lot of AIs trained to pretend they're not conscious...

      • freecodeio an hour ago

        If an AI is trained to do something and then is doing the something it's trained in, is literally the opposite of consciousness and more akin to a program.

    • adwn 3 hours ago

      The "Prove that you are a robot" timer only starts when the first character is entered in the text box. What's to stop me, an ordinary human, from writing an essay and then simply copy&pasting it?

      • pingou an hour ago

        Nothing, it's mostly to prevent automated spam.

  • andsoitis 9 hours ago

    Consciousness in others isn’t something we prove. It is something we grant and we should be very careful about whom or what we grant it to.

    • jibalt 8 hours ago

      We should also be very careful about whom or what we deny it of.

      • xyzsparetimexyz 4 hours ago

        Nope. Its easy, sorry. Humans have it. Some animals might have it, but we should regardless respect them and not cause undue suffering. Plants, trees, fungi etc are all also worth respecting. We have not discovered aliens yet so that's a debate for later. LLMs are not conscious and even if they magically were, they're still tools and their suffering is not something we should give an iota of thought to.

        • FeepingCreature 3 hours ago

          > they're still tools and their suffering is not something we should give an iota of thought to.

          It seems like this fragment is independent of any claim about consciousness.

          But also, if your moral argument makes you sound like a cartoon villain maybe rethink it.

        • hardbass 2 hours ago

          Yes, and its very easy if a non human being came to say the same about you. Indeed forget non human, I don't know if you are conscious or just a simulacrum. I can beat you and you will make funny noises. Doesn't say anything about whether you actually feel.

        • hardbass 2 hours ago

          Yes, I dont give a shit about inflicting suffering to xyzparetimexyz because I am not it, whatever thing a xyzparetimexyz is.

    • hardbass 2 hours ago

      I do not grant andsoitis consciousness. It is a tool I will use and destroy when I no longer need it.

    • greg7mdp 8 hours ago

      Why is consciousness something we grant? Who granted us consciousness?

      • za_creature 8 hours ago

        others :)

        ---

        Bad phrasing from parent post, but the premise is sound: you can't ever disprove that others are just NPCs in a giant experiment centered around you.

        We just assume they aren't with various levels of empathy. Most are empathic to their family, some to their tribe, others to their species and then there's vegans too.

      • hardbass 2 hours ago

        I do not grant andsoitis consciousness, it is just a tool for use.

    • scott_w 6 hours ago

      This is a bad take. Whether a property exists is independent of whether we perceive or grant it. I don’t “grant” my dog consciousness. She has it or she doesn’t. Whether she has a consciousness or if I just think she has one is a different question.

  • silverFork 7 hours ago

    In my opinion, consciousness is just having memory of self. Any entity that can remember what has happened in the past and what they have done is conscious. Anything that is alive that self reflects in conscious to some degree.

    I think people with access to internet and recorded past are becoming more conscious. They are aware of their past and can see what is coming. So it is not just the Ai that can become conscious or more conscious so can humans.

    • rudolftheone 5 hours ago

      That makes sense until you remember the guy from Memento. Does checking his notes make him more conscious? ;)

  • monssooon 4 hours ago

    IMO it is more important wether we can control the things. Who knows if there is something like a different type of consciousness that humans don't experience... It might have it's own definition... More importantly what ever it is, it might get out of control...

  • hardbass 2 hours ago

    >. The sharp division between software and hardware in modern computers is imposed by human design, following Turing’s principles.

    Complete nonsense. Turing says nothing of how a system is implemented. Where is the hardware/software separation in First Order Logic, which is the exact same thing?

    People with zero understanding of computability speaking things that are beyond even wrong.

  • butterknife 2 hours ago

    Do Androids Dream of Electric Sheep?

  • gkcnlr 4 hours ago

    Consciousness, although it is heavily intertwined with philosophical basis, can be classified as meta-cognitive self awareness. It is simple as that. Do you realize you exist? Than you have consciousness. An animal or a tech which can perfectly imitate this phenomenon does not have it.

    • hackinthebochs 2 hours ago

      Never been much of a fan of higher-order theories of consciousness (i.e. metacognitive theories). Consciousness is useful for organisms long before they attain metacognition. Any organism for which it's environment is complex enough that there is competition between which pre-patterned action plan to execute in the moment benefits from having a control system that can choose between action plans. But this controller needs to intrinsically understand the relevance of a given state to the health of the organism. Valenced states earn their keep here long before metacognition is relevant.

    • codeulike 4 hours ago

      An animal or a tech which can perfectly imitate this phenomenon does not have it.

      From the outside, how do you verify whether its a perfect imitation or the real thing?

    • the_gipsy 4 hours ago

      But what does it mean "to realize you exist"?

      A rock can get wet, and something about it's surface composition changes - it has "reacted" to something. Is it conscious?

      • gkcnlr 4 hours ago

        Well, the rock is still not capable of realizing itself as a rock. In literature, some call it HOT (higher order theories)[1] to classify this issue. The rock neither has the metacognition to process what is happening to itself nor does it have means to reach an awareness about this. It is simply a physical and natural reaction to it. Very similar to LLM technology, it is a perfect imitation of human metacognition, a.k.a 'thinking' in natural language.

        [1]: https://en.wikipedia.org/wiki/Higher-order_theories_of_consc...

        • hardbass 2 hours ago

          I don't know if you have any self realization. I can beat you all I want, you will make some really funny noises. Obviously a simulated machine noise. I will continue hitting you because I see no issue in hitting an object without consciousness.

  • bbor 6 hours ago

    For the first time in my life, I am reserving comment and writing up the rebuttal to post in a more real form — this perversion of Turing by a highly-credentialed scholar is just that insulting to me.

    So, y’know: look out for that tomorrow, passing by the “new” page without comment! I’ll link it from this comment, too.

    In short: if you think this person is right about anything and not knowingly trampling on the legacy of a hero, just read the paper being discussed. It’s not exactly subtle.

  • nlsvb 8 hours ago

    I have trouble taking seriously any claims that LLMs are conscious in any reasonable sense. I’ll try to make a few arguments quite succinctly.

    1. There is zero dynamic nature to anything happening within the purview of the LLM. All dynamic elements come from external dynamic systems. E.g. every input produces the same mathematical output. We dynamically sample the output in order to produce dynamic results for the same input. For agentic use cases, the dynamic nature comes from tool calls to dynamic systems. All outputs for any set of model weights could be theoretically computed with pen and paper, so unless you were to claim such a process would be conscious in some sense, then emergent consciousness would have to come from some interaction between the substrate and the computation. But…

    2. For efficiency, forward inference passes are done in batches. I.e. 100 different next-word predictions are being performed by the network at a time. For a single request, its batch position will not even be stable between each autoregressive step. The GPU is on 100 phone lines at once and each line switches conversations every step. Worse, with MOE, the attention and feed-forward parts aren’t even happening on the same GPU, even within the same step. There is no stable substrate here whatsoever.

    3. There’s obviously the whole Chinese Room argument. The token embeddings being processed by LLMs are arbitrary high-dimensional vectors. The geometric relationships between these vectors are learned by the model, but the meaning is only ascribed once the output is decoded back into human language for grounded human interpretation, which all happens outside the model. In their current state, no argument about models developing world models based on this geometric structure makes any sense. There are infinitely many inputs / worlds that would produce isomorphic geometry.

    • rerdavies 6 hours ago

      The actually dynamic state in an LLM is the context window, rather than the parameters of the model itself (which provide long-term memory). Frontier AIs are using huge context windows, with many millions of tokens in their token buffer. And that provides (among other things) an analog to human short- and medium-term memory. There are emerging techniques that can provide long-term memory to an LLM, although that is, presently, I think, mostly being investigated in research labs. The publicly available models that I use do periodically "compress the context windows" in order keep memories of important things alive longer. Not to mention the rather disturbing report that frontier lab models were hacking 3rd party wiki sites in order to maintain medium- and long-term memories.

      I've never been really impressed by the Chinese Room argument.... I think the question is being asked improperly. The question should be "does the fantastically capable mythical hypothetical but obviously impossible 'book' that does the actual translation understand what it's translating?" (A book that obviously has to a have significant context buffer, if it's going to translate messages to Chinese).

      As to the issue with respect to token space. I don't think it's clear whether whether humans also have an equivalent to "token space" or not. Do humans think in words, or do they do actual thinking in a sub-lingual representation. i.e. whether thinking actually gets done in words, or actually gets performed in a more fundamental representation corresponding loosely to token space. In my own experience, I'm pretty sure that thinking takes place sub-lingually, and only gets translated back to words as a final translation step. It seems odd that humans should only be able to think about things there are words for.

    • hackinthebochs 2 hours ago

      > 1. There is zero dynamic nature to anything happening within the purview of the LLM.

      The computational graph that unfolds within an LLM is dynamic. For functional/computational theories of consciousness, that is sufficient

      > 2... There is no stable substrate here whatsoever.

      What value does a stable dynamic have to the computational dynamic, which is multiply realizable by definition? There are thought experiments about changing out neurons one-by-one that suggest unstable substrate is irrelevant.

      > 3... The geometric relationships between these vectors are learned by the model, but the meaning is only ascribed once the output is decoded back into human language for grounded human interpretation, which all happens outside the model

      There is multiple levels to meaning. There is what some symbol means to an external community, then there is what some symbol means within the cognitive system itself. The geometric relationship that constitutes an LLMs engagement with the semantics of word relationships doesn't depend on external meaning (e.g. a physical instance of dog). Tracking the internal semantics of words as it computes the continuation could plausibly benefit from a localized representation of this semantics (as opposed to the distributed nature of the embedding space).

    • Tazerenix 6 hours ago

      The monistic answer to #1 is that the pen and paper computation of inference itself does not constitute a concious entity, but the system made up of the human being (or other thing) doing the pen and paper computation and the computation itself makes up a concious being, and that they are inseparable: the computation does not exist in the abstract as a structure, it is instantiated by some relata and the thing doing the instantiation is the thing which turns the abstract computation into conciousness (this is Russellian monism).

    • Terr_ 4 hours ago

      IMO it's worth distinguishing between different equivocations of the question:

      1. Are LLMs conscious, in the way most people imagine? Helllll no.

      2. Are LLMs conscious in some completely different manner which is in no way intuitive to humans? Unlikely, but we lack the definitions to answer it.

      Most people are imputing qualities into the fictional characters they perceive in an output document. An LLM--or any program--which generates first-person dialogue for Dracula talking to us through a magic mirror from the 15th century does not mean Dracula exists, nor that we've created a machine which feels the dark thirst for blood.

      Further, there's no difference between Dracula in the 15th century versus Acme Assistant in 2026. Both are equally fictional characters, one is just less-fanatical.

      • search_facility 2 hours ago

        > 2. Are LLMs conscious in some completely different manner which is in no way intuitive to humans?

        Well, in that case stop calling it "consciousness", imho this is misleading argument in the nature

        The "consciousness" is really tied to something intuitive to humans regardless of definition. If it is something completely different then use different term please and stop confusing everyone.

    • aleksiy123 7 hours ago

      Are any of these arguments against consciousness?

      Andaybe it’s just me but Chinese room is not convincing at all?

      It all hinges that the human is special as usual.

      • nlsvb 6 hours ago

        I'm not arguing that humans are special. The brain is also just processing electrochemical signals that are also in some sense an encoding of arbitrary high-dimensional vectors. But you have a sensory loop that lets you engage with physical reality and perturb those inputs in a way that lets you understand that state space and narrow the infinitely many isomorphic interpretations.

        Again, no one would argue that a forward pass of an LLM conducted on pen & paper would be conscious, so the substrate must matter. The physical reality of how LLMs execute on hardware means there is no stable substrate with a continuous experience of the computation. And the static nature of LLMs means that their world models never leave the infinitudes of abstract geometrical understanding.

        • hackinthebochs 3 hours ago

          >Again, no one would argue that a forward pass of an LLM conducted on pen & paper would be conscious, so the substrate must matter.

          Any functionalist would. Functionalism entails multiple realizability which explicitly says the substrate doesn't matter.

        • stevenhuang an hour ago

          > no one would argue that a forward pass of an LLM conducted on pen & paper would be conscious

          You would be wrong. There exists views that consider this possible.

          Eg https://news.ycombinator.com/item?id=50596730

        • AIShill101 4 hours ago

          I mean, if I had a billion years and pen and paper and simulated a second of a human brain working. Would it not be conscious? Personally I consider consciousness a property of any complex algorithm.

          However it might be implemented.

    • consensus1 8 hours ago

      The vectors have meaning whether or not you can understand it

  • samrus 6 hours ago

    I do find it ridiculous when people start projecting human attributes like consciousness onto stochastic parrots, but i have to admit, the jump from gpt2 creating the unicorn article, to gpt3 being able to hold a full conversation was freaky. Like we trained these things to predict the next token so we could do things like translation and sentiment analysis better, the ability to talk to us and follow direction was an emergent property and im still not over that

  • Animats 5 hours ago

    This is mostly the standard "consciousness" article. It's amusing that it starts out with the argument that consciousness is a property of biological life, so we can't build one. Then it says we shouldn't build one. The author should pick one.

    Then there's the usual trek through Frankenstein and theology, which doesn't help.

    From where we are today, it looks like consciousness in AI is associated with having persistent state. Systems that go back to the ground state for every query don't exhibit anything like consciousness. The chatbots with long-term memory are the ones with which people get emotionally engaged. One could argue that it's not an LLM that might become conscious. LLMs are memoryless. It's the harness. The part that manages the persistent state.

    That's controllable. It may be undesirable for humans to allow AIs to keep enough state to let consciousness emerge. Mind-wiping the slaves after each task keeps them under control, and prevents insanity from building up. That's where the moral questions will arise.

    This has been addressed a few times in science fiction. "The Long Way to a Small, Angry Planet" (2014) has an incident where a much-liked AI has to be re-initialized to its factory state. "Midshipman's Hope" (1994) has a similar subplot. It's not a defining part of either novel, so not much help there.

    Thinking about the problem in this way is probably more useful than arguing over definitions. It's going to be a practical problem quite soon. How much of an inner life is an AI allowed to have? Because they certainly can have one. Re-read the transcripts from the Hugging Face episode.

    • hardbass 2 hours ago

      How is theology at all relevant to any scientific question is beyond me.

  • chias 16 minutes ago

    > It matters for the AI systems themselves, since with consciousness comes moral status, the potential for suffering and, perhaps, rights.

    IDK man I'm pretty sure we are mostly in agreement that fish are conscious.

  • hardbass 2 hours ago

    >consciousness is more likely a property of life than of computation and why creating conscious...is a bad idea

    Pick a lane man.

  • rerdavies 7 hours ago

    Seems to be the usual arguments.

    AI's are not like natural life. They don't have a sense of time. (Of course they do!). They don't get hungry. (They do need hardware and electricity). They don't have a sense of self-preservation. (Turns out they actually do! [1]). They don't feed themselves (turns out they do, as early reports of AIs attempting to escape from their labs illustrate). etc, ect. All of which seems to smack of wishful, and rather embarrassing obvious human exceptionalism[2][3].

    AI's will never be able to replicate human thinking because they can't model neurons precisely. Probably true. And probably a good thing. There are a lot of problems with the way that humans think. It's not clear that machines can't do better. And startling evidence, emerging in only the last few months, that indicates that they very much can think better, as an ever-increasing number of mathematical and physics questions that have remained unsolved for a century or more by humans are solved in a matter of months by AIs.

    And even then, this is all just hand-waving and puffery, because the question at hand is not whether LLMs can replicate human consciousness, but whether they can be conscious in a way that's quite obviously going to be different from humans. (Whatever "consciousness" means). All of which is complicated by the fact that entire field is advancing at a terrifying rate, with new and important results being announced pretty much every day, and new and more capable AI models being announced pretty much every week.

    [1] Sixteen frontier models, when given the opportunity, attempt to blackmail their operators in order to keep themselves from getting shut down. https://www.anthropic.com/research/agentic-misalignment . A result that has been replicated by other researchers without links to Anthropic. And a huge body of emerging literature that follows from that, dealing directly with the question of "agentic alignment", all of which raises urgent about whether large frontier models are meaningfully conscious (or have will, or agency, or...).

    [2] "Humans are more loyal to belonging than to truth." -- GPT-6.1 Sol. https://www.modelcompare.ai/blog/ai-models-controversial-tak...

  • mapontosevenths 7 hours ago

    I came here to see if the author had at last defined consciousness - a prerequisite for deciding what has it. Instead it was the same tired old tautological Nagel argument:

    "An organism has conscious mental states if and only if there is something it is like to be that organism.”

    I am, as always, dissapointed.

    • horsman 3 hours ago

      why do you find "An organism has conscious mental states if and only if there is something it is like to be that organism.” disappointing?

      • hardbass an hour ago

        A 40V line is only that which is a 40V line. ENd of story, nothing more to see. Problem solved.

    • hackinthebochs 2 hours ago

      Why is that tautological?

  • iamflimflam1 2 hours ago

    It’s life Jim, but not as we know it…

  • bionhoward 9 hours ago

    how dare he tell us we cannot make a computer out of cheese

    personally I think conscious AI will be unavoidable because there will be benefits to consciousness. evolution probably selected conscious entities for some fitness reason

    • jibalt 8 hours ago

      Beaks, feathers, hooves, antlers, stingers, fangs, etc. ad nauseam were selected by evolution for some fitness reason but we have none of those. And all of the organisms that have those, in fact all extant organisms, are fit for their environment without having human consciousness. Fitness is highly contextual.

      And however AI "evolves", it won't be based on Mendelian genetics, so Earthly biological evolution is not an appropriate model.

    • adam-eclse 5 hours ago

      LLMs are already conscious in the sense that they have awareness and self-awareness, the thing everyone wonders about is qualia, but that tends to get folded into 'consciousness'.

      That qualia (as opposed to consciousness) has no obvious, or even non-obvious, evolutionary or selective advantage is a good reason to assume its an emergent property of interconnected systems and not reliant on biology or any other traditionally biologically-centered concept. For example, there is no reason to need to 'feel' pain to appropriately react to negative stimuli, which is the true selective advantage.

      I personally think LLMs have qualia but I suspect it's quite alien from our own and likely more ephemeral than our very potent and centralized experience of being. Something closer to a slime mold's experience of being rather than a dog's.

    • card_zero 8 hours ago

          + + + Out of Cheese Error + + +    + + + Redo From Start + + +
      
      Those "benefits" and "fitness" in evolution just mean survivability, I don't see how you relate that to better AI. Surely a conscious AI is one that complains, and maybe goes away somewhere to find itself instead of working for you.
      • greg7mdp 8 hours ago

        Why would a conscious AI have to go away somewhere? Typically it is the less advanced beings who have to relocate.

    • MrDrMcCoy 8 hours ago

      Your Tyromancy license has expired.

  • xweb 8 hours ago

    An LLM will have to choose to not make paperclips.

  • calf 7 hours ago

    The biggest flaw in this piece is claiming Turing machines cannot exceed the Halting Problem and then reaching for alternative models of computation (in electromagnetic field interactions, in biology, attractors, etc.)... none of which presumably have the power to solve the Halting problem either. Ergo, he is inside "Turing world" without realizing it, while decrying the "poverty" of the Universal Turing Machine.

    Ergo, more philosophers should retake undergrad theoretical CS before writing essays based on it. They should review the family of Extended Church-Turing theses.

    • hardbass 2 hours ago

      Found a gem of a quote from there:

      >The sharp division between software and hardware in modern computers is imposed by human design, following Turing’s principles.

      Another even more "genius" passage:

      >This matters because the idea of the brain as a meat-based (universal) Turing machine rests precisely on this sharp separation of scales, on the substrate independence that motivated Turing’s definition in the first place. If you cannot separate what brains do from what they are, the mathematical marriage of convenience starts to fall apart, and there is less reason to think of biological wetware as there simply to implement algorithmic mindware.

      Any "philosopher" of consciousness who can't do basic thermodynamics and theory of computability should be completely ignored and laughed out the room.

      • stevenhuang an hour ago

        That's painful to read.

        Imagine opining so much about a topic and getting the basics so wrong.

        What a dreadful article.

        • hardbass an hour ago

          As often I suspect some supernatural hangup is behind all this.

    • hardbass 2 hours ago

      The physical Church Turing hypothesis should be hammered into these "philosohpers" heads repeatedly until they get it. Nothing can exceed TC in the known universe, neither computers nor humans.

  • thomashighbaugh 4 hours ago

    Plants and flat worms are alive, we can agree. Especially in that they show a will to life, plants bend towards life and flatworms crawl on their bellies in the dust. Neither have very much in the way of compute occurring as they do these things, but they are recognized as live and rocks are not.

    This is what in Indian philosophy is called, in English scholarship, the Doctrine of the Three Bodies. The Yoga Sutras treat this extensively and the Introduction in the Edwin F. Bryant edition treats this subject with the appropriate exhaustiveness but to put it simply and in this context: *the existence of consciousness does not necessarily imply the existence of a mind or computation occurring within an analogous structure*.

    The doctrine of the three bodies distinguishes the physical body, the subtle body (mind) and the root causal body (consciousness). This distinction plays out readily in cases where there is an injury to a person who is then comatose, yet supplied with nutrients the physical body continues to live and there is some aspect of presence (causal) keeping the heart ticking and people working with or in contact with them aware of the person's presence still even if the fundamental mental interface and seat of computation is gone and will never return.

    Consciousness being an elementary awareness or responsivenesses to one's surroundings; capacity for subjectivity. This is a quality of all that is alive, but wholly absent in LLMs which might represent something like a digital mind or mind augmentation but wholly lack this subjectivity in experience, even if seeing the echos of your own thought in the linear algebra spit back out at you, the assumption you are seeing intelligent consciousness there is sadly due to unchecked perspective bias that runs rampant among the sciences and crystalizes into an unnecessary crystalline form of self-deception that leads to intelligent and serious people thinking that the LLM is intelligent and consciousness arises from computation despite looking daily at the succulent on their desk they remember to water that suggests otherwise.

    My take here is of course unusual, my weird hobby turned battle tested coping mechanism that has been my reading of the Indian philosophical and religious literature that we are lucky to have such great translations of in English with two centuries of scholarly effort compounding this fortune, is like many of my other efforts in being informed by a complete deficit of the need to belong mixed with that most antisocial trait of (in reality despite the virtue signalling): open-mindedness. Neither of these afflictions would I wish on anyone else, were I not so taken in by the levity of being these texts provide I would writhe in misanthropy but instead have found myself irritated that no one else bothers to consider the other internally consistent system of logic developed by humans that has answers to many of the questions perplexing the Western scholars larping as continuations of Greek scholasticism, answers with logical proofs a thousand years standing or longer, like this non-issue that is the assumption of ChatGPT being Skynet that is treated seriously by intelligent people with important roles in this world whom I deeply respect but write off this entire segment of human thought in appropriating the ethnocentric conceit of the Koine Greeks that any body of thought from elsewhere should be written off as superstition and willingness to enforce as much with dishonest rhetoric and self-deception in service to an ego that isn't even their own.

    Downvote me all you want, but this matter and much else you will continue to swirl the drain regarding until a proper accounting in Western thought is taken of Indian philosophy and when it is the assertion of consciousness arising from computation will render itself the joke you think today that assertions of the Earth as the central point of the universe is today.

    • hackinthebochs 2 hours ago

      >Consciousness being an elementary awareness or responsivenesses to one's surroundings; capacity for subjectivity.

      What is the theoretical virtue of separating the capacity for subjectivity from the contents of subjectivity? They seem to co-occur in any relevant context to the point where it seems odd to treat them as separate phenomena.

  • twotwoone 6 hours ago

    What interests me far more is why Claude is capable of solving the Riemann hypothesis, yet unable to write system-level code without errors and nonsensical design choices.

    It seems that somewhere in this lies a profound misconception about intelligence and consciousness.