> Like everyone, I’m tremendously impressed by the lifelikeness of these systems compared to the history of earlier AI. But the disposition of some philosophers to take seriously the question whether these things are conscious or might soon become conscious is just ridiculous. They are assuming that the physical substrate is irrelevant. They all think that the material conditions for consciousness can be given in functionalist terms.
> (...)
> Functionalism is substrate-independent. It’s more than likely that the truth is one which links physical processes and systems of certain kinds to mental phenomena. Complicated data centers have absolutely nothing in common with the physical character of a nervous system. We have strong reasons to believe that while there may be consciousness elsewhere in the universe, it would require the appearance of some kind of organism with something analogous to a nervous system.
Why should we assume our substrate is the "right" substrate for truly conscious beings?
Perhaps we have fake consciousness too (just like programs running on silicon), and some advanced alien being could look at our biological neurons stuff and immediately recognize our fakeness. Perhaps those aliens could proceed to assert that their own substrate is the only one that could possibly be conscious - all the other substrates (like ours) being fake.
We couldn't possibly know, because we don't know what consciousness is.
> Why should we assume our substrate is the "right" substrate for truly conscious beings?
No one assumes that it is the only right substrate. But if you don't believe we are a right substrate, are you waiving your own consciousness? Seems like you are, which does make me question how coherently you can argue about this.
> We couldn't possibly know, because we don't know what consciousness is.
We don't know it's deepest mechanisms. That also holds true for the universe. Doesn't mean we can't see the difference in dynamics.
Simply put, consciousness cannot be turned off and on. No definition of consciousness holds up to a system that can turn off, then be turned on 100 years later to carry on generating text as it was.
Evolutionarily, such a thing would be quite idiotic and maladaptive. It's not feasible to be an "agent" in the world when you are, by default, off, and require a prompt somewhere upstream to come awake.
Biology, on the other hand, is exquisitely timed. Each cell (including each neuron) is a self-sustained oscillator, with oscillatory rhythms that impact behavior, learning and memory. These clocks are endogenous, meaning they continue as long as long as the organism is alive, even without external periodic signals. LLMs have no such endogenous self-sustained osciallation operating continuously.
Go back to before the LLM's are conscious hype started. Find me one definition or example of consciousness that can be switched off. And no, neither sleep nor anesthesia count. They're different states of consciousness, but the brain and body are not "off".
Nor will any definition of consciousness pre-LLMs even countenance the idea that a frozen set of weights is conscious. The very idea of consciousness without continual learning is utter stupidity. All the sadder that it's spread so wide and has already cost us trillions.
Unless you believe the molecular state of the body cannot be in theory ever recorded and resynthesized, how is it not possible in this universe to turn humans or other animals on and off?
Obviously its impossible right now. But let me get this clear are you saying its impossible as in say putting a lander on Pluto (impossible today, no theoretical roadblocks to be able to do it in future) or physically impossible. After all (morality aside) one can simply slice off molecules from the body using eg a similar method as atomic microscopy. You could argue that the information might be hard to read, or there might be a proof that its impossible to read without altering it appreciably (quite possible), but we would need such a propf first, thats the only possible argument you can give. Obviously its aburd to say the molecules int the body are different from the same molecules synthesized elsewhere.
The amount of precision depends on how sensitive the body and brain is, eg we don't need to go to qm to design or copy a car or motor.
I'm saying it is physically impossible. And it's not some vague theory. It's pretty darn well established.
> one can simply slice off molecules from the body using eg a similar method as atomic microscopy.
This sentence doesn't make sense. Atomic force microscopy (which I assume is what you're referring to) doesn't itself slice any molecules from the body. It can be used to study molecules or cells ripped from the body, sure. In the process, the molecular state of the body you wanted to copy changes, so even if you reconstitute the bit you got out and put under the microscope (itself a big if, since no known microscope can give you exact quantity and type of every molecule in your sample), that bit is already in a different state, so you're not reconstiting the state of that segment of the body, let alone the entire body.
A human body has about 36 trillion human cells. About an equal number of bacterial, archeal and unicellular eukaryotic cells (bacteria dominate this cluster). Together, they have about an octillion molecules. 99% of that is water, but the dynamics of that water make the Naveir stokes equation feel like the two-times multiplication table in comparison.
The rest is considerably harder. Proteins, for example, aren't floating around bearing the frozen crystal structures AlphaFold can so quickly solve. Instead, dissolved in all that water, protein molecules cycle through all kinds of unstable confirmational states, sometimes for very brief instances, which are nonetheless functionally critical, and part of the state description of any cell. But wait, in the seconds it took you to read the previous two sentences, a significant chunk of those proteins have gone through multiple confirmational changes. And somehow you're going to find and read the state of every protein in a picosecond? There's about a 100 sextillion of those in your body.
So which moment are you planning to copy? And how? Any read attempt is going to be destructive, especially to these fragile confirmational states. Freeze the cell and those states shift. Break up the cell to measure it, and those states shift. Use light to image it, and you can image a subset of proteins but either need to modify them, or kill and fix the cell first, and in either case, sufficient light to illuminate every part of the body in an instant to be read by whatever detector you design still won't get you what you want, because guess what, light changes the proteins too, and enough will destroy them.
Then consider lipids, metabolites, small peptides, and small RNAs (which also do all kinds of confirmational hijinks second to second)... None of these can be fully measured body wide for you to clone the body's state.
It's just not even within the realm of plausibility to do what you're suggesting. Even a simulation that's accurate down to atomic resolution, if we someday invent computers that can do it, will drift from the actual system really really rapidly.
The body isn't a car or a motor. It doesn't have their simple regular structure. And literally tiny molecular changes have huge impact on overall organism state. Thinking about the body like it's some engineered machine is probably why you fail to see that even though it is made of entirely boring matter that's pretty common in the universe, what separates it from non-life is the depth and complexity of it's dynamics. And no, no LLM comes remotely close to the complexity of a bacterium let alone a human.
Yes I meant a probr that reads a molecule then breaks off its bonds to check the further inside the object. As I said before I am quite open to any proof along the lines of its impossible to measure a particular state without appreciably changing it due to sensitvity arguments. It might be true but unless we have a proof I can't say either way.
Completely engineered systems can be unpredictable without fully solving it eg chaos, I don't have any complaints about that as a line of defense.
Yes some kind of mathematical proof that any measurement of this system by the time it had taken place has changed the system appreciably in a way that can't be inferred back or accounted for.
That's a physical proof, not mathematical. You're not going to derive this fact from abstraction, but from actual empirical work.
But if you want to, go right ahead and ask an AI model if what you said is possible. Maybe it'll find the right math to convince you that physical reality is not susceptible to instantaneous cloning of states.
I don't think current ai's are able to do this kind of math unguided. If you are so sure of it you should have a proof ready. I am not against this fact being possibly disproven but it has to be an actual proof using arguments of information theory/dynamical systems etc
> We have strong reasons to believe that while there may be consciousness elsewhere in the universe, it would require the appearance of some kind of organism with something analogous to a nervous system.
At least he gets to the point quickly. But I'd like to know what those "strong reasons" are.
> Perhaps we have fake consciousness too (just like programs running on silicon), and some advanced alien being could look at our biological neurons stuff and immediately recognize our fakeness.
"That’s ridiculous. How can meat make a machine? You’re asking me to believe in sentient meat."
> We couldn't possibly know, because we don't know what consciousness is.
Great minds discuss ideas, average minds discuss events, small minds discuss people, philosphers discuss "consciousness".
Ok I'm on the total opposite of this guy, I am almost tempted to write a short letter telling him why Thomas Nagel, famous American philosopher, is wrong on this.
Basically his opinion is derivative of stochastic parrot theory. He admits as much in the interview. (Since stochastic parrot theory is wrong, I could stop right here, and so should he if he had a truly sufficient argument in the first place...)
He is committing the mistake as Scott Aaronson says, justa-ism. It's "just a language model" therefore it cannot possibly really be intelligent/alive/conscious/goalpostmoving. Followed by some tricky rhetorical move appealing to human exceptionalism.
The fact is that philosophers would do well to take computational complexity theory, the one involving actual math homework. All Nagel's woo-woo talk about a substrate and a need for an "analogous nervous system" shows yet again how CS as a nascent field, and theoretical CS in particular, has not fully been internalized by other disciplines.
Nagel says a bat is conscious. But an LLM with trillion dimensional neuron parameters, obtained through stochastic gradient descent, cannot possibly conscious because it is on the wrong substrate? Really?
In fact it is the other way around. The true "substrate" is the neural network model, of which humans (and bats) are merely one example of. That is the correct way to understand what LLM computation is. The abstract LLM neural net is what Alan Turing himself envisioned as the alternative to classical computation.
Yeah, I wouldn't necessarily call LLMs conscious, but I'm reminded of an argument that I heard once that computers could never hold consciousness because they don't process things in parallel. There's a lot of incurious reasoning abound when it comes to commentary on this topic.
Its absurd in this day an age that "philosophers" talking about consciousness who don't address at least thermodynamics, computation theory and information theory should not just be immediately laughed out of the room.
If they cannot at least acknowledge that the universe is a Turing computer, all Turing computers have equal power, that information has an energy cost so some special immaterial dual soul will have to violate thermodynamics to generate thoughts and actions, their arguments should be dismissed.
I wouldn't assume the universe is a Turing machine. The holographic theory does suggest that it has limited state in any given space, but that doesn't limit or to finite state transitions, or to finite space.
We can't see enough of space to know whether it's infinite.
I just want someone to define "consciousness" before they start arguing who can and cannot have it.
If the definition is "the processes of a biological brain" then clearly a computer cannot have it. It's also not very useful.
Turing machines have no trouble being infinite. That the universe is a tm comes from the fact that as of yet there is no knownaw that enables hypercomputation.
>theoretical CS in particular, has not fully been internalized by other disciplines.
Whereas it appears theoretical computer Science sometimes forgets there are many routes to computing per its own fundamentals, and certainly ignores the fact that Neuroscience didn't stop the day McCulloch and Pitts first conceived of the artificial neuron.
> But an LLM with trillion dimensional neuron parameters, obtained through stochastic gradient descent, cannot possibly conscious because it is on the wrong substrate? Really?
Yes really. You seem to think the word "trillion" should awe us. It's hard to understand why. Care to tell us?
Critically, whether a trillion or a significantly larger number, LLMs parameters have characteristics that make talk of consciousness absurd. To whit: they are frozen, and between two forward passes, a million years can pass and the model wouldn't know.
> In fact it is the other way around. The true "substrate" is the neural network model, of which humans (and bats) are merely one example of. That is the correct way to understand what LLM computation is. The abstract LLM neural net is what Alan Turing himself envisioned as the alternative to classical computation.
Here's just one problem with this theory. All neurons are 24-h oscillators, and have shorter oscialltions nested under this. A fact neither Turin nor McCullough nor Pitts could know when they were trying to imagine how the brain works. They assumed a global clock, which is absolutely false in biology.
And this rhythmicity is evolutionarily older than the rise of neurons. Cyanobacteria have 24 hour rhythms (though using different components), and can anticipate seasonal changes by this system, and first evolved this 2.5 billion years ago.
The issue isn't just substrate. Some fundamental comoutstuonal features of cells, neurons, brains and bodies need to be thoroughly ignored to land on a claim that LLMs are the true substrate of which human brains are an expression.
And also... Have you heard of astrocytes? Roughly as many of those as neurons in the brain, and they're part of every synapse. If artificial neural nets are the formula for consciousness, bats and humans sure seem to spend a lot on these "useless" cells, no? Interestingly, astrocyte size and complexity goes up as you go from mouse to human, and implanting human astrocytes in mice enhances their intelligence. One heck of an odd result to explain if it's all LLM like dynamics in the brain.
First, it is bullshit sophistry to accuse TCS to "appear to forget fundamentals" when barely anyone studies TCS in its form TODAY. It is also an tu quoque ad hominem and does not deserve further response. The fact that Nagel in THIS interview has zero understanding of formal computability and complexity theory, TWO SEMESTERS of niche undergrad specialization, is like talking about biology while NEVER taking genetics. It is unacceptable in the 21st century. They say, "stay in your lane", well, with TCS concepts/intuitions, it's more like the whole highway needs an upgrade.
I shouldn't CARE to tell you the significance of a trillion dimensions. The whole point is, if you TOOK modern complexity/computability you would understand the it is the OPPOSITE of "awe". The whole moral of the story is that THERE IS NO COMPUTING MAGIC. Turing equivalence and the Church Turing thesis* being the only sane interpretation of reality. If you believe brute scaling is insufficient then you violate the Church Turing thesis. It is a negative thesis, a constraint on our world. (n.b. I am handwavily referring to the class of extended Church-Turing theses per Aaronson, please do not confuse this.)
Again, the fact that your theory of mind is that I should find this "awe" is evidence that you have no idea what I think in a TCS context. I am certainly not interested in the usual vulgar caricature of CS/TCS.
If people actually took a class in it and passed the class they would get this intuition right away. Explaining it to someone who never actually did the classwork, the problem sets, is like talking about math without calculus or biology without genetics. Won't get very far and puts huge burden on scientific literacy. It is not MY job to waste time explaining undergrad level material as a basis for a sane discussion. There's no way around not doing the homework. If after people do the classwork and still disagree, then I'll listen.
Case in point, the whole astrocytes/cells/blah blah blah is a well-known fallacious objection. If you understood the Church Turing thesis you would not be using this argument. The only thing I will currently allow around it if you believe intelligencee/consciousness requires quantum computation, a la Penrose.
The whole point is that abstract neural network are the fundamental model, the DUAL to the tape Turing machine. Computationally, everything that exists else reduces to either of these. There's nothing else.
In contrast the way you are trying to argue about oscillators, clocks, astrocyte, cells, is like worry about epicycles in the face of a Newtonian paradigm. Will results about astrocytes help? Who knows. They might not be necessary at all. Or they might provide efficiencies. But they are not fundamental. Same applies to analog computing ("what about the electrical voltage levels in synapses??, we'll never model those!!" Etc.)
Basically a lay confusion of biological mechanism with computational laws of nature, totally different things.
Since people are resorting to ad hominems anyways I will point out that Hinton and Aaronson experts who I largely agree with. They understood the paradigm shift which requires a level of actual mathematical background to understand what computational abstractions are. The only way mere mortals can grasp this is by doing the coursework. And who has done it?
I can make a similar argument and say if only you'd spent a few hours looking at a real neuron under a microscope, you'd understand that computer science is hopelessly useless to explain what you see.
I'm happy you took your class and got something from it. But since what you seem to have gotten is religion, it's kinda hard to care.
> Like everyone, I’m tremendously impressed by the lifelikeness of these systems compared to the history of earlier AI. But the disposition of some philosophers to take seriously the question whether these things are conscious or might soon become conscious is just ridiculous. They are assuming that the physical substrate is irrelevant. They all think that the material conditions for consciousness can be given in functionalist terms.
> (...)
> Functionalism is substrate-independent. It’s more than likely that the truth is one which links physical processes and systems of certain kinds to mental phenomena. Complicated data centers have absolutely nothing in common with the physical character of a nervous system. We have strong reasons to believe that while there may be consciousness elsewhere in the universe, it would require the appearance of some kind of organism with something analogous to a nervous system.
Why should we assume our substrate is the "right" substrate for truly conscious beings?
Perhaps we have fake consciousness too (just like programs running on silicon), and some advanced alien being could look at our biological neurons stuff and immediately recognize our fakeness. Perhaps those aliens could proceed to assert that their own substrate is the only one that could possibly be conscious - all the other substrates (like ours) being fake.
We couldn't possibly know, because we don't know what consciousness is.
> Why should we assume our substrate is the "right" substrate for truly conscious beings?
No one assumes that it is the only right substrate. But if you don't believe we are a right substrate, are you waiving your own consciousness? Seems like you are, which does make me question how coherently you can argue about this.
> We couldn't possibly know, because we don't know what consciousness is.
We don't know it's deepest mechanisms. That also holds true for the universe. Doesn't mean we can't see the difference in dynamics.
Simply put, consciousness cannot be turned off and on. No definition of consciousness holds up to a system that can turn off, then be turned on 100 years later to carry on generating text as it was.
Evolutionarily, such a thing would be quite idiotic and maladaptive. It's not feasible to be an "agent" in the world when you are, by default, off, and require a prompt somewhere upstream to come awake.
Biology, on the other hand, is exquisitely timed. Each cell (including each neuron) is a self-sustained oscillator, with oscillatory rhythms that impact behavior, learning and memory. These clocks are endogenous, meaning they continue as long as long as the organism is alive, even without external periodic signals. LLMs have no such endogenous self-sustained osciallation operating continuously.
Go back to before the LLM's are conscious hype started. Find me one definition or example of consciousness that can be switched off. And no, neither sleep nor anesthesia count. They're different states of consciousness, but the brain and body are not "off".
Nor will any definition of consciousness pre-LLMs even countenance the idea that a frozen set of weights is conscious. The very idea of consciousness without continual learning is utter stupidity. All the sadder that it's spread so wide and has already cost us trillions.
Unless you believe the molecular state of the body cannot be in theory ever recorded and resynthesized, how is it not possible in this universe to turn humans or other animals on and off?
Huh?
"the molecular state of the body cannot be in theory ever recorded and resynthesized"
This is 100% true. This is physically impossible to do.
What’s a hypothetical mechanism to get a molecular accurate clone that also preserves thermal and temporal states?
Heck, what’s hypothetical mechanism to read The exact state of a body down to every molecule?
Obviously its impossible right now. But let me get this clear are you saying its impossible as in say putting a lander on Pluto (impossible today, no theoretical roadblocks to be able to do it in future) or physically impossible. After all (morality aside) one can simply slice off molecules from the body using eg a similar method as atomic microscopy. You could argue that the information might be hard to read, or there might be a proof that its impossible to read without altering it appreciably (quite possible), but we would need such a propf first, thats the only possible argument you can give. Obviously its aburd to say the molecules int the body are different from the same molecules synthesized elsewhere.
The amount of precision depends on how sensitive the body and brain is, eg we don't need to go to qm to design or copy a car or motor.
I'm saying it is physically impossible. And it's not some vague theory. It's pretty darn well established.
> one can simply slice off molecules from the body using eg a similar method as atomic microscopy.
This sentence doesn't make sense. Atomic force microscopy (which I assume is what you're referring to) doesn't itself slice any molecules from the body. It can be used to study molecules or cells ripped from the body, sure. In the process, the molecular state of the body you wanted to copy changes, so even if you reconstitute the bit you got out and put under the microscope (itself a big if, since no known microscope can give you exact quantity and type of every molecule in your sample), that bit is already in a different state, so you're not reconstiting the state of that segment of the body, let alone the entire body.
A human body has about 36 trillion human cells. About an equal number of bacterial, archeal and unicellular eukaryotic cells (bacteria dominate this cluster). Together, they have about an octillion molecules. 99% of that is water, but the dynamics of that water make the Naveir stokes equation feel like the two-times multiplication table in comparison.
The rest is considerably harder. Proteins, for example, aren't floating around bearing the frozen crystal structures AlphaFold can so quickly solve. Instead, dissolved in all that water, protein molecules cycle through all kinds of unstable confirmational states, sometimes for very brief instances, which are nonetheless functionally critical, and part of the state description of any cell. But wait, in the seconds it took you to read the previous two sentences, a significant chunk of those proteins have gone through multiple confirmational changes. And somehow you're going to find and read the state of every protein in a picosecond? There's about a 100 sextillion of those in your body.
So which moment are you planning to copy? And how? Any read attempt is going to be destructive, especially to these fragile confirmational states. Freeze the cell and those states shift. Break up the cell to measure it, and those states shift. Use light to image it, and you can image a subset of proteins but either need to modify them, or kill and fix the cell first, and in either case, sufficient light to illuminate every part of the body in an instant to be read by whatever detector you design still won't get you what you want, because guess what, light changes the proteins too, and enough will destroy them.
Then consider lipids, metabolites, small peptides, and small RNAs (which also do all kinds of confirmational hijinks second to second)... None of these can be fully measured body wide for you to clone the body's state.
It's just not even within the realm of plausibility to do what you're suggesting. Even a simulation that's accurate down to atomic resolution, if we someday invent computers that can do it, will drift from the actual system really really rapidly.
The body isn't a car or a motor. It doesn't have their simple regular structure. And literally tiny molecular changes have huge impact on overall organism state. Thinking about the body like it's some engineered machine is probably why you fail to see that even though it is made of entirely boring matter that's pretty common in the universe, what separates it from non-life is the depth and complexity of it's dynamics. And no, no LLM comes remotely close to the complexity of a bacterium let alone a human.
Yes I meant a probr that reads a molecule then breaks off its bonds to check the further inside the object. As I said before I am quite open to any proof along the lines of its impossible to measure a particular state without appreciably changing it due to sensitvity arguments. It might be true but unless we have a proof I can't say either way.
Completely engineered systems can be unpredictable without fully solving it eg chaos, I don't have any complaints about that as a line of defense.
Define "proof" in this context please. Do you mean a math proof?
Yes some kind of mathematical proof that any measurement of this system by the time it had taken place has changed the system appreciably in a way that can't be inferred back or accounted for.
That's a physical proof, not mathematical. You're not going to derive this fact from abstraction, but from actual empirical work.
But if you want to, go right ahead and ask an AI model if what you said is possible. Maybe it'll find the right math to convince you that physical reality is not susceptible to instantaneous cloning of states.
I don't think current ai's are able to do this kind of math unguided. If you are so sure of it you should have a proof ready. I am not against this fact being possibly disproven but it has to be an actual proof using arguments of information theory/dynamical systems etc
> We have strong reasons to believe that while there may be consciousness elsewhere in the universe, it would require the appearance of some kind of organism with something analogous to a nervous system.
At least he gets to the point quickly. But I'd like to know what those "strong reasons" are.
Gottem!!
> Perhaps we have fake consciousness too (just like programs running on silicon), and some advanced alien being could look at our biological neurons stuff and immediately recognize our fakeness.
"That’s ridiculous. How can meat make a machine? You’re asking me to believe in sentient meat."
> We couldn't possibly know, because we don't know what consciousness is.
Great minds discuss ideas, average minds discuss events, small minds discuss people, philosphers discuss "consciousness".
Please use the original title (A Famed Philosopher Takes on AI Consciousness)
> https://news.ycombinator.com/newsguidelines.html
Ok I'm on the total opposite of this guy, I am almost tempted to write a short letter telling him why Thomas Nagel, famous American philosopher, is wrong on this.
Basically his opinion is derivative of stochastic parrot theory. He admits as much in the interview. (Since stochastic parrot theory is wrong, I could stop right here, and so should he if he had a truly sufficient argument in the first place...)
He is committing the mistake as Scott Aaronson says, justa-ism. It's "just a language model" therefore it cannot possibly really be intelligent/alive/conscious/goalpostmoving. Followed by some tricky rhetorical move appealing to human exceptionalism.
The fact is that philosophers would do well to take computational complexity theory, the one involving actual math homework. All Nagel's woo-woo talk about a substrate and a need for an "analogous nervous system" shows yet again how CS as a nascent field, and theoretical CS in particular, has not fully been internalized by other disciplines.
Nagel says a bat is conscious. But an LLM with trillion dimensional neuron parameters, obtained through stochastic gradient descent, cannot possibly conscious because it is on the wrong substrate? Really?
In fact it is the other way around. The true "substrate" is the neural network model, of which humans (and bats) are merely one example of. That is the correct way to understand what LLM computation is. The abstract LLM neural net is what Alan Turing himself envisioned as the alternative to classical computation.
Yeah, I wouldn't necessarily call LLMs conscious, but I'm reminded of an argument that I heard once that computers could never hold consciousness because they don't process things in parallel. There's a lot of incurious reasoning abound when it comes to commentary on this topic.
Its absurd in this day an age that "philosophers" talking about consciousness who don't address at least thermodynamics, computation theory and information theory should not just be immediately laughed out of the room.
If they cannot at least acknowledge that the universe is a Turing computer, all Turing computers have equal power, that information has an energy cost so some special immaterial dual soul will have to violate thermodynamics to generate thoughts and actions, their arguments should be dismissed.
I wouldn't assume the universe is a Turing machine. The holographic theory does suggest that it has limited state in any given space, but that doesn't limit or to finite state transitions, or to finite space.
We can't see enough of space to know whether it's infinite.
I just want someone to define "consciousness" before they start arguing who can and cannot have it.
If the definition is "the processes of a biological brain" then clearly a computer cannot have it. It's also not very useful.
Turing machines have no trouble being infinite. That the universe is a tm comes from the fact that as of yet there is no knownaw that enables hypercomputation.
>theoretical CS in particular, has not fully been internalized by other disciplines.
Whereas it appears theoretical computer Science sometimes forgets there are many routes to computing per its own fundamentals, and certainly ignores the fact that Neuroscience didn't stop the day McCulloch and Pitts first conceived of the artificial neuron.
> But an LLM with trillion dimensional neuron parameters, obtained through stochastic gradient descent, cannot possibly conscious because it is on the wrong substrate? Really?
Yes really. You seem to think the word "trillion" should awe us. It's hard to understand why. Care to tell us?
Critically, whether a trillion or a significantly larger number, LLMs parameters have characteristics that make talk of consciousness absurd. To whit: they are frozen, and between two forward passes, a million years can pass and the model wouldn't know.
> In fact it is the other way around. The true "substrate" is the neural network model, of which humans (and bats) are merely one example of. That is the correct way to understand what LLM computation is. The abstract LLM neural net is what Alan Turing himself envisioned as the alternative to classical computation.
Here's just one problem with this theory. All neurons are 24-h oscillators, and have shorter oscialltions nested under this. A fact neither Turin nor McCullough nor Pitts could know when they were trying to imagine how the brain works. They assumed a global clock, which is absolutely false in biology.
And this rhythmicity is evolutionarily older than the rise of neurons. Cyanobacteria have 24 hour rhythms (though using different components), and can anticipate seasonal changes by this system, and first evolved this 2.5 billion years ago.
The issue isn't just substrate. Some fundamental comoutstuonal features of cells, neurons, brains and bodies need to be thoroughly ignored to land on a claim that LLMs are the true substrate of which human brains are an expression.
And also... Have you heard of astrocytes? Roughly as many of those as neurons in the brain, and they're part of every synapse. If artificial neural nets are the formula for consciousness, bats and humans sure seem to spend a lot on these "useless" cells, no? Interestingly, astrocyte size and complexity goes up as you go from mouse to human, and implanting human astrocytes in mice enhances their intelligence. One heck of an odd result to explain if it's all LLM like dynamics in the brain.
First, it is bullshit sophistry to accuse TCS to "appear to forget fundamentals" when barely anyone studies TCS in its form TODAY. It is also an tu quoque ad hominem and does not deserve further response. The fact that Nagel in THIS interview has zero understanding of formal computability and complexity theory, TWO SEMESTERS of niche undergrad specialization, is like talking about biology while NEVER taking genetics. It is unacceptable in the 21st century. They say, "stay in your lane", well, with TCS concepts/intuitions, it's more like the whole highway needs an upgrade.
I shouldn't CARE to tell you the significance of a trillion dimensions. The whole point is, if you TOOK modern complexity/computability you would understand the it is the OPPOSITE of "awe". The whole moral of the story is that THERE IS NO COMPUTING MAGIC. Turing equivalence and the Church Turing thesis* being the only sane interpretation of reality. If you believe brute scaling is insufficient then you violate the Church Turing thesis. It is a negative thesis, a constraint on our world. (n.b. I am handwavily referring to the class of extended Church-Turing theses per Aaronson, please do not confuse this.)
Again, the fact that your theory of mind is that I should find this "awe" is evidence that you have no idea what I think in a TCS context. I am certainly not interested in the usual vulgar caricature of CS/TCS.
If people actually took a class in it and passed the class they would get this intuition right away. Explaining it to someone who never actually did the classwork, the problem sets, is like talking about math without calculus or biology without genetics. Won't get very far and puts huge burden on scientific literacy. It is not MY job to waste time explaining undergrad level material as a basis for a sane discussion. There's no way around not doing the homework. If after people do the classwork and still disagree, then I'll listen.
Case in point, the whole astrocytes/cells/blah blah blah is a well-known fallacious objection. If you understood the Church Turing thesis you would not be using this argument. The only thing I will currently allow around it if you believe intelligencee/consciousness requires quantum computation, a la Penrose.
The whole point is that abstract neural network are the fundamental model, the DUAL to the tape Turing machine. Computationally, everything that exists else reduces to either of these. There's nothing else.
In contrast the way you are trying to argue about oscillators, clocks, astrocyte, cells, is like worry about epicycles in the face of a Newtonian paradigm. Will results about astrocytes help? Who knows. They might not be necessary at all. Or they might provide efficiencies. But they are not fundamental. Same applies to analog computing ("what about the electrical voltage levels in synapses??, we'll never model those!!" Etc.)
Basically a lay confusion of biological mechanism with computational laws of nature, totally different things.
Since people are resorting to ad hominems anyways I will point out that Hinton and Aaronson experts who I largely agree with. They understood the paradigm shift which requires a level of actual mathematical background to understand what computational abstractions are. The only way mere mortals can grasp this is by doing the coursework. And who has done it?
/endrantdump
If you can't explain it, you don't understand it.
I can make a similar argument and say if only you'd spent a few hours looking at a real neuron under a microscope, you'd understand that computer science is hopelessly useless to explain what you see.
I'm happy you took your class and got something from it. But since what you seem to have gotten is religion, it's kinda hard to care.
lmao nothing makes the ai bros on this site more mad than suggesting claude isn’t a sentient being