Total speculation, but I think it may be possible for transformer models to internalize a deep understanding of animal language, and possibly generate animal language. For some reason I also believe that they may never be able to translate it for us, and animals and humans will be unable to communicate just as much as today.
I agree, transformer models trained solely on animal language may be able to predict the next "token" and generate coherent animal "sentences", even if we don't understand what any of those sounds/vocalizations mean.
However, I do think there is a path to some sort of translation to human languages. An English dictionary may seem pointless because it defines every word using only other English words. But the meaning is contained in the _relationship_ between the words.
The _relationship_ between certain animal-language tokens may look close enough to the _relationship_ between certain human-language tokens to bridge the gap, even if both languages are entirely disconnected and we've never seen a direct translation between them.
Just correlate the tokens with the state of the environment and surrounding behavior and the meaning should reveal itself. The extent that vocalizations predict subsequent environment states, or the environment predicts subsequent words, is the meaning captured by those vocalizations.
Because animals doesn't use words, they use the equivalent of laughter, crying, body language, shrieks etc. So you already understand your dog and cat, as it isn't that hard to read these signals in them.
Why is much detail required for the pace of change comparison?
Animal language doesn’t change much. Dogs sound pretty much the same as they did 10 years ago.
But AI generated sounds have changed rapidly in the past couple of years. In terms of music generation, sound to sound translation, and audio to text transcription.
It’s an interesting thought - even with a translator, could their cognition and world be so alien to us as to be incompressible?
I would think largely not - particularly for mammals - it’s easy to think you have nothing in common with a shrew or a whale, but we are quite recent cousins, and much of our cognitive architecture and sensorium is the same.
That said, there could well be challenges and revelations beyond the obvious - for instance, can domesticated members of a species communicate with their wild cousins, or have they lost their culture/language? Cetaceans - do they talk in holograms? These are perhaps still obvious questions, but if it’s something we figure out, we may get answers to things we’ve never even considered asking.
I can totally understand an animal associating some of the sounds that it has heard in its lifetime, perhaps even a "name" for itself. There may also be possibilities for some animal populations e.g. crows if enough generations have passed with common sounds, to mean different things, especially when those things were encountered enough times during the lifetimes of them. So that could build to some kind of "for the birds of this region, this sound/combination may represent something specific".
Same for dogs and so on.
There may also be some kind of more subtle genetics/morphology based distinctions, beyond the obvious "snarl for most dogs means afraid, angry, aggressive, possessive, or in pain"...
But... it seems that the collapse of tower of babel happens very frequently for non-humans, so unsure how "useful" an AI training may be for the majority of the cases.
in my opinion, derived from just sitting in the woods being observant of the bird song, I don't think any individual bird has much to say (besides the requisite 'somebody screw me' and 'get the fuck away') but there is a lot of nuance'd information being transmitted in aggregate. How long its been since there was a predator, how soon is the next rain, are we in agreement that just about all the food is gone here (quorum sensing / collective decision making).
Crows and other corvids of course are another level of sophistication. When they're gathering in trees (like when instead of leaves a tree has crows, that kind of gathering) somebody told me that there's elections going on, which individuals will be given decision making power are being chosen through bickering and persuasion.
As for dogs (and a brief google doesn't recall my source so take it with a grain of salt, but,) I'll add that they're the only animal besides humans that have been observed making pacts, promises, "if you go I'll go" style. IANA-consensus-in-animal-populations however, and body language goes a long way in signaling submission, agreeableness, resistance, doubt etc.
It's a fair bet that anything intellectual that a dog can do, many other animals can do, too. If it appears that it's only dogs, that's almost certainly just because we get to observe them so much more than other creatures.
we have, besides a dog, cats, and they all know their individual names very clearly, repeatably, demonstrably. they also seem to know the names of one another, but of course cooperation is more eager in dogs than in cats, so harder to demonstrate that.
I imagine this line of research would be of interest to intelligence agencies. Especially when you start taking a multi-modal approach that incorporates sensors such as heart rate, perspiration, etc, then we're basically talking about pre-Judgment Day Terminators.
We are translating language from a ton of other species. Now we can see that they are using names and probably sharing information.
The brightest minds now want to use machine learning to tease out more low level features of language itself, across various species.
We are already calling monkeys and elephants by their native names y'all... And they are responding to this...
Think about it... This research is unlikely to stall. We're barely scratching the surface. The animals were having more nuanced conversation than we thought. They've been doing this for millions of years...
Condescension toward nature... Usually a mistake. I'm not saying they are all philosophers, but it seems very likely they've been having some pretty advanced conversations surrounding their own affairs. If we can call them by their name... Probably we're going to unlock some pretty interesting communication.
There recently was a 2025 predictions thread, my prediction just got adjusted to "the Department of Defense will bribe birds to spy for them by talking to them through AI".
> The ducks in St. James' Park are so used to being fed bread by secret agents meeting clandestinely that they have developed their own Pavlovian reaction. Put a St. James' Park duck in a laboratory cage and show it a picture of two men—one usually wearing a coat with a fur collar, the other something somber with a scarf—and it'll look up expectantly. The Russian cultural Attachés black bread is particularly sought after by the more discerning duck, while the head of M19's soggy Hovis with Marmite is relished by the connoisseurs.
I think each animal has its own language, for example my dog make a noise with his noise when he is frustrated (expected something better that what he got), and move the tail sideways when he feels like to play or put his tail up when he is happy, so it is a body language. He can't learn other dogs language because there is no consensus on what words mean and so there is not feed-back.
Total speculation, but I think it may be possible for transformer models to internalize a deep understanding of animal language, and possibly generate animal language. For some reason I also believe that they may never be able to translate it for us, and animals and humans will be unable to communicate just as much as today.
I agree, transformer models trained solely on animal language may be able to predict the next "token" and generate coherent animal "sentences", even if we don't understand what any of those sounds/vocalizations mean.
However, I do think there is a path to some sort of translation to human languages. An English dictionary may seem pointless because it defines every word using only other English words. But the meaning is contained in the _relationship_ between the words.
The _relationship_ between certain animal-language tokens may look close enough to the _relationship_ between certain human-language tokens to bridge the gap, even if both languages are entirely disconnected and we've never seen a direct translation between them.
Just correlate the tokens with the state of the environment and surrounding behavior and the meaning should reveal itself. The extent that vocalizations predict subsequent environment states, or the environment predicts subsequent words, is the meaning captured by those vocalizations.
to what extent do ethologists already (manually) look for these correlations?
See also: word embedding and sentence embedding models
What makes you think this? On the face of it, what you're saying seems to be absolutely nonsensical, so I'm curious.
Because animals doesn't use words, they use the equivalent of laughter, crying, body language, shrieks etc. So you already understand your dog and cat, as it isn't that hard to read these signals in them.
Animals will probably evolve to speak well before AI figures out what animals are saying.
Given the evolution of AI vs the evolution of animal language, the probability is actually heavily leaning towards the opposite.
I would really like a _very_ detailed explanation of how AI is evolving instead of just having more computation power and shortcuts applied.
Why is much detail required for the pace of change comparison?
Animal language doesn’t change much. Dogs sound pretty much the same as they did 10 years ago.
But AI generated sounds have changed rapidly in the past couple of years. In terms of music generation, sound to sound translation, and audio to text transcription.
Why would animals ever have an evolutionary need to learn to speak english?
So we can sell them things, obviously.
It’s an interesting thought - even with a translator, could their cognition and world be so alien to us as to be incompressible?
I would think largely not - particularly for mammals - it’s easy to think you have nothing in common with a shrew or a whale, but we are quite recent cousins, and much of our cognitive architecture and sensorium is the same.
That said, there could well be challenges and revelations beyond the obvious - for instance, can domesticated members of a species communicate with their wild cousins, or have they lost their culture/language? Cetaceans - do they talk in holograms? These are perhaps still obvious questions, but if it’s something we figure out, we may get answers to things we’ve never even considered asking.
I can totally understand an animal associating some of the sounds that it has heard in its lifetime, perhaps even a "name" for itself. There may also be possibilities for some animal populations e.g. crows if enough generations have passed with common sounds, to mean different things, especially when those things were encountered enough times during the lifetimes of them. So that could build to some kind of "for the birds of this region, this sound/combination may represent something specific".
Same for dogs and so on.
There may also be some kind of more subtle genetics/morphology based distinctions, beyond the obvious "snarl for most dogs means afraid, angry, aggressive, possessive, or in pain"...
But... it seems that the collapse of tower of babel happens very frequently for non-humans, so unsure how "useful" an AI training may be for the majority of the cases.
in my opinion, derived from just sitting in the woods being observant of the bird song, I don't think any individual bird has much to say (besides the requisite 'somebody screw me' and 'get the fuck away') but there is a lot of nuance'd information being transmitted in aggregate. How long its been since there was a predator, how soon is the next rain, are we in agreement that just about all the food is gone here (quorum sensing / collective decision making).
Crows and other corvids of course are another level of sophistication. When they're gathering in trees (like when instead of leaves a tree has crows, that kind of gathering) somebody told me that there's elections going on, which individuals will be given decision making power are being chosen through bickering and persuasion.
As for dogs (and a brief google doesn't recall my source so take it with a grain of salt, but,) I'll add that they're the only animal besides humans that have been observed making pacts, promises, "if you go I'll go" style. IANA-consensus-in-animal-populations however, and body language goes a long way in signaling submission, agreeableness, resistance, doubt etc.
It's a fair bet that anything intellectual that a dog can do, many other animals can do, too. If it appears that it's only dogs, that's almost certainly just because we get to observe them so much more than other creatures.
https://english.elpais.com/science-tech/2023-09-13/pacts-amo...
They are more sophisticated in their communication than you are portraying.
> "Perhaps" even a name for itself
No perhaps needed, dogs will respond to a name and know it refers to them. You might be interested in reading this study about dogs doing fMRI while responding to new and old words: https://www.frontiersin.org/journals/neuroscience/articles/1...
we have, besides a dog, cats, and they all know their individual names very clearly, repeatably, demonstrably. they also seem to know the names of one another, but of course cooperation is more eager in dogs than in cats, so harder to demonstrate that.
So your comment is just “I don’t know how useful this new science will be”?
What separates any science from corporate R&D is basically that exact statement.
I wouldn’t be surprised if we learn more about ourselves from this research, not to mention the animals it covers.
Don't get me wrong, I'm excited for it, just trying to understand or predict the extents... Also I'm a layman in this field, so who knows!
I imagine this line of research would be of interest to intelligence agencies. Especially when you start taking a multi-modal approach that incorporates sensors such as heart rate, perspiration, etc, then we're basically talking about pre-Judgment Day Terminators.
My dog knows all the neighbor dogs names, and their owner's names. He's a smart breed but I feel like you're really underestimating animal cognition.
We are translating language from a ton of other species. Now we can see that they are using names and probably sharing information.
The brightest minds now want to use machine learning to tease out more low level features of language itself, across various species.
We are already calling monkeys and elephants by their native names y'all... And they are responding to this...
Think about it... This research is unlikely to stall. We're barely scratching the surface. The animals were having more nuanced conversation than we thought. They've been doing this for millions of years...
Condescension toward nature... Usually a mistake. I'm not saying they are all philosophers, but it seems very likely they've been having some pretty advanced conversations surrounding their own affairs. If we can call them by their name... Probably we're going to unlock some pretty interesting communication.
Q: What do you get when you cross artificial "intelligence" with "dumb" animals?
A: I don't know. Further research required.
There recently was a 2025 predictions thread, my prediction just got adjusted to "the Department of Defense will bribe birds to spy for them by talking to them through AI".
> The ducks in St. James' Park are so used to being fed bread by secret agents meeting clandestinely that they have developed their own Pavlovian reaction. Put a St. James' Park duck in a laboratory cage and show it a picture of two men—one usually wearing a coat with a fur collar, the other something somber with a scarf—and it'll look up expectantly. The Russian cultural Attachés black bread is particularly sought after by the more discerning duck, while the head of M19's soggy Hovis with Marmite is relished by the connoisseurs.
Good Omens, by Terry Pratchett and Neil Gaiman
I think each animal has its own language, for example my dog make a noise with his noise when he is frustrated (expected something better that what he got), and move the tail sideways when he feels like to play or put his tail up when he is happy, so it is a body language. He can't learn other dogs language because there is no consensus on what words mean and so there is not feed-back.
Exactly it is all expressive.
"Donning his new canine decoder, Professor Schwartzman becomes the first human being on Earth to hear what barking dogs are actually saying"
"Hey!"
These were the memes that we brought to and from school uphill both ways, kids.
We are about to watch alot of mysteries unravel