I've always found this interesting. Think+transmit seems more likely to be the bottleneck vs receive, given that we can easily parse most podcasts etc at 3-4X speed. If being understood by everyone wasn't required, I wonder if one could learn to boost both send and receive rates?
But what you said made me curious. I listened to this podcast at 3x. I was able to understand all the words, but my conceptual understanding decreased. I also have to listen actively -- not passively.
At 4x, I could only understand the shape of the sentences but could no longer make out the words. But I turned on the captions and found I could keep up. Turns out reading at 4x works, listening at 4x doesn't.
English is also spoken with different prosodies and cadences. For instance, I can understand Singaporean English perfectly, but it's less amenable to being sped up. I tried listening to this lecture in Singaporean English in 3x and found that I could barely understand it.
This suggest that maybe human minds are able to form ideas at a certain speed, and language has evolved to convey these ideas and not be a bottleneck, but there is no reason to improve language beyond this.
Hm, I’ve always struggled with the other bottleneck - speaking. I tend to slur my words because my brain is forming the thoughts faster than I can speak. I could perhaps speed up my speech to match my brain but I’d sound like a maniac.
Don’t get me started on writing. My letters will often transition halfway into a letter thats 3 words ahead.
3-4x! Are you the flash? I usually run things about 1.5x when I am commuting. 3x would require laser focus without distractions. Or you mean more a “can technically absorb the language being spoken” sense?
Not OP but listen to podcasts at highly accelerated settings:
The information density of ‘two dudes talking’ or any unscripted format is very low, so it time-compresses well. Specific podcasts, typically scripted monologues with technical content, such as Causality [0] (recommended!), I need to listen to much slower. Ditto if it is in an accent which isn’t mine, which slows my comprehension. I also slow the speed if I’m driving. So, yes, it takes mental overhead, but is doable. Go one click at a time and it will feel natural.
I suppose the format is a huge differentiator. I exclusively listen to highly produced content which has essentially no dead time. The content is already a compressed transmission of information.
I wonder, does this account for compression? Are the bits counting a concrete representation of ideas transmitted? Or are bits counting a simple abstraction over syllables, similar to a token? Language is compression. And while humans may transmit at a universal rate, maybe some languages/cultures have a more dense compression of ideas than others? Like in Turkish, how there's a word for "moonlight on the water" https://ihearthesamewinds.wordpress.com/2011/11/01/142/.
>Each participant read aloud 15 identical passages that had been translated into their mother tongue. After noting how long the speakers took to get through their readings, the researchers calculated an average speech rate per language, measured in syllables/second.
Maybe more interesting: what’s the average reading speed per language, and what’s the variation? I know that reading speed varies a lot ( also depends on topic, a math textbook reads slower than an Ian Fleming book).
Are simultaneous translator’s brains different? They need to process two languages at once, and I never could do that even though I’m fluent in more than one language.
The paper does not seem to support that human speech has a universal transmission rate or that every spoken language has a universal information rate. They showed that information rate varied by individual and by language, just that it varied less than the syllable rate.
This was also bounded by a reading task, so the performance shown per language could be influenced by the average reading skills of people who speak those languages. They also asked them to pronounce differently than they normally would.
If you take the 17 languages they tested and get the average between them, you get 39bits/s. For English and French, the information rate they recorded was higher, with an average closer to ~45bits/s (just eyeballing their chart). Their results also showed Thai at ~35bits/s. A 10bits/s swing from median to median is pretty huge.
From the paper:
"We collected recordings of 170 native adult speakers of the aforementioned 17 languages, each reading at their normal rate a standardized set of 15 semantically similar texts across the languages (for a total amount of approximately 240,000 syllables). Speakers became familiar with the texts, by reading them several times before being recorded, so that they understand the described situation and minimize reading errors"
"Together, our findings show that while there is wide interspeaker variation in speech and IRs (information rates), this variation is also structured by language. This means that an individual’s speech behavior is not entirely due to individual characteristics but is further constrained by the language being spoken."
"However, languages seem to stably inhabit an optimal range of IRs, away from the extremes that can still be available to individual speakers. Languages achieve this balance through a trade-off between ID (information density) and SR (syllable rate), resulting in a narrower distribution of IRs compared to SRs. In the introduction, we rhetorically asked whether too low or too high an IR would impede communicative and/or cognitive efficiency. Our results here suggest that the answer to both questions is positive and that human communication seems to avoid two extreme sociolinguistic profiles: on the one hand, high ID languages spoken fast by their speakers (“high-fast”), and, on the other, low ID languages spoken slowly by their speakers (“low-slow”)."
This definitely fits with my experience as a language learner studying French and Japanese.
Japanese is definitely a faster spoken language than French, but French words tend to be a lot more verbose and packed with meaning. For Japanese speakers to communicate as much meaning as French speakers, they would need to speak faster.
its not the language that communicated its the people. You recruit mediocres in different languages they all communicated in the same mid , 39 bits or whatever. Let two geniuses have a chat or even two teenagers - its a different ball game than 39 bits
Maybe not just spoken language. I would suggest sign language has a similar bit rate as spoken. The evidence is that a sign interpreter conveys about the same information in a conversation in the same time.
There is also a lot of nonverbal data. Imagine the horror of discovering your conversational ML build could hold a plausible verbal conversation only guessing 58% of spoken words accurately... then realizing humans likely fair much worse. =3
The bits are defined in another paper by the authors using "syntagmatic density of information ratio (SDIR)" which seems to be related to the number of syllables required to convey the same information in different languages, using Vietnamese as the baseline.
I would expect you can only transmit so fast in morse code given the need for the dots and dashes to be clearly distinguished from each other and identifiable by the recipient.
Of course, if you know both ends are computers you can just transmit in some other encoding at a much higher rate.
As Morse speeds up, you stop relying on individual dots and dashes and begin recognizing common combinations of letters. Faster still and you are mainly hearing word stems and suffixes.
The faster the information comes at you, the less important any particular bit is, because you have more context with which to autocorrect.
The point is, Morse code is as many "bits per second" as you want. You could send Morse code at 10 gigabits per second if you wanted. It is not meaningful to say that Morse code implies a particular data rate.
Historically the metric for Morse code is words per minute. Morse is similar to a Huffman code where common letters are allocated fewer elements, so it's not very meaningful to talk about "bits per second" with respect to Morse even if you do specify the number of words per minute. The number of "bits" will vary based on the letters being transmitted.
I mean if you want that level of pedantry "I" can't send Morse code at 10 gigabits per second. I can make a computer transmit it at that speed, but I am not personally sending it that rate. And, because one generally needs a machine to transmit Morse code, one cold argue that "I" never send Morse code ever.
Oh, come on, this is just being coy for no reason. Given the context, it is abundantly clear they mean "normal, human operated morse code".
A skilled operator is around 30 WPM. The average English word is 5 (rounded up) characters. Add one character for the space. That's 180 characters per minute, or 3 character per second. With 37 characters available in morse code, that's log_2(37) or 5.2 bits per charater.
So 15.6 bits per second. Just under half of the 39 bits they got for speech, like they said.
I've always found this interesting. Think+transmit seems more likely to be the bottleneck vs receive, given that we can easily parse most podcasts etc at 3-4X speed. If being understood by everyone wasn't required, I wonder if one could learn to boost both send and receive rates?
I usually listen at 1.5x passively.
But what you said made me curious. I listened to this podcast at 3x. I was able to understand all the words, but my conceptual understanding decreased. I also have to listen actively -- not passively.
https://www.youtube.com/watch?v=9SFkwdm0PP0
At 4x, I could only understand the shape of the sentences but could no longer make out the words. But I turned on the captions and found I could keep up. Turns out reading at 4x works, listening at 4x doesn't.
English is also spoken with different prosodies and cadences. For instance, I can understand Singaporean English perfectly, but it's less amenable to being sped up. I tried listening to this lecture in Singaporean English in 3x and found that I could barely understand it.
https://www.youtube.com/watch?v=LA1dnUYzRWU
Blind people often go upto 600 words per minute and more with text-to-speech, which is, I think, would be an equivalent to 5x and more.
In fairness, they cannibalize their visual cortex to do that:)
This suggest that maybe human minds are able to form ideas at a certain speed, and language has evolved to convey these ideas and not be a bottleneck, but there is no reason to improve language beyond this.
Hm, I’ve always struggled with the other bottleneck - speaking. I tend to slur my words because my brain is forming the thoughts faster than I can speak. I could perhaps speed up my speech to match my brain but I’d sound like a maniac.
Don’t get me started on writing. My letters will often transition halfway into a letter thats 3 words ahead.
3-4x! Are you the flash? I usually run things about 1.5x when I am commuting. 3x would require laser focus without distractions. Or you mean more a “can technically absorb the language being spoken” sense?
Not OP but listen to podcasts at highly accelerated settings:
The information density of ‘two dudes talking’ or any unscripted format is very low, so it time-compresses well. Specific podcasts, typically scripted monologues with technical content, such as Causality [0] (recommended!), I need to listen to much slower. Ditto if it is in an accent which isn’t mine, which slows my comprehension. I also slow the speed if I’m driving. So, yes, it takes mental overhead, but is doable. Go one click at a time and it will feel natural.
[0]: https://engineered.network/causality/
I suppose the format is a huge differentiator. I exclusively listen to highly produced content which has essentially no dead time. The content is already a compressed transmission of information.
I wonder, does this account for compression? Are the bits counting a concrete representation of ideas transmitted? Or are bits counting a simple abstraction over syllables, similar to a token? Language is compression. And while humans may transmit at a universal rate, maybe some languages/cultures have a more dense compression of ideas than others? Like in Turkish, how there's a word for "moonlight on the water" https://ihearthesamewinds.wordpress.com/2011/11/01/142/.
Yes:
>Each participant read aloud 15 identical passages that had been translated into their mother tongue. After noting how long the speakers took to get through their readings, the researchers calculated an average speech rate per language, measured in syllables/second.
Maybe more interesting: what’s the average reading speed per language, and what’s the variation? I know that reading speed varies a lot ( also depends on topic, a math textbook reads slower than an Ian Fleming book).
Are simultaneous translator’s brains different? They need to process two languages at once, and I never could do that even though I’m fluent in more than one language.
The paper does not seem to support that human speech has a universal transmission rate or that every spoken language has a universal information rate. They showed that information rate varied by individual and by language, just that it varied less than the syllable rate.
This was also bounded by a reading task, so the performance shown per language could be influenced by the average reading skills of people who speak those languages. They also asked them to pronounce differently than they normally would.
If you take the 17 languages they tested and get the average between them, you get 39bits/s. For English and French, the information rate they recorded was higher, with an average closer to ~45bits/s (just eyeballing their chart). Their results also showed Thai at ~35bits/s. A 10bits/s swing from median to median is pretty huge.
From the paper:
"We collected recordings of 170 native adult speakers of the aforementioned 17 languages, each reading at their normal rate a standardized set of 15 semantically similar texts across the languages (for a total amount of approximately 240,000 syllables). Speakers became familiar with the texts, by reading them several times before being recorded, so that they understand the described situation and minimize reading errors"
"Together, our findings show that while there is wide interspeaker variation in speech and IRs (information rates), this variation is also structured by language. This means that an individual’s speech behavior is not entirely due to individual characteristics but is further constrained by the language being spoken."
"However, languages seem to stably inhabit an optimal range of IRs, away from the extremes that can still be available to individual speakers. Languages achieve this balance through a trade-off between ID (information density) and SR (syllable rate), resulting in a narrower distribution of IRs compared to SRs. In the introduction, we rhetorically asked whether too low or too high an IR would impede communicative and/or cognitive efficiency. Our results here suggest that the answer to both questions is positive and that human communication seems to avoid two extreme sociolinguistic profiles: on the one hand, high ID languages spoken fast by their speakers (“high-fast”), and, on the other, low ID languages spoken slowly by their speakers (“low-slow”)."
https://www.science.org/doi/10.1126/sciadv.aaw2594
This definitely fits with my experience as a language learner studying French and Japanese.
Japanese is definitely a faster spoken language than French, but French words tend to be a lot more verbose and packed with meaning. For Japanese speakers to communicate as much meaning as French speakers, they would need to speak faster.
https://www.science.org/content/article/human-speech-may-hav...
its not the language that communicated its the people. You recruit mediocres in different languages they all communicated in the same mid , 39 bits or whatever. Let two geniuses have a chat or even two teenagers - its a different ball game than 39 bits
Maybe not just spoken language. I would suggest sign language has a similar bit rate as spoken. The evidence is that a sign interpreter conveys about the same information in a conversation in the same time.
(2019)
Some discussion then: https://news.ycombinator.com/item?id=20880789
Universal between languages maybe, but certainly not universal between individual speakers.
There is also a lot of nonverbal data. Imagine the horror of discovering your conversational ML build could hold a plausible verbal conversation only guessing 58% of spoken words accurately... then realizing humans likely fair much worse. =3
"Prisencolinensinainciusol" (Adriano Celentano)
https://www.youtube.com/watch?v=Ax9c8f82QZA
[2019]
The bits are defined in another paper by the authors using "syntagmatic density of information ratio (SDIR)" which seems to be related to the number of syllables required to convey the same information in different languages, using Vietnamese as the baseline.
I cannot wait for independent replication!
https://www.researchgate.net/publication/235971274_A_cross-L...
Adam says to not listen at 2X.
Who is Adam, and why would that be?
Unforgivable that it is not 42 bits per second.
39 bits per second, about twice the speed of Morse code
Guessing there's something very fundamental that the author misunderstands about Morse code.
I would expect you can only transmit so fast in morse code given the need for the dots and dashes to be clearly distinguished from each other and identifiable by the recipient.
Of course, if you know both ends are computers you can just transmit in some other encoding at a much higher rate.
As Morse speeds up, you stop relying on individual dots and dashes and begin recognizing common combinations of letters. Faster still and you are mainly hearing word stems and suffixes.
The faster the information comes at you, the less important any particular bit is, because you have more context with which to autocorrect.
> the dots and dashes to be clearly distinguished from each other
Yes, spaces are part of the morse code spec. It looks like a binary encoding but in fact it's ternary.
We can invent a 5-bit (or 6-bit, to include numbers and punctuation) morse-like code to avoid needing spaces.
The point is, Morse code is as many "bits per second" as you want. You could send Morse code at 10 gigabits per second if you wanted. It is not meaningful to say that Morse code implies a particular data rate.
Historically the metric for Morse code is words per minute. Morse is similar to a Huffman code where common letters are allocated fewer elements, so it's not very meaningful to talk about "bits per second" with respect to Morse even if you do specify the number of words per minute. The number of "bits" will vary based on the letters being transmitted.
I mean if you want that level of pedantry "I" can't send Morse code at 10 gigabits per second. I can make a computer transmit it at that speed, but I am not personally sending it that rate. And, because one generally needs a machine to transmit Morse code, one cold argue that "I" never send Morse code ever.
Yes, computers can transmit any language at any speed (if we include parallelism).
That wasn't contested.
Oh, come on, this is just being coy for no reason. Given the context, it is abundantly clear they mean "normal, human operated morse code".
A skilled operator is around 30 WPM. The average English word is 5 (rounded up) characters. Add one character for the space. That's 180 characters per minute, or 3 character per second. With 37 characters available in morse code, that's log_2(37) or 5.2 bits per charater.
So 15.6 bits per second. Just under half of the 39 bits they got for speech, like they said.