> Picture an LLM as a stack of books. Each book is a layer of basic computational units known as neurons, with each neuron in one layer passing information to the neurons in the layers above. The books at the bottom of the stack are the input layers, which process the text coming into the model. The books at the top are the output layers, which prepare the text that the model is about to produce.
How about instead we picture an LLM as a stack of pancakes? This would have been an equally useful analogy, because the author doesn’t even use any relatable features of a book.
All of the actual examples seem rather trivial, and exactly what you would expect to be happening without a 'hidden space'. Happy to be wrong, but this looks like they're reading a whole new hidden space into not very much.
> Picture an LLM as a stack of books. Each book is a layer of basic computational units known as neurons, with each neuron in one layer passing information to the neurons in the layers above. The books at the bottom of the stack are the input layers, which process the text coming into the model. The books at the top are the output layers, which prepare the text that the model is about to produce.
How about instead we picture an LLM as a stack of pancakes? This would have been an equally useful analogy, because the author doesn’t even use any relatable features of a book.
Maybe we've just been using books wrong the entire time.
Maybe this is where all the books an LLM consumed during training goes.
I wouldn’t quite label this blogspam, but the original Anthropic writeup (and paper) are a much better read: https://www.anthropic.com/research/global-workspace
HN discussion: https://news.ycombinator.com/item?id=48808002
All of the actual examples seem rather trivial, and exactly what you would expect to be happening without a 'hidden space'. Happy to be wrong, but this looks like they're reading a whole new hidden space into not very much.