I miss the old Steve Yegge that actually knew how to write coherently. AI psychosis is apparently a hell of a drug... this is pages and pages of complete dreck.
Anti-AI psychosis. Anyone treating AI as a serious topic beyond "it's spicy auto complete/obvious scam/stochastic parrots" is some mixture of evil and stupid and can be dismissed out of hand, regardless of their argument.
I disagree with Steve Yegge's assessment that the curve is close to leveling off. It's not the models, it's the harnesses and the result automation possibilities that are the true unlock. LLMs stabilizing around a current local maximum is actually not much of a big deal. If we just use the models we have today there is so much more unlock available.
We have only just begun our ascent up the hockey stick and the most intense change is yet to come.
The real danger is how big of a gap will exist once the curve does level off. If we are just at the start of the sigmoid curve and starting our ascent, then many jobs will be thrown off by the time we hit the peak and begin to level off.
No politician or corporation is preparing for this sufficiently.
> It's not the models, it's the harnesses and the result automation possibilities that are the true unlock. LLMs stabilizing around a current local maximum is actually not much of a big deal. If we just use the models we have today there is so much more unlock available.
This. And we don't even need LLMs. Orchestrated SLMs that are actually good at their topical areas of training (imagine that) are more in-tune with what the enterprise wants.
> The real danger is how big of a gap will exist once the curve does level off.
I think we've seen that is has. Parlor tricks like what Anthropic is pulling with Fable / Mythos is ultimately a cheat. Basically what you describe up top: the harnesses and underlying automation built in - and that's exactly what Anthropic didn't tell people Fable was, yet it is. And here we are. Is Fable amazing without Claude Code under the covers? Probably not much outside Opus 4.8. The models themselves aren't accelerating as fast as they were years ago, but the marketing clankers have kicked in and are pushing out benchmarks and numbers and all the things, but in reality they're fractions of a point improved. And they still get the obvious things wrong, even when you're paying the inflated high prices. I just can't wait until Anthropic tells everyone that API access to a model is "dangerous", because you know damn well that's coming. There's no moat unless they keep digging. And they're digging right now, that's very clear.
I find it very interesting to read blogs like this which describe and predict the societal impact of AI. However, this impact is all framed in terms of developers. In the end this is a niche (albeit an important one) use of AI. I don’t believe for a moment that software development will be the primary use of LLMs that changes society, at least not in the long term.
I found a lot of interesting, if speculative, thoughts in the article, but...
> Superhuman means unverifiable
is not true for at least large classes of problems. The recent solution of the "unit distance" problem comes to mind, or any future AI-solved math problem that was beyond the capabilities of humans. You can tell it's superhuman (it's doing things humans can't) and you can easily verify its results are correct.
For other classes of problems (eg, policy suggestions for large scale systems like the economy), the point is fair.
Yes this exact line caught my attention. A sign of genius is being able to simplify and teach. If something really is superhumanly smart, it should be able to bring us along with its thinking and explain why its answers are right.
The focus is placed on "AI Literacy", but it seems to use this to just mean 'volume of AI use'. The discussion of the Netflix case study is extra perplexing, since the summary here admits they didn't find any actual productivity improvement, just that only a few hours of "training" could induce on the order of $50/person/day on tokens.
Nuclear weapons… somewhat complicated to build and complicated to maintain, mass-produce, transport, smuggle, hide, and use.
Models on the other hand are just software. You can download Fable / Mythos / Praxia / Concordicon and hide a copy in your shoe. You can make two copies even! One per shoe!
We have lived through decades now of the glorious open source revolution so it’s easy to forget about a different time, in the 1990s, when a black market in CDs jammed full of cracked, slightly off branded copies of Windows / Photoshop / Corel / Office, etc. was all the rage.
Execution isn’t trivial. If anything should be controlled like nuclear weapons, again like the 90s, it ought to be compute hardware. I’m not sure it should.
Not only that, but it also relies on assumption that everything that LLM does cannot be factored into small reasoning core (something like Lean or Prolog, but in more modal/fuzzy logic) plus knowledge base (like Wikipedia).
It then raises a question, which of these two is really the precious thing. (Although there could be a 3rd thing, a memoization of some shortcuts, like tautologies in logic.)
National Security! Selling to the government. Getting tax dollars for something the public will not be able to evaluate. All deals are behind closed doors. Undisturbed grift for all involved.
SaaS is back? I anger coded an open Canva as I'm fed up of predatory "pay to print, or free for guantlet" for my daughter to use it. And I did it—between two weightlifting sets.
I aint even gonna open source it but someone will.
> And since most companies aren’t going to want to send their code and problems to the model vendors, I think the world will learn to live with the models we do have access to.
I guess what Steve is saying is that we don't need all of these data centers then! In his picture he paints a bleak world where, just outside, pollution seems to be the norm. Maybe this is a new rendition of Animal Farm he's working on?
I do agree with him, though... I don't think most companies will want to send their code to OAI, Anthropic or Google either. Not because they can't afford it or are worried about that being stolen (Steve: the point is rarely the code, it's the customer base) but more because they're starting to wake up to the snake oil AI market. Always has been and always will be when you're hocking a subjective and non-deterministic product! We've seen this time and time again within the cyber security space and, well look at that, now the problem space for AI has achieved global charlatanism!
Does it have value? For sure. It does. But it's also not "nuclear weapons" level. OAI was just putting out roulette porn last week yet now we're here? This is all another "hacking tool" strawman that the USG loves to run around with like Chicken Little.
Finally, speaking of the USG. Those in power absolutely love this. Why? Because the USG has always destroyed what could have been great platforms by scope creeping them into mediocrity. Look at the F-35 (JSF). It a mediocre airframe because it had very different requirements for every branch of the military. Just the same way LLMs are not useful for 80% of the enterprise. Nobody gives two shits about a chatbot that can write a haiku. They want a platform that can solve specific problems. Do we really think the enterprise is dumb enough to think that they should redirect a significant spend into LLMs just to be in the exact same position as their competitor? Rewind the clock and start with SLMs and you'd see a much more palatable AI market right now. But instead we're trying to hit an impossible target through brute force and raw dollars. Not through strategic tact but wasteful and egregious weaponization of money that's flowing to some of the most narcissistic people on the face of the earth.
This long-winded screed appears to be an AI proselytizer trying to convince people that, no, actually, you're just holding the model wrong if you don't believe they're exponentially growing in intelligence every generation. The proof? His react client.
Far too dismissive of oss models. This sounds like MS employees talking about linux circa 2002. This attitude would have written off linux in the early 00s as doomed for not "keeping up" with windows. The oss option will always appear behind the curve but they inevitably catch up... if they even need too. AI is no different. The free/oss option will be niche and disregarded by the bigs but it will survive and thrive, just as linux has.
I think that the open source systems being behind is sort of the point, they are the baseline. A paid proprietary system must be better than this. An example.
If you want me to pay for your compiler it must be that much better than the compiler I can get for free that allows me much more freedom of agency in how I use it, if it is not I am going to use the free compiler. repeat for operating systems, word processors, databases and now LLM's
I miss the old Steve Yegge that actually knew how to write coherently. AI psychosis is apparently a hell of a drug... this is pages and pages of complete dreck.
Same. Used to publish some good stuff 10+ years ago.
Why is it dreck? Other than being predictions so probably wrong in that sense?
I do disagree with parts of it, but dreck?
Anti-AI psychosis. Anyone treating AI as a serious topic beyond "it's spicy auto complete/obvious scam/stochastic parrots" is some mixture of evil and stupid and can be dismissed out of hand, regardless of their argument.
I disagree with Steve Yegge's assessment that the curve is close to leveling off. It's not the models, it's the harnesses and the result automation possibilities that are the true unlock. LLMs stabilizing around a current local maximum is actually not much of a big deal. If we just use the models we have today there is so much more unlock available.
We have only just begun our ascent up the hockey stick and the most intense change is yet to come.
The real danger is how big of a gap will exist once the curve does level off. If we are just at the start of the sigmoid curve and starting our ascent, then many jobs will be thrown off by the time we hit the peak and begin to level off.
No politician or corporation is preparing for this sufficiently.
> It's not the models, it's the harnesses and the result automation possibilities that are the true unlock. LLMs stabilizing around a current local maximum is actually not much of a big deal. If we just use the models we have today there is so much more unlock available.
This. And we don't even need LLMs. Orchestrated SLMs that are actually good at their topical areas of training (imagine that) are more in-tune with what the enterprise wants.
> The real danger is how big of a gap will exist once the curve does level off.
I think we've seen that is has. Parlor tricks like what Anthropic is pulling with Fable / Mythos is ultimately a cheat. Basically what you describe up top: the harnesses and underlying automation built in - and that's exactly what Anthropic didn't tell people Fable was, yet it is. And here we are. Is Fable amazing without Claude Code under the covers? Probably not much outside Opus 4.8. The models themselves aren't accelerating as fast as they were years ago, but the marketing clankers have kicked in and are pushing out benchmarks and numbers and all the things, but in reality they're fractions of a point improved. And they still get the obvious things wrong, even when you're paying the inflated high prices. I just can't wait until Anthropic tells everyone that API access to a model is "dangerous", because you know damn well that's coming. There's no moat unless they keep digging. And they're digging right now, that's very clear.
I find it very interesting to read blogs like this which describe and predict the societal impact of AI. However, this impact is all framed in terms of developers. In the end this is a niche (albeit an important one) use of AI. I don’t believe for a moment that software development will be the primary use of LLMs that changes society, at least not in the long term.
I found a lot of interesting, if speculative, thoughts in the article, but...
> Superhuman means unverifiable
is not true for at least large classes of problems. The recent solution of the "unit distance" problem comes to mind, or any future AI-solved math problem that was beyond the capabilities of humans. You can tell it's superhuman (it's doing things humans can't) and you can easily verify its results are correct.
For other classes of problems (eg, policy suggestions for large scale systems like the economy), the point is fair.
Yes this exact line caught my attention. A sign of genius is being able to simplify and teach. If something really is superhumanly smart, it should be able to bring us along with its thinking and explain why its answers are right.
NP problems can be verified in P time.
I'm pretty confused by most of the article.
The focus is placed on "AI Literacy", but it seems to use this to just mean 'volume of AI use'. The discussion of the Netflix case study is extra perplexing, since the summary here admits they didn't find any actual productivity improvement, just that only a few hours of "training" could induce on the order of $50/person/day on tokens.
That seems... the opposite of literacy?
Measuring education in dollars spent is rather consistent with measuring the productivity of LLMs in lines of code excreted.
Nuclear weapons… somewhat complicated to build and complicated to maintain, mass-produce, transport, smuggle, hide, and use.
Models on the other hand are just software. You can download Fable / Mythos / Praxia / Concordicon and hide a copy in your shoe. You can make two copies even! One per shoe!
We have lived through decades now of the glorious open source revolution so it’s easy to forget about a different time, in the 1990s, when a black market in CDs jammed full of cracked, slightly off branded copies of Windows / Photoshop / Corel / Office, etc. was all the rage.
Execution isn’t trivial. If anything should be controlled like nuclear weapons, again like the 90s, it ought to be compute hardware. I’m not sure it should.
Not only that, but it also relies on assumption that everything that LLM does cannot be factored into small reasoning core (something like Lean or Prolog, but in more modal/fuzzy logic) plus knowledge base (like Wikipedia).
It then raises a question, which of these two is really the precious thing. (Although there could be a 3rd thing, a memoization of some shortcuts, like tautologies in logic.)
If the most expensive models yet to come will end up behind bars by default, what is the economic incentive to make them in the first place?
National Security! Selling to the government. Getting tax dollars for something the public will not be able to evaluate. All deals are behind closed doors. Undisturbed grift for all involved.
I wouldn’t be surprised, good point. But closed door deals in the trillions? Because we’re coming up on those numbers.
SaaS is back? I anger coded an open Canva as I'm fed up of predatory "pay to print, or free for guantlet" for my daughter to use it. And I did it—between two weightlifting sets.
I aint even gonna open source it but someone will.
> And since most companies aren’t going to want to send their code and problems to the model vendors, I think the world will learn to live with the models we do have access to.
I guess what Steve is saying is that we don't need all of these data centers then! In his picture he paints a bleak world where, just outside, pollution seems to be the norm. Maybe this is a new rendition of Animal Farm he's working on?
I do agree with him, though... I don't think most companies will want to send their code to OAI, Anthropic or Google either. Not because they can't afford it or are worried about that being stolen (Steve: the point is rarely the code, it's the customer base) but more because they're starting to wake up to the snake oil AI market. Always has been and always will be when you're hocking a subjective and non-deterministic product! We've seen this time and time again within the cyber security space and, well look at that, now the problem space for AI has achieved global charlatanism!
Does it have value? For sure. It does. But it's also not "nuclear weapons" level. OAI was just putting out roulette porn last week yet now we're here? This is all another "hacking tool" strawman that the USG loves to run around with like Chicken Little.
Finally, speaking of the USG. Those in power absolutely love this. Why? Because the USG has always destroyed what could have been great platforms by scope creeping them into mediocrity. Look at the F-35 (JSF). It a mediocre airframe because it had very different requirements for every branch of the military. Just the same way LLMs are not useful for 80% of the enterprise. Nobody gives two shits about a chatbot that can write a haiku. They want a platform that can solve specific problems. Do we really think the enterprise is dumb enough to think that they should redirect a significant spend into LLMs just to be in the exact same position as their competitor? Rewind the clock and start with SLMs and you'd see a much more palatable AI market right now. But instead we're trying to hit an impossible target through brute force and raw dollars. Not through strategic tact but wasteful and egregious weaponization of money that's flowing to some of the most narcissistic people on the face of the earth.
This long-winded screed appears to be an AI proselytizer trying to convince people that, no, actually, you're just holding the model wrong if you don't believe they're exponentially growing in intelligence every generation. The proof? His react client.
Is this a troll
He's one of the chief hallucinators, show some respect.
GPT-2 was already extremely dangerous.
So was the computational capabilities of Playstation 2. It could be used to simulate nuclear weapons, I heard.
Also Dario was right! As right as a broken clock.
This article, just in the first paragraph, seems fueled by an episode of AI psychosis.
Far too dismissive of oss models. This sounds like MS employees talking about linux circa 2002. This attitude would have written off linux in the early 00s as doomed for not "keeping up" with windows. The oss option will always appear behind the curve but they inevitably catch up... if they even need too. AI is no different. The free/oss option will be niche and disregarded by the bigs but it will survive and thrive, just as linux has.
I think that the open source systems being behind is sort of the point, they are the baseline. A paid proprietary system must be better than this. An example.
If you want me to pay for your compiler it must be that much better than the compiler I can get for free that allows me much more freedom of agency in how I use it, if it is not I am going to use the free compiler. repeat for operating systems, word processors, databases and now LLM's
[dead]