We really need to band together to fund / sponsor targeted inducement prizes (a la Nobel laureate Michael Kremer) for open models.
Every 6-12 months, give out $200K to the first model to hit a min threshold on a set of ~5-10 hard benchmarks (+ perhaps one secret benchmark) using a total of 16GB / 32GB / 64GB / 128GB of VRAM (at a min context length of 200K), then move the threshold up. Quantization etc. is dealers choice, it just needs to nail the benchmark on a reference machine by using exactly that much VRAM (no mapping to RAM / disk etc.)
You could crowdsource the funding, and cross subsidize by adding targeted prizes focused on corporate needs (the classic one is PDF processing benchmarks), and say that 25% of each corporate prize funding also flows into the general prize pool.
For a lot of these open-source model companies, it's less about the $s (though $200K is nothing to sneeze at), it's the clear recognition that helps their model efforts stand out, gain usage etc.
I think the Korean government did have a competition like this, I remember last year we got a bunch of models released at the same time to make the cut for the next stage. The models weren't anything to write home about, IMO.
Having it with clear hw requirements tiers is a nice differentiator. The only issue is that the benchmarks would 100% need to be closed, no other way around it. And then you have the issue of creating and curating good evals for every "stage" of the project. That's a hard task even for "honest" lab-internal evals. And you'd have to publish those evals after each round (for trust purposes), and start over for the next round. Doable, but it would cost a lot (probably more than the prize pools) and you'd have to keep doing this.
It was a competition organized by the Korean government but the directive wasn't for the same cause as the writer. It was more for constructing Sovereign AI for the country. Also, all models except Exaone had some weights copied from Chinese models, and from the corporate point of view, developing from-scratch model is not cheap despite the financial support from the government.
Yes, I hope the open model communities will someday be able to run current frontier models which will be able to handle autonomous tasks and the hardware to run it will be served at consumer level; however, like how recycling isn't profitable, no companies will fully commit to the movement. Don't get it twisted, I don't have a solution but maybe a global scale movement to liberate knowledge-library could suffice.
This seems like a good idea but also just fun. I can’t train a frontier model but maybe I could compete in the 16 GB tier. I would suspect there are a ton of optimizations out there for the taking that aren’t being considered because frontier models are way above these weight classes
The private AI companies should be forced to release the models as open weights with a license (I.e no commercial use) due the high risk they present and the data they basically steal from everyone to train their models.
This should be the safety push not the regulatory capture that Dario is trying.
All I see if a bunch of people who haven't loaded an ad in 20 years and have a 5TB collection of pirated movies and music suddenly decrying that LLM's doing next token prediction over a dataset is theft.
You don't get to change your mind 20 years into "I'm never going to pay for anything binary" ethos (cough look at what this post is cough) that has dominated the internet for decades. If people are genuinely upset about LLMs training on all available data without compensation, all I can say is "Reap what you sow".
Ok then can the feds bust the executives of these companies like they did with Kim Dotcom? It’s not like Dario trained his AI in the bedroom or do a small business by letting people scrapping IP work to train their models. He is actively stealing and selling the stolen work and even preaches us safety measures: how to protect us from our own knowledge that he stole!
It would be all fine and good if they kept it non-commercial, but when you start charging money for it that's when you're stepping out of bounds. If they want to benefit from fair use, they should contribute their results back to the public domain. Getting it for free and then commercializing it is what's unethical.
>Getting it for free and then commercializing it is what's unethical.
Like all those unethical software devs taking salaries for turning stackoverflow into products? I suppose the blood still flows since now they just use LLM output?
Any way you try and slice LLM morality, you end up with "It's bad because they are not me" reasons. "When I monetize information I get for free, it's good, when they monetize information they get for free, it's bad"
It is kind of incredible that you're not focused at all on the copyright holders, instead focusing on random tech people you had online disagreements with.
Artists and writers got screwed first by piracy, then by generative AI. They didn't sow anything. They just got reaped.
And the only thing the copyright hypocrites are "reaping" is a feeling of hypocrisy. Congrats for pointing that out. Your comment is simply myopic.
Agreed. Especially since now competitors have more difficulties getting the same advantage. They don't have to do so immediately and perhaps not their specific tuning. But the weights of the raw training data at least should be publicised.
Wow, thank you for this! I'd forgotten about enclosure for some years.
> The law locks up the man or woman / Who steals the goose from off the common / But leaves the greater villain loose / Who steals the common from the goose.
FOSS is the wrong analogy. Building frontier LLMs isn’t primarily an engineering discipline, it’s a scientific research program.
Of course we do have basically open source research programs, including most universities and big projects like CERN. But AI grew up in universities until it transpired that sufficient capital could only be found in the private sector.
It would be possible to make a decent publicly funded AI research program. But it would look more like the Manhattan or Apollo projects (which frontier labs already model themselves after) than some extra research grants for universities.
The Manhatten project cost about $40 billion in total adjusted for inflation. Anthropic's latest funding round alone raised $65 billion.
The entire Apollo project at the peak of the cold war cost about $300 billion in today's dollars. That's approximately what OpenAI and Anthropic have raised together in total until now.
I don't think governments can supply this amount of money for AI in the current political and economic climate. The LHC cost less than $10 billion by comparison and it was spread out over a much longer timeframe.
> I don't think governments can supply this amount of money for AI in the current political and economic climate.
I'm a believer in Keynes' "anything we can do we can afford". It could be afforded .. if there was a sufficiently good reason. And there isn't. This is way behind "governments, especially the EU, should have a sovereign cloud". It is also way behind "governments need to keep global warming below 2C by the end of the century" and "governments need to ensure affordable energy", objectives which the current AI buildout is in direct conflict with.
This is before we get into the question of whether AI has net positive social value in non-software use cases. Even in software the case for AI is explicitly job-destroying and raising electricity prices for everyone else.
There are things you can't do - even as a society - when the costs start ballooning beyond the GDP of smaller European nations. I mean, yes, in an emergency you could cancel all social security and public infrastructure spending and instead dump it into AI. But even then it will take several big nations to foot the bill if you want to still have a country left after you're done. With the level of political heckle everywhere, this is just not going to happen, because it would need unanimous support from all sides.
Govts can take it over. Corporations dont maintain standing armies. So there is a pecking order that corps have never been able to invert. They rely on Govt for their own security.
History is full of these take overs if there is risk(usually happens after some catastrophe). See the finance sector(once upon a time private banks invented and printed money), nuclear industry, febrtilizer industry, crypto, a whole bunch of processes in biotech/synthbio. Classic textbook example is the East India Company. It was much richer that the British Govt or the King.
> I don't think governments can supply this amount of money for AI in the current political and economic climate.
You understand how the system works if you’re thinking in terms of government/non-government. The current political and economic state is not a bug, it’s by design which serves a purpose.
Yes, if the US was modeled like the USSR they could extract 100% of everyone’s private assets and the politburo could spend them on whatever moonshot projects they wanted to, unchecked by silly democratic votes.
But…the USSR and that entire model failed spectacularly? So not sure what you’re getting at here. Is there some fantasy economic model you believe you’ve innovated that will lead to utopia and the end of resource scarcity?
> But AI grew up in universities until it transpired that sufficient capital could only be found in the private sector.
One way of looking at it. Another is that AI research progressed within universities, but it was only until recently that the private sector saw the profit potential when combined with modern CPU/GPU technology.
You could argue we're saying the same thing, but I think these angles are different. An academic research programme would not have spent billions on a datacentre to provide AI for free to the general public, for example.
I agree an academic research program wouldn’t spend billions on a datacenter, but that’s just a reason why foss AI as described in this article won’t work - because big compute is required to do anything of real relevance.
They already invest in open-source AI, but nothing is truly free. Commercial AI will usually dominate because devs are paid to make it their primary effort. Goodwill and part-time contributions cannot reliably compete with livelihood and profit incentives.
That's what people said about operating systems, and databases, and compilers, and so many other big complicated categories of software that over time became increasingly dominated by OSS
OSS only dominates for software that is commoditized and the published computer science research for that software domain is close to the frontier.
OSS struggles at being relevant when software is non-commodity e.g. office suites. In software domains like databases where the state-of-the-art computer science research is often unpublished, OSS struggles to be relevant at the higher end of the market on technical merits.
When deciding what should be OSS, it is useful to consider the preconditions that have made it successful.
I personally expect token production to commoditized like mobile data. It's already happening.
See open weights gaining adoption, OpenAi talking about how 5.6 is cheaper than Fable, people are taking multiple approaches to reduce their token spend, expectations for progress in hardware and algos, and certain Ai leaders talking about how token prices should be 10-100x lower than they are.
LLMs are nearly commoditized already. I can switch between a dozen of them from four different providers as easily as flipping a toggle in my VSCode editor.
OSS does not necessarily mean the contributions are from "goodwill or part-time contributions". In fact, I would wager the most widely used OSS software is largely written by contributors paid to do so by corporations. At least for Linux, about 80% - 85% of contributions are from developers paid to do it (https://newsletter.pragmaticengineer.com/p/how-linux-is-buil...)
Corporations have had many reasons to invest their money in open source software -- custom requirements, marketing / developer mindshare, commoditizing complements -- but as cutting edge LLMs get more and more expensive to train, you'd be hard-pressed to find corporations who will put in that kind of money if they cannot recoup their investments.
I think the main problem in LLM models is that you can not make a PR to an open source project to tweak some training parameters, prove it is an improvement and merge it.
If you can not run the training yourself you can not contribute. So open source contribution model does not work. All examples you gave have a fairly low threshold of capital expenditure required to be a contributor (basically a laptop).
Even back in the 90s a person could get a standard, but powerful, PC to do these things. The one exception was 3d graphics which took quite some time to become affordable and even there it was a single one-time expenditure (a workstation) per contributor.
For normal OSS the only competition between contributors was for attention of maintainers to review and accept patches.
In an open-source LLM model contributors would compete with each other for computing resources for model tweaks and changes. The alternative model is that the contributor pays for the compute, but that increases the bar really high for contributions.
on the small chance that the four billionaires who currently have near-exclusive control of closed sota models, (that is altman, amodei, zuckerberg and musk), are not fleecing their investors and actually build AGI, closed source leaves a choice of powerful government or powerful oligopoly/monarchy.
further explanation of this list:
musk - structural command
zuckerberg - structural command
altman - de facto command after purging rivals and privatisation, loyalty of personnel
amodei - influential, could potentially overthrow current governance
“The economists need to start charting this out, if we are in a post scarcity world, how does everyone benefit from that?, obviously its not correct for just a few people or a few companies or even a few nations to be benefiting from this technology, it has to broadly accrue the benefits to everyone, but how is that gonna be done? We really need answers now. ” -Demmis Hassabis <Google Deep mind>
Just because a software is closed-source doesn't mean the knowledge can't be shared. You don't need to see the underlying code to explain to someone architectural patterns or best practices.
The library analogy in the scenario would hold true if LLM providers refused to answer any questions about RL or Transformers.
I am a big proponent of open-source open-weight models, but mostly because I think it's just a better product. We've seen that they are much cheaper to train and operate. Frontier intelligence might not be needed for most tasks. Just let the market decide. My bet is that LLMs will become analogous to programming languages, and big labs will make their money by fine-tuning models for very specific use cases or by deploying them for customers.
Listening to those pushing AI, why should we fund open source developers when we can reproduce their work with a few judiciously chosen LLM prompts? Perhaps the AI companies should fund FOSS so as to get more solutions to memorize but normals?
The library analogy works (for me), but the uncomfortable part is that most "open" models are closer to receiving a compiled binary than receiving the library
> Even if the current approaches will continue to scale, this would be as if in the early days of computing, perhaps someone invented a bubble sort for sorting numbers (an n-squared algorithm), and the tech companies at the time decided they were going to build vast data centers to sort numbers and not bother to figure out that there's an n-log-n way of doing it <laughs>
...to which I have to say: yes, definitely! And he's right about open-source AI too.
Yes, there's a present shortage of usable frontier compute, but that doesn't establish that every proposed data center will earn a decent return, or that today’s hardware + model architecture will remain economically competitive, or more to Siegel's point, that algorithmic efficiency could not dramatically reduce compute requirements.
To borrow a real example from a prior boom, railroads were congested during the initial build-out while people simultaneously funded and built too many railroads for future demand. Likewise, Anthropic et al can be compute-starved now while the industry as a whole is overbuilding expensive, depreciating infrastructure.
I'm not sure comparing this one to the "real life" previous ones is worth doing. Digital has a way of scaling that IRL doesn't. You don't get exponetials with humans, but you do with models. Scale has shown time and time again that it works. Despite everything you've read in '25 about walls, lack of x, and so on it still does. And the more you have you find ways of using it even more. RL can use ~ 7:1 ratio of inference : training. That is, you run 7 nodes of inference for every node of training. And it keeps improving. Every algorithmic find on inference (and there've been plenty) translates in improvements. And we're not seeing any signs of this not continuing. And then there's the "zero" method, that hasn't been tried at true scale yet. Bitter lesson and all that.
I'll grant that the scaling laws have held so far, and that the bitter lesson is yet to be proven wrong. However, that doesn't exclude the possibility that the current architecture's S-curve will flatten, and a new one will replace it, which was Siegel's point. At greater layers of abstraction, what is observed is a raw capabilities/intelligence increase, but what form it takes is subject to change. We'll see where it all goes, the only way out is through.
Let me re-iterate the main lesson of decades of FOSS work: the advantages of open collaboration and knowledge-sharing are so enormous that FOSS software wins out eventually even if financial interest are stacked against it.
I fully agree with this article - please let's skip the chapter of closed and enshittified AI and go for the good stuff directly!
I must have missed the press release when Two Sigma open-sourced their algorithmic trading models so that the rest of us could make the best stock investments for our retirements. /s
No, I want govt to tax themmore. So far frontier AI companies produce negative value to near everyone (by sheer power cost increase it adds essentially tax to every other business) but themselves economy wise.
Yeah, wooho, new model found a bunch of bugs, now the bad guys can do it too so security expenses spiked! It's only good for shovel sellers.
We really need to band together to fund / sponsor targeted inducement prizes (a la Nobel laureate Michael Kremer) for open models.
Every 6-12 months, give out $200K to the first model to hit a min threshold on a set of ~5-10 hard benchmarks (+ perhaps one secret benchmark) using a total of 16GB / 32GB / 64GB / 128GB of VRAM (at a min context length of 200K), then move the threshold up. Quantization etc. is dealers choice, it just needs to nail the benchmark on a reference machine by using exactly that much VRAM (no mapping to RAM / disk etc.)
You could crowdsource the funding, and cross subsidize by adding targeted prizes focused on corporate needs (the classic one is PDF processing benchmarks), and say that 25% of each corporate prize funding also flows into the general prize pool.
For a lot of these open-source model companies, it's less about the $s (though $200K is nothing to sneeze at), it's the clear recognition that helps their model efforts stand out, gain usage etc.
I think the Korean government did have a competition like this, I remember last year we got a bunch of models released at the same time to make the cut for the next stage. The models weren't anything to write home about, IMO.
Having it with clear hw requirements tiers is a nice differentiator. The only issue is that the benchmarks would 100% need to be closed, no other way around it. And then you have the issue of creating and curating good evals for every "stage" of the project. That's a hard task even for "honest" lab-internal evals. And you'd have to publish those evals after each round (for trust purposes), and start over for the next round. Doable, but it would cost a lot (probably more than the prize pools) and you'd have to keep doing this.
It was a competition organized by the Korean government but the directive wasn't for the same cause as the writer. It was more for constructing Sovereign AI for the country. Also, all models except Exaone had some weights copied from Chinese models, and from the corporate point of view, developing from-scratch model is not cheap despite the financial support from the government.
Yes, I hope the open model communities will someday be able to run current frontier models which will be able to handle autonomous tasks and the hardware to run it will be served at consumer level; however, like how recycling isn't profitable, no companies will fully commit to the movement. Don't get it twisted, I don't have a solution but maybe a global scale movement to liberate knowledge-library could suffice.
Yeah, I suspect the $200K check would be the cheapest line item in the whole thing
This seems like a good idea but also just fun. I can’t train a frontier model but maybe I could compete in the 16 GB tier. I would suspect there are a ton of optimizations out there for the taking that aren’t being considered because frontier models are way above these weight classes
I would just add a reproducibility requirement and avoid keeping the exact same benchmarks for too long
Not sure that would even cover power for training
The private AI companies should be forced to release the models as open weights with a license (I.e no commercial use) due the high risk they present and the data they basically steal from everyone to train their models. This should be the safety push not the regulatory capture that Dario is trying.
> the data they basically steal from everyone
What's the precedent?
All I see if a bunch of people who haven't loaded an ad in 20 years and have a 5TB collection of pirated movies and music suddenly decrying that LLM's doing next token prediction over a dataset is theft.
You don't get to change your mind 20 years into "I'm never going to pay for anything binary" ethos (cough look at what this post is cough) that has dominated the internet for decades. If people are genuinely upset about LLMs training on all available data without compensation, all I can say is "Reap what you sow".
Ok then can the feds bust the executives of these companies like they did with Kim Dotcom? It’s not like Dario trained his AI in the bedroom or do a small business by letting people scrapping IP work to train their models. He is actively stealing and selling the stolen work and even preaches us safety measures: how to protect us from our own knowledge that he stole!
It would be all fine and good if they kept it non-commercial, but when you start charging money for it that's when you're stepping out of bounds. If they want to benefit from fair use, they should contribute their results back to the public domain. Getting it for free and then commercializing it is what's unethical.
>Getting it for free and then commercializing it is what's unethical.
Like all those unethical software devs taking salaries for turning stackoverflow into products? I suppose the blood still flows since now they just use LLM output?
Any way you try and slice LLM morality, you end up with "It's bad because they are not me" reasons. "When I monetize information I get for free, it's good, when they monetize information they get for free, it's bad"
I am not convinced by your retort, which essentially boils down to “some people pirate, so its ironic some people, maybe different, are mad at dario!”
Artists and writers got screwed first by piracy, then by generative AI. They didn't sow anything. They just got reaped.
And the only thing the copyright hypocrites are "reaping" is a feeling of hypocrisy. Congrats for pointing that out. Your comment is simply myopic.
> the data they basically steal from everyone
Agreed. Especially since now competitors have more difficulties getting the same advantage. They don't have to do so immediately and perhaps not their specific tuning. But the weights of the raw training data at least should be publicised.
> release the models as open weights with a license (I.e no commercial use) due the high risk they present
For some kinds of risk (ex: walking people through on how to make infectious bioweapons) an open-weights approach would increase risk.
Good idea! I'd never heard that. Very interesting.
But wouldn't this help china build models just as good as ours immediately? Wouldn't it make the investment in training a model worth a lot less?
> no commercial use
Why? It's not like they did the hard work. It's disgraceful that this kind of commons enclosure has been allowed in the first place.
Wow, thank you for this! I'd forgotten about enclosure for some years.
> The law locks up the man or woman / Who steals the goose from off the common / But leaves the greater villain loose / Who steals the common from the goose.
In an earlier phase of AI I might have agreed but now the publicly available data they’ve trained on is increasingly useless for the frontier.
RL/post-training is now a much bigger part of it, with that being largely proprietary and expensive work that the open source model just won’t fund.
I'd rather they published the training data.
Then we can verify that there's nothing nasty hiding in it.
FOSS is the wrong analogy. Building frontier LLMs isn’t primarily an engineering discipline, it’s a scientific research program.
Of course we do have basically open source research programs, including most universities and big projects like CERN. But AI grew up in universities until it transpired that sufficient capital could only be found in the private sector.
It would be possible to make a decent publicly funded AI research program. But it would look more like the Manhattan or Apollo projects (which frontier labs already model themselves after) than some extra research grants for universities.
The Manhatten project cost about $40 billion in total adjusted for inflation. Anthropic's latest funding round alone raised $65 billion.
The entire Apollo project at the peak of the cold war cost about $300 billion in today's dollars. That's approximately what OpenAI and Anthropic have raised together in total until now.
I don't think governments can supply this amount of money for AI in the current political and economic climate. The LHC cost less than $10 billion by comparison and it was spread out over a much longer timeframe.
> I don't think governments can supply this amount of money for AI in the current political and economic climate.
I'm a believer in Keynes' "anything we can do we can afford". It could be afforded .. if there was a sufficiently good reason. And there isn't. This is way behind "governments, especially the EU, should have a sovereign cloud". It is also way behind "governments need to keep global warming below 2C by the end of the century" and "governments need to ensure affordable energy", objectives which the current AI buildout is in direct conflict with.
This is before we get into the question of whether AI has net positive social value in non-software use cases. Even in software the case for AI is explicitly job-destroying and raising electricity prices for everyone else.
There are things you can't do - even as a society - when the costs start ballooning beyond the GDP of smaller European nations. I mean, yes, in an emergency you could cancel all social security and public infrastructure spending and instead dump it into AI. But even then it will take several big nations to foot the bill if you want to still have a country left after you're done. With the level of political heckle everywhere, this is just not going to happen, because it would need unanimous support from all sides.
Govts can take it over. Corporations dont maintain standing armies. So there is a pecking order that corps have never been able to invert. They rely on Govt for their own security.
History is full of these take overs if there is risk(usually happens after some catastrophe). See the finance sector(once upon a time private banks invented and printed money), nuclear industry, febrtilizer industry, crypto, a whole bunch of processes in biotech/synthbio. Classic textbook example is the East India Company. It was much richer that the British Govt or the King.
> I don't think governments can supply this amount of money for AI in the current political and economic climate.
You understand how the system works if you’re thinking in terms of government/non-government. The current political and economic state is not a bug, it’s by design which serves a purpose.
Remember, the purpose of a system is what it does
Yes, if the US was modeled like the USSR they could extract 100% of everyone’s private assets and the politburo could spend them on whatever moonshot projects they wanted to, unchecked by silly democratic votes.
But…the USSR and that entire model failed spectacularly? So not sure what you’re getting at here. Is there some fantasy economic model you believe you’ve innovated that will lead to utopia and the end of resource scarcity?
> But AI grew up in universities until it transpired that sufficient capital could only be found in the private sector.
One way of looking at it. Another is that AI research progressed within universities, but it was only until recently that the private sector saw the profit potential when combined with modern CPU/GPU technology.
You could argue we're saying the same thing, but I think these angles are different. An academic research programme would not have spent billions on a datacentre to provide AI for free to the general public, for example.
I agree an academic research program wouldn’t spend billions on a datacenter, but that’s just a reason why foss AI as described in this article won’t work - because big compute is required to do anything of real relevance.
They already invest in open-source AI, but nothing is truly free. Commercial AI will usually dominate because devs are paid to make it their primary effort. Goodwill and part-time contributions cannot reliably compete with livelihood and profit incentives.
That's what people said about operating systems, and databases, and compilers, and so many other big complicated categories of software that over time became increasingly dominated by OSS
OSS only dominates for software that is commoditized and the published computer science research for that software domain is close to the frontier.
OSS struggles at being relevant when software is non-commodity e.g. office suites. In software domains like databases where the state-of-the-art computer science research is often unpublished, OSS struggles to be relevant at the higher end of the market on technical merits.
When deciding what should be OSS, it is useful to consider the preconditions that have made it successful.
I personally expect token production to commoditized like mobile data. It's already happening.
See open weights gaining adoption, OpenAi talking about how 5.6 is cheaper than Fable, people are taking multiple approaches to reduce their token spend, expectations for progress in hardware and algos, and certain Ai leaders talking about how token prices should be 10-100x lower than they are.
LLMs are nearly commoditized already. I can switch between a dozen of them from four different providers as easily as flipping a toggle in my VSCode editor.
OSS does not necessarily mean the contributions are from "goodwill or part-time contributions". In fact, I would wager the most widely used OSS software is largely written by contributors paid to do so by corporations. At least for Linux, about 80% - 85% of contributions are from developers paid to do it (https://newsletter.pragmaticengineer.com/p/how-linux-is-buil...)
Corporations have had many reasons to invest their money in open source software -- custom requirements, marketing / developer mindshare, commoditizing complements -- but as cutting edge LLMs get more and more expensive to train, you'd be hard-pressed to find corporations who will put in that kind of money if they cannot recoup their investments.
I think the main problem in LLM models is that you can not make a PR to an open source project to tweak some training parameters, prove it is an improvement and merge it.
If you can not run the training yourself you can not contribute. So open source contribution model does not work. All examples you gave have a fairly low threshold of capital expenditure required to be a contributor (basically a laptop).
Even back in the 90s a person could get a standard, but powerful, PC to do these things. The one exception was 3d graphics which took quite some time to become affordable and even there it was a single one-time expenditure (a workstation) per contributor.
For normal OSS the only competition between contributors was for attention of maintainers to review and accept patches.
In an open-source LLM model contributors would compete with each other for computing resources for model tweaks and changes. The alternative model is that the contributor pays for the compute, but that increases the bar really high for contributions.
all those have either a consulting company around it or few big corporate contributors
AGI is not software
on the small chance that the four billionaires who currently have near-exclusive control of closed sota models, (that is altman, amodei, zuckerberg and musk), are not fleecing their investors and actually build AGI, closed source leaves a choice of powerful government or powerful oligopoly/monarchy.
further explanation of this list:
musk - structural command
zuckerberg - structural command
altman - de facto command after purging rivals and privatisation, loyalty of personnel
amodei - influential, could potentially overthrow current governance
if i'm wrong i would like to understand
“The economists need to start charting this out, if we are in a post scarcity world, how does everyone benefit from that?, obviously its not correct for just a few people or a few companies or even a few nations to be benefiting from this technology, it has to broadly accrue the benefits to everyone, but how is that gonna be done? We really need answers now. ” -Demmis Hassabis <Google Deep mind>
Just because a software is closed-source doesn't mean the knowledge can't be shared. You don't need to see the underlying code to explain to someone architectural patterns or best practices.
The library analogy in the scenario would hold true if LLM providers refused to answer any questions about RL or Transformers.
I am a big proponent of open-source open-weight models, but mostly because I think it's just a better product. We've seen that they are much cheaper to train and operate. Frontier intelligence might not be needed for most tasks. Just let the market decide. My bet is that LLMs will become analogous to programming languages, and big labs will make their money by fine-tuning models for very specific use cases or by deploying them for customers.
I would not count on the market alone. Like customers do not always choose the technically best or cheapest option
Listening to those pushing AI, why should we fund open source developers when we can reproduce their work with a few judiciously chosen LLM prompts? Perhaps the AI companies should fund FOSS so as to get more solutions to memorize but normals?
just remove "AI" from title and its good:
— Governments, companies, nonprofits should invest in free, open source.
The library analogy works (for me), but the uncomfortable part is that most "open" models are closer to receiving a compiled binary than receiving the library
I wonder if some sort of member owned cooperative would be the way forward if we the people want to retain any control.
Ah, this from the same David Siegel who said almost 2 yrs ago (in a talk found here: https://youtu.be/0z60xUDo-NI?si=PTDe11-sn2P53qo5&t=420) that the AI data center buildout was premature because:
> Even if the current approaches will continue to scale, this would be as if in the early days of computing, perhaps someone invented a bubble sort for sorting numbers (an n-squared algorithm), and the tech companies at the time decided they were going to build vast data centers to sort numbers and not bother to figure out that there's an n-log-n way of doing it <laughs>
...to which I have to say: yes, definitely! And he's right about open-source AI too.
> AI data center buildout was premature
Ask Amodei how he feels about going to spaceman bad for compute that he couldn't find anywhere else in the market.
Yes, there's a present shortage of usable frontier compute, but that doesn't establish that every proposed data center will earn a decent return, or that today’s hardware + model architecture will remain economically competitive, or more to Siegel's point, that algorithmic efficiency could not dramatically reduce compute requirements.
To borrow a real example from a prior boom, railroads were congested during the initial build-out while people simultaneously funded and built too many railroads for future demand. Likewise, Anthropic et al can be compute-starved now while the industry as a whole is overbuilding expensive, depreciating infrastructure.
I'm not sure comparing this one to the "real life" previous ones is worth doing. Digital has a way of scaling that IRL doesn't. You don't get exponetials with humans, but you do with models. Scale has shown time and time again that it works. Despite everything you've read in '25 about walls, lack of x, and so on it still does. And the more you have you find ways of using it even more. RL can use ~ 7:1 ratio of inference : training. That is, you run 7 nodes of inference for every node of training. And it keeps improving. Every algorithmic find on inference (and there've been plenty) translates in improvements. And we're not seeing any signs of this not continuing. And then there's the "zero" method, that hasn't been tried at true scale yet. Bitter lesson and all that.
I'll grant that the scaling laws have held so far, and that the bitter lesson is yet to be proven wrong. However, that doesn't exclude the possibility that the current architecture's S-curve will flatten, and a new one will replace it, which was Siegel's point. At greater layers of abstraction, what is observed is a raw capabilities/intelligence increase, but what form it takes is subject to change. We'll see where it all goes, the only way out is through.
Let me re-iterate the main lesson of decades of FOSS work: the advantages of open collaboration and knowledge-sharing are so enormous that FOSS software wins out eventually even if financial interest are stacked against it.
I fully agree with this article - please let's skip the chapter of closed and enshittified AI and go for the good stuff directly!
Title was: I argued with the father of open source for 2 years. Now the AI fight is the same — only bigger
Op-ed alt link: https://fortune.com/2026/07/03/open-source-ai-same-fight-as-...
FOSS is far from enough anymore.
_LEAN_ FOSS, including the SDK then the computer languages too.
All computer languages with an ultra-complex syntax are excluded de facto.
Then there is the stability in time.
developer/vendor lock-in on software, planned obsolescence, are much more common in FOSS nowdays.
I must have missed the press release when Two Sigma open-sourced their algorithmic trading models so that the rest of us could make the best stock investments for our retirements. /s
ABSLOUTELY NOT.
this is like saying "gov should invest in pyramid schem, because everyone is doing it". or btc. or web3 pictures of monkeys.
what i expect the gov to do is to add a 999% tax or tarif on top of GPUs bougth for AI, after the first 100mi that company spends on it each year.
No, I want govt to tax themmore. So far frontier AI companies produce negative value to near everyone (by sheer power cost increase it adds essentially tax to every other business) but themselves economy wise.
Yeah, wooho, new model found a bunch of bugs, now the bad guys can do it too so security expenses spiked! It's only good for shovel sellers.