- the insane frothing hype behind AI is showing me a new kind of market failure - where resources can be massively misallocated just because some small class of individuals THINK or HOPE it will result in massive returns. Even if it squeezes out every single other sector that happens to want to use SDRAM to do things OTHER than buffer memory before it's fed into a PCIE lane for a GPU.
- I'm really REALLY glad i decided to buy brand new gaming laptops for my wife and I just a couple months ago, after not having upgraded our gaming laptops for 7 and 9 years respectively. It seems like gamers are going to have this the worst - GPUs have been f'd for a long time due to crypto and AI, and now even DRAM isn't safe. Plus SSD prices are going up too. And unlike many other DRAM users where it's a business thing and they can to some degree just hike prices to cover - gamers are obviously not running businesses. It's just making the hobby more expensive.
It is a weird form of centralized planning. Except there's no election to get on to the central committee, it's like in the Soviet era where you had to run in the right circles and have sway in them.
There's too much group-think in the executive class. Too much forced adoption of AI, too much bandwagon hopping.
The return-to-office fad is similar, a bunch of executives following the mandates of their board, all because there's a few CEOs who were REALLY worked up about it and there was a decision that workers had it too easy. Watching the executive class sacrifice profits for power is pretty fascinating.
Edit: A good way to decentralize the power and have better decision making would be to have less centralized rewards in the capital markets. Right now are living through a new gilded age with a few barons running things, because we have made the rewards too extreme and too narrowly distributed. Most market economics assumes that there's somewhat equal decision making power amongst the econs. We are quickly trending away from that.
This was Lina Khan's big thing, and I'd argue that our current administration is largely a result of Silicon Valkey no longer being able to get exits in the form or mergers and IPOs.
Perhaps a better approach to anti-monopoly and anti-trust is possible, but I'm not sure anybody knows what that is. Khan was very well regarded and I don't know anybody who's better at it.
Another approach would be a wealth and income taxation strategy to ensure sigmoid income for the population. You can always make more, but with diminishing returns to self, and greater returns to the rest of society.
a better approach might be to farming out shares to stakeholders. that seems a lot more dynamic and self-correcting than periodic taxation battles after the fact
I think a better solution is exponential tax on a company size. I.e. once a company starts to earn above, say, 1 billion, it will be taxed by income by ever increasing amount. Or put it another way, use taxes to break the power law and winner takes effect all into a Gaussian distribution of company sizes.
This would keep DRAM prices permanently elevated. Memory fabricators either have billions of dollars in income each year or they can't keep going. There are no little Mom and Pop businesses that can do photolithography on leading process nodes.
Is that revenue, or profit? If revenue, it'll slam certain kinds of high-volume low-profit businesses, and if it's profit then the company will just arrange to have big compensation "expenses" for executives.
The latter would have to be backstopped by taxes on individual income.
how is this centralized planning? It’s a corporate decision making operating in a free market to optimize for what majority shareholders want (though the majority of shares are owned by few).
If you get paid for being rich in proportion to how rich you are -- because that's how assets work -- it turns into an exponential, runs away, and concentrates power until something breaks.
We have been living on the investment of previous centuries and decades in the West for close to 40 years now. Everything is broken but that didn't matter because everything that needed a functioning physical economy had moved to the East.
AI is the first industrial breakthrough in a century that needs the sort of infrastructure that previous industrial revolutions needed: namely a ton of raw power.
The bubble is laying bare just how terrible infrastructure is and how we've ignored trillions of maintenance to give a few thousand people tax breaks they don't really need.
This is part of how free markets self correct, misallocate resources and you run out of resources.
You can blame irrational exuberance, bubbles, or whatnot markets are ultimately individual choices times economic power. Ai, Crypto, housing, Dotcom etc going back through history all had excess because it’s not obvious when to join and when to stop.
It's not exactly a new type of failure. It's roughly equivalent to Riccardian rent, or pecuniary externalities for the general term. Though I suppose this is a speculative variant, which could be worse somehow.
I wonder, is there any way to avoid this kind of market failure? Even a planned economy could succumb to hype - promises that improved societal efficiency are just around the corner.
> Is there any way to avoid this kind of market failure?
There are potentially undesirable tradeoffs and a whole new game of cheats and corruption, but you could frustrate rapid, concentrated growth with things like an increasing tax on raised funds.
Right now, we basically let people and companies concentrate as much capital as they want, as rapidly as they want, with almost no friction, presumably because it helped us economically outcompete the adversary during the Cold War. Broadly, we're now afraid of having any kind of brake or dampener on investments and we are more afraid of inefficiency and corruption if the government were to intervene than we are of speculation or exploitation if it doesn't.
In democratically regulated capitalism, there are levers to pull that could slow down this kind of freight train before it were to get out of control, but the arguments against pulling them remain more thoroughly developed and more closely held than those in favor of them.
There is a way, and if anyone tells you we have to go full Hitler or Stalin to do it they are liars because last time we let inequality cook this hard FDR and the New Deal figured out how to thread the needle and proved it could be done.
Unfortunately, that doesn't seem to be the flavor of politics on tap at the moment.
Sam Altman cornering the DRAM market is a joke, of course, but if the punchline is that they were correct to invest this amount of resources in job destruction, it's going to get very serious very quickly and we have to start making better decisions in a hurry or this will get very, very ugly.
They're treating it as a "winner takes it all"-kind of business. And I'm not sure this is a reasonable bet.
The only way the massive planned investments make sense is if you think the winner can grab a very large piece of a huge pie. I've no idea how large the pie will be in the near future, but I'm even more skeptical that there will be a single winner.
What's odd about this is I believe there does exist a winner takes all technology. And that it's AR.
The more I dream about the possibilities of AR, the more I believe people are going to find it incredibly useful. It's just the hardware isn't nearly ready. Maybe I'm wrong but I believe these companies are making some of the largest strategic blunders possible at this point in time.
Markets are voting machines in the short term and weighing machines in the long term. We’re in the short term popularity phase of AI at the moment. The weighing will come along eventually.
For example: allocating the resources to only few industries deprives everyone else: small players, hobbyists, gamers, tinkerers from opportunities to play with their toys. And small players playing with random toys is a source of multiple innovations.
> resources can be massively misallocated just because some small class of individuals THINK or HOPE it will result in massive returns
That's basically what the rich usually do. They command disproportionate amount of resources and misallocate them freely on a whim, outside of any democratic scrutiny, squeezing incredible number of people and small buisness out of something.
Whether that's a strength of the system or the weakness, I'm sure some rearch will show.
It's a little ironic but to call this a market failure due to resource misalocation because prices are high when high prices is how misalocation is avoided.
I'm a little suspicious that "misalocation" just means it's too expensive for you. That's a feature, not a bug.
> Well, patience as a consumer might pay off in the next year or so when the music stops and hyperscalers are forced to dump their inventories.
Their inventories are not what consumers use.
Consumer DDR5 motherboards normally take UDIMMs. Server DDR5 motherboards normally take RDIMMs. They're mechanically incompatible, and the voltages are different. And the memory for GPUs is normally soldered directly to the board (and of the GDDRn family, instead of the DDRn or LPDDRn families used by most CPUs).
As for GPUs, they're also different. Most consumer GPUs are PCIe x16 cards with DP and HDMI ports; most hyperscaler GPUs are going to have more exotic form factors like OAM, and not have any DP or HDMI ports (since they have no need for graphics output).
So no, unfortunately hyperscalers dumping their inventories would be of little use to consumers. We'll have to wait for the factories to switch their production to consumer-targeted products.
Edit: even their NVMe drives are going to have different form factors like E1.S and different connectors like U.2, making them hard for normal consumers to use.
The problem is that it is not entirely clear that the hyperscalers are buying DDR5, instead it seems that supplies are being diverted so that more HBM/GDDR wafers can be produced.
HBM/GDDR is not necessarily as useful to the average person as DDR4/DDR5
Its a bit of a shame these AI GPUs don't actually have displayport/hdmi output ports because they would make for nice cheap and powerful gaming GPUs with a lot of VRAM, they would potentially be really good graphics cards.
Will just have to settle for insanely cheap second hand DDR5 and NVMe drives I guess.
If we're going to see retailers price-gouging on DDR5, maybe people will be willing to buy slightly older gear with DDR4 (and corresponding motherboard and CPU).
Especially for systems for which the workloads are actually bound by GPU compute, network, or storage.
2 months ago there were a load of second gen xeon scalable servers on offer. Now every one of them has had the ram stripped out and its just the chassis on offer.
I'm still on DDR4 but I hope this price gouging will be over by the time I need to finally upgrade :( I have a Ryzen so I did upgrade to the latest AM4 generation.
I just snagged an Asrock Rack mobo (X570), 5900x and 128gb ecc ddr4 for $680. Felt like a steal with how memory prices are going these days, ECC to boot.
Haven't these memory companies been caught price fixing multiple times over the years? Just how sure are we the AI bubble is the entire reason for these absurd prices?
All I can say is,
- the insane frothing hype behind AI is showing me a new kind of market failure - where resources can be massively misallocated just because some small class of individuals THINK or HOPE it will result in massive returns. Even if it squeezes out every single other sector that happens to want to use SDRAM to do things OTHER than buffer memory before it's fed into a PCIE lane for a GPU.
- I'm really REALLY glad i decided to buy brand new gaming laptops for my wife and I just a couple months ago, after not having upgraded our gaming laptops for 7 and 9 years respectively. It seems like gamers are going to have this the worst - GPUs have been f'd for a long time due to crypto and AI, and now even DRAM isn't safe. Plus SSD prices are going up too. And unlike many other DRAM users where it's a business thing and they can to some degree just hike prices to cover - gamers are obviously not running businesses. It's just making the hobby more expensive.
It is a weird form of centralized planning. Except there's no election to get on to the central committee, it's like in the Soviet era where you had to run in the right circles and have sway in them.
There's too much group-think in the executive class. Too much forced adoption of AI, too much bandwagon hopping.
The return-to-office fad is similar, a bunch of executives following the mandates of their board, all because there's a few CEOs who were REALLY worked up about it and there was a decision that workers had it too easy. Watching the executive class sacrifice profits for power is pretty fascinating.
Edit: A good way to decentralize the power and have better decision making would be to have less centralized rewards in the capital markets. Right now are living through a new gilded age with a few barons running things, because we have made the rewards too extreme and too narrowly distributed. Most market economics assumes that there's somewhat equal decision making power amongst the econs. We are quickly trending away from that.
We need better antitrust and anti-monopoly enforcement. Break up the biggest companies, and then they'll have to actually participate in markets.
This was Lina Khan's big thing, and I'd argue that our current administration is largely a result of Silicon Valkey no longer being able to get exits in the form or mergers and IPOs.
Perhaps a better approach to anti-monopoly and anti-trust is possible, but I'm not sure anybody knows what that is. Khan was very well regarded and I don't know anybody who's better at it.
Another approach would be a wealth and income taxation strategy to ensure sigmoid income for the population. You can always make more, but with diminishing returns to self, and greater returns to the rest of society.
a better approach might be to farming out shares to stakeholders. that seems a lot more dynamic and self-correcting than periodic taxation battles after the fact
I think a better solution is exponential tax on a company size. I.e. once a company starts to earn above, say, 1 billion, it will be taxed by income by ever increasing amount. Or put it another way, use taxes to break the power law and winner takes effect all into a Gaussian distribution of company sizes.
This would keep DRAM prices permanently elevated. Memory fabricators either have billions of dollars in income each year or they can't keep going. There are no little Mom and Pop businesses that can do photolithography on leading process nodes.
Is that revenue, or profit? If revenue, it'll slam certain kinds of high-volume low-profit businesses, and if it's profit then the company will just arrange to have big compensation "expenses" for executives.
The latter would have to be backstopped by taxes on individual income.
Ah yes, the same tax mentality that is working great for EU innovation.
how is this centralized planning? It’s a corporate decision making operating in a free market to optimize for what majority shareholders want (though the majority of shares are owned by few).
If you get paid for being rich in proportion to how rich you are -- because that's how assets work -- it turns into an exponential, runs away, and concentrates power until something breaks.
Every corporation is a (not so) little pocket of centrally planned economy.
The only saving grace is that it can die and others will scoop up released resources.
When country level planned economy dies, people die and resources get destroyed.
I disagree.
We have been living on the investment of previous centuries and decades in the West for close to 40 years now. Everything is broken but that didn't matter because everything that needed a functioning physical economy had moved to the East.
AI is the first industrial breakthrough in a century that needs the sort of infrastructure that previous industrial revolutions needed: namely a ton of raw power.
The bubble is laying bare just how terrible infrastructure is and how we've ignored trillions of maintenance to give a few thousand people tax breaks they don't really need.
Why not follow the time-honoured approach and put the data centres in low-income countries?
Because you only do that once the tech has been comodatized and you have wrung all the benefit for your country that you can.
The British didn't industrialise Indian for a reason.
I assume they don't have good enough power infrastructure.
>AI is the first industrial breakthrough in a century
Is it?
This is part of how free markets self correct, misallocate resources and you run out of resources.
You can blame irrational exuberance, bubbles, or whatnot markets are ultimately individual choices times economic power. Ai, Crypto, housing, Dotcom etc going back through history all had excess because it’s not obvious when to join and when to stop.
Usually companies run out of resources before they screw up global prices in massive markets.
If it was a couple billion dollars of memory purchasing nobody would care.
> Usually companies run out of resources before they screw up global prices in massive markets.
It happens more often than you might expect.
The Onion Futures Act and what led to it is always a fun read: https://en.wikipedia.org/wiki/Onion_Futures_Act
It's not exactly a new type of failure. It's roughly equivalent to Riccardian rent, or pecuniary externalities for the general term. Though I suppose this is a speculative variant, which could be worse somehow.
I wonder, is there any way to avoid this kind of market failure? Even a planned economy could succumb to hype - promises that improved societal efficiency are just around the corner.
> Is there any way to avoid this kind of market failure?
There are potentially undesirable tradeoffs and a whole new game of cheats and corruption, but you could frustrate rapid, concentrated growth with things like an increasing tax on raised funds.
Right now, we basically let people and companies concentrate as much capital as they want, as rapidly as they want, with almost no friction, presumably because it helped us economically outcompete the adversary during the Cold War. Broadly, we're now afraid of having any kind of brake or dampener on investments and we are more afraid of inefficiency and corruption if the government were to intervene than we are of speculation or exploitation if it doesn't.
In democratically regulated capitalism, there are levers to pull that could slow down this kind of freight train before it were to get out of control, but the arguments against pulling them remain more thoroughly developed and more closely held than those in favor of them.
> there are levers to pull that could slow down this kind of freight train before it were to get out of control
Care to share some keywords here?
There is a way, and if anyone tells you we have to go full Hitler or Stalin to do it they are liars because last time we let inequality cook this hard FDR and the New Deal figured out how to thread the needle and proved it could be done.
Unfortunately, that doesn't seem to be the flavor of politics on tap at the moment.
Sam Altman cornering the DRAM market is a joke, of course, but if the punchline is that they were correct to invest this amount of resources in job destruction, it's going to get very serious very quickly and we have to start making better decisions in a hurry or this will get very, very ugly.
They're treating it as a "winner takes it all"-kind of business. And I'm not sure this is a reasonable bet.
The only way the massive planned investments make sense is if you think the winner can grab a very large piece of a huge pie. I've no idea how large the pie will be in the near future, but I'm even more skeptical that there will be a single winner.
What's odd about this is I believe there does exist a winner takes all technology. And that it's AR.
The more I dream about the possibilities of AR, the more I believe people are going to find it incredibly useful. It's just the hardware isn't nearly ready. Maybe I'm wrong but I believe these companies are making some of the largest strategic blunders possible at this point in time.
Why would AR be particularly likely to have a single winnner?
Markets are voting machines in the short term and weighing machines in the long term. We’re in the short term popularity phase of AI at the moment. The weighing will come along eventually.
Going to be awesome tho when OpenAI et al fail because the market is going to be flooded with cheap parts.
why do you think allocating hardware to gamers is proper usage?
maybe AI cures cancer, or at least writes some code
Or you could look at reality where it generates fake social media posts s lot and we could all ask, why is this valuable?
For example: allocating the resources to only few industries deprives everyone else: small players, hobbyists, gamers, tinkerers from opportunities to play with their toys. And small players playing with random toys is a source of multiple innovations.
What do you think happens when the majority of consumers are priced not only out of bread, but also circuses?
> resources can be massively misallocated just because some small class of individuals THINK or HOPE it will result in massive returns
That's basically what the rich usually do. They command disproportionate amount of resources and misallocate them freely on a whim, outside of any democratic scrutiny, squeezing incredible number of people and small buisness out of something.
Whether that's a strength of the system or the weakness, I'm sure some rearch will show.
> where resources can be massively misallocated
It's a little ironic but to call this a market failure due to resource misalocation because prices are high when high prices is how misalocation is avoided.
I'm a little suspicious that "misalocation" just means it's too expensive for you. That's a feature, not a bug.
how is that market failure??? this is literally market of supply and demand at its core
yeah but the demand is based on empty promises
New type? Lol.
It’s a classic ‘tulip bubble’.
I'm so mad about this, I need DDR5 for a new mini-PC I bought and prices have literally gone up by 2.5x..
128GB used to be 400$ in June, and now it's over $1,000 for the same 2x64GB set..
I have no idea if/when prices will come back down but it sucks.
I just looked at the invoice for my current PC parts that I bought in April 2016: I paid 177 EUR (~203 USD) for 32GB (DDR4-2800).
It's kinda sad when you grow up in a period of rapid hardware development and now see 10 years going by with RAM $/GB prices staying roughly the same.
Aside, $203 USD back then would be about $276 USD after inflation. Not a primary effect, but contributory.
Ordered some servers 6 months ago ~12k USD per unit.
Same order, same bill of materials, 17.5K USD per unit today.
That is roughly a 5.5k increase for 768GB of DDR5 ECC memory and the 4 2tb nvme ssds.
Doubled in the last 4 months https://www.youtube.com/watch?v=o5Zc-FsUDCM
Upgraded by adding 64GB.. last Friday I sold the 32 GB I took out for what I paid for the 64 GB in July... insane
Time to start scouring used-PC sales to reclaim the RAM and sell it for a profit?
Damn I bought a whole computer with 128GB RAM & 16-core Ryzen CPU for £325 a few months ago.
> I have no idea if/when prices will come back down but it sucks.
Years, or when the AI bubble pops, whatever comes first.
Similar situation with QLC flash and HDDs btw.
I'm still on DDR3 :)
Well, patience as a consumer might pay off in the next year or so when the music stops and hyperscalers are forced to dump their inventories.
There still isn't a clear path to profitability for any of these AI products and the capital expenditure has been enormous.
> Well, patience as a consumer might pay off in the next year or so when the music stops and hyperscalers are forced to dump their inventories.
Their inventories are not what consumers use.
Consumer DDR5 motherboards normally take UDIMMs. Server DDR5 motherboards normally take RDIMMs. They're mechanically incompatible, and the voltages are different. And the memory for GPUs is normally soldered directly to the board (and of the GDDRn family, instead of the DDRn or LPDDRn families used by most CPUs).
As for GPUs, they're also different. Most consumer GPUs are PCIe x16 cards with DP and HDMI ports; most hyperscaler GPUs are going to have more exotic form factors like OAM, and not have any DP or HDMI ports (since they have no need for graphics output).
So no, unfortunately hyperscalers dumping their inventories would be of little use to consumers. We'll have to wait for the factories to switch their production to consumer-targeted products.
Edit: even their NVMe drives are going to have different form factors like E1.S and different connectors like U.2, making them hard for normal consumers to use.
The problem is that it is not entirely clear that the hyperscalers are buying DDR5, instead it seems that supplies are being diverted so that more HBM/GDDR wafers can be produced.
HBM/GDDR is not necessarily as useful to the average person as DDR4/DDR5
Its a bit of a shame these AI GPUs don't actually have displayport/hdmi output ports because they would make for nice cheap and powerful gaming GPUs with a lot of VRAM, they would potentially be really good graphics cards.
Will just have to settle for insanely cheap second hand DDR5 and NVMe drives I guess.
I wouldn't mind my own offline Gemini or ChatGPT 5. But even if the hardware and model were free, I don't know how I'd afford the electricity.
If we're going to see retailers price-gouging on DDR5, maybe people will be willing to buy slightly older gear with DDR4 (and corresponding motherboard and CPU).
Especially for systems for which the workloads are actually bound by GPU compute, network, or storage.
DDR4 will be just as expensive because it's made in the same fabs.
I'm thinking second-hand and new-old-stock, with less demand for it.
Nope, second hand is already evaporating.
2 months ago there were a load of second gen xeon scalable servers on offer. Now every one of them has had the ram stripped out and its just the chassis on offer.
Fabs are not wasting their time on DDR4 now.
I'm still on DDR4 but I hope this price gouging will be over by the time I need to finally upgrade :( I have a Ryzen so I did upgrade to the latest AM4 generation.
I just snagged an Asrock Rack mobo (X570), 5900x and 128gb ecc ddr4 for $680. Felt like a steal with how memory prices are going these days, ECC to boot.
I'm ready to fight for my 401k to defend DRAM cartels
Does anyone know if the increase in prices in high-end RAM will affect lower end RAM used in embedded devices, e.g. LPDDR4?
Haven't these memory companies been caught price fixing multiple times over the years? Just how sure are we the AI bubble is the entire reason for these absurd prices?