The article claims that AI services are currently over-utilised. Well isn't that because customers are being undercharged for services? A car when in neutral will rev up easily if the accelerator pedal is pushed even very slightly, because there's no load on the engine. But in gear the same engine will rev up much less when the accelerator is pushed the same amount. Will there be the same overutilisation occurring if users have to financially support the infrastructure, either through subscriptions or intrusive advertising?
I doubt it.
And what if the technology to locally run these systems without reliance on the cloud becomes commonplace, as it now is with open source models? The expensive part is in the training of these models more than the inference.
> Will there be the same overutilisation occurring if users have to financially support the infrastructure, either through subscriptions or intrusive advertising?
> I doubt it.
I agree. Right now a lot of AI tools are underpriced to get customers hooked, then they'll jack up the prices later. The flaw is that AI does not have the ubiquitous utility internet access has, and a lot of people are not happy with the performance per dollar TODAY, much less when prices rise 80%. We already see companies like Google raising prices stating it's for "AI" and we customers can't opt out of AI and not pay the fee.
At my company we've already decided to leave Google Workspace in the spring. GW is a terrible product with no advanced features, garbage admin tools, uncompetitive pricing, and now AI shoved in everywhere and no way to granularly opt out of a lot of it. Want spell check? Guess what, you need to leave Gemini enabled! Shove off, Google.
> The article claims that AI services are currently over-utilised. Well isn't that because customers are being undercharged for services?
Absolutely, not only are most AI services free but even the paid portion is coming from executives mandating that their employees use AI services. It's a heavily distorted market.
And a majority of those workers do not reveal their AI usage, so they either take credit for the faster work or use the extra time for other activities, which further confounds the impact of AI.
This is also distorting the market, but in other ways.
Yes, over-utilization is a natural response to being undercharged. And being undercharged is a natural result when investors are throwing money at you. During bubbles, Silicon Valley often goes to "lose money, make it up with scale". With the vague idea that after you get to scale, THEN you can figure out how to make money. And fairly consistently, their idea for how to make money is "sell ads".
Past successes like Google encourage hope in this strategy. Sure, it mostly doesn't work. Most of of everything that VCs do doesn't work. Returns follow a power law, and a handful of successes in the tail drive the whole portfolio.
The key problem here doesn't lie in the fact that this strategy is being pursued. The key problem is that it is rare for first mover advantages to last with new technologies. That's why Netscape and Yahoo! aren't among the FAANGs today. The long-term wins go to whoever successfully create a sufficient moat for themselves to protect lasting excess returns. And the capabilities of each generation of AI leapfrogs the last so well that nobody has figured out how to create such a moat.
Today, 3 years after launching the first LLM chatbot, OpenAI is nowhere near as dominant as Netscape was in late 1997, 3 years after launching Netscape Navigator. I see no reason to expect that 30 years from now OpenAI will be any more dominant than Netscape is today.
Right now companies are pouring money into their candidates to win the AI race. But if the history of browsers repeats itself, the company that wins in the long-term would launch in about a year from now, focused on applications on top of AI. And its entrant into the AI wars wouldn't get launched until a decade after that! (Yes, that is the right timeline for the launch of Google, and Google's launch of Chrome.)
Investing in silicon valley is like buying a positive EV lottery ticket. An awful lot of people are going to be reminded the hard way that it is wiser to buy a lot of lottery tickets, than it is to sink a fortune into a single big one.
> Today, 3 years after launching the first LLM chatbot, OpenAI is nowhere near as dominant as Netscape was in late 1997.
Incorrect. There were about 150 millions Internet users in 1998, or 3.5% of total population. The number grew 10 times by 2008 [0]. Netwcape had about 50% of the browser market at the time [1]. In other words, Netscape dominated a small base and couldn’t keep it up.
ChatGPT has about 800 millions monthly users, or already 10% of total current population. Granted, not exclusively. ChatGPT is already a household name. Outside of early internet adopters, very few people knew who Netscape or what Navigator was.
Furthermore my point that the early market leaders are seldom the lasting winners is something that you can see across a large number of past financial bubbles through history. You'll find the same thing in, for example, trains, automobiles, planes, and semiconductors. The planes example is particularly interesting. Airline companies not only don't have a good competitive moat, but the mechanics of chapter 11 mean that they keep driving each other bankrupt. It is a successful industry, and yet it has destroyed tons of investment capital!
Despite your quibbles over the early browser market, my broader point stands. It's early days. AI companies do not have a competitive moat. And it is way to premature to reliably pick a winner.
We're talking miraculous level of improvement for a SOA LLM to run on a phone without crushing battery life this decade.
People are missing the forest for the trees here. Being the go to consumer Gen AI is a trillion+ dollar business. How many 10s of billions you waste on building unnecessary data centers is a rounding error. The important number is your odds of becoming that default provider in the minds of consumers.
In fact it's apparently $5.2 trillion by 2030 [0] (out of $6.7T total data center spend; meaning all of "traditional IT needs" are less than a quarter of the total). That's the total if you add up all of the firms chasing this opportunity.
I do wonder, if you (and the commenter you replied to) think this is a good thing, will you be OK with a data center springing up in your neighbourhood, driving up water or power prices, emitting CO2? Then if SOTA LLMs become efficient enough to run on a smartphone will you be OK with a data center bailout coming from your tax dollars?
> The important number is your odds of becoming that default provider in the minds of consumers.
I haven't seen any evidence that any Gen AI provider will be able to build a moat that allows for this.
Some are better than others at certain things over certain time periods, but they are all relatively interchangeable for most practical uses and the small differences are becoming less pronounced, not more.
I use LLMs fairly frequently now and I just bounce around between them to stay within their free tiers. Short of some actual large breakthrough I never need to commit to one, and I can take advantage of their own massive spends and wait it out a couple of years until I'm running a local model self-hosted with a cloudflare tunnel if I need to access it on my phone.
And yes, most people won't do that, but there will be a lot of opportunity for cheap providers to offer that as a service with some data center spend, but nowhere near the massive amounts OpenAI, Google, Meta, et al are burning now.
I used ChatGPT for every day stuff, but in my experience their responses got worse and I had to wait much longer to get them. I switched to Gemini and their answers were better and were much faster.
I don’t have any loyalty to Gemini though. If it gets slow or another provider gives better answers, I’ll change. They all have the same UI and they all work the same (from a user’s perspective).
There is no moat for consumer genAI. And did I mention I’m not paying for any of it?
It’s like quick commerce, sure it’s easy to get users by offering them something expensive off of VC money. The second they raise prices or offer degraded experience to make the service profitable, the users will leave for another alternative.
My hot (maybe just warm these days) take is, the problem with voice assistants on phones is they have to be able to have reasonable responses to a long tail or users will learn not to use them, since the use cases aren’t discoverable and the primarily value is talking to it like a person.
So voice assistants backed by very large LLMs over the network are going to win even if we solve the (substantial) battery usage issue.
Why even bother with the text generation then? You could just make a phone call to an LLM with a TTS frontend. Like with directory enquiries back in the day. Which can be set up as easily as a BBS if you have a home server rack like Jeff Geerling makes youtube videos about.
Not many people buy Windows, they buy laptops that happen to have Windows installed. IMO this is a worthwhile distinction because most people don’t really care about operating systems anyway, and would happily (I suspect, at least) use an Open Source one if it came installed and configured on a device that they got in a store.
Installing an OS is seen as a hard/technical task still. Installing a local program, not so much. I suspect people install LLM programs from app stores without knowing if they are calling out to the internet or running locally.
Besides the fact that this article is obviously AI generated (and not even well, why is there mismatches in british/american english? I can only assume that the few parts in british english are the human author's writing or edits), yes "overutilization" is not a real thing. There is a level of utilization at every price point. If something is "overutilizated" that actually means it's just being offered at a low price, which is good for consumers. It's a nice scare word though and there's endless appetite at the moment for ai-doomer articles.
Sorry but it's highly suspect to be spelling the same word multiple different ways across paragraphs. You switch between using centre/center or utilization/utilisation? It is a very weird mistake to make for a human.
Will the OpenRouter marketplace of M clouds X N models die if the investor money stops? I believe its a free and profitable service, offered completely pay as you go.
I don't. This is simply the "drug dealer" model where the first hit is free. They know that once people are addicted, they will keep coming back.
The question of course is, will they keep coming back? I think they very much will. There are indications that GenAI adoption is already increasing labor producitivity labor improvements at a national scale, which is quite astounding for a technology just 3 years old: https://news.ycombinator.com/item?id=46061369
Imagine a magic box where you put in some money and get more productivity back. There is no chance Capitalism (with a capital "C") is going to let such a powerful growth machine wither on the vine. This mad AI rush is all about that.
OpenAI has 800,000,000 weekly users but only 20,000,000 are paying while 780,000,000 are free riding. Should they by accident under provision then they could simply remove the freebee and raise the prices for the paying clients. But that is not what they want.
IMHO the investors are betting on a winner-takes-it-all market and that some magic AGI will be coming out of OpenAI or Anthropic.
The questions are:
How much money can they make by integrating advertising and/or selling user profiles?
What is the model competition going to be?
What is the future AI hardware going to be - TPUs, ASICs?
Will more people have powerful laptops/desktops to run a mid-sized models locally and be happy with it?
The internet didn't stop after the dotcom crash and the AI wont stop either should there be a market correction.
>OpenAI has 800,000,000 weekly users but only 20,000,000 are paying while 780,000,000 are free riding.
By itself, this doesn't tell us much.
The more interesting metric would be token use comparison across free users, paid users, API use, and Azure/Bedrock.
I'm not sure if these numbers are available anywhere. It's very possible B2B use could be a much bigger market than direct B2C (and the free users are currently providing value in terms of training data).
I just thought about it, and honestly, from my surroundings of people aged 12-70s, across multiple continents, I can’t think of anyone who isn’t using some sort of LLM once a week.
The thing that makes AI investment hard to reason about for individuals is that our expectations are mostly driven by a single person’s usage, just like many of the numbers reported in the article.
But the AI providers are betting, correctly in my opinion, that many companies will find uses for LLM’s which are in the trillions of tokens per day.
Think less of “a bunch of people want to get recipe ideas.”
Think more of “a pharma lab wants to explore all possible interactions for a particular drug” or “an airline wants its front-line customer service fully managed by LLM.”
It’s unusual that individuals and industry get access to basically similar tools at the same time, but we should think of tools like ChatGPT and similar as “foot in the door” products which create appetite and room to explore exponentially larger token use in industry.
> Think more of “a pharma lab wants to explore all possible interactions for a particular drug”
Pharma does not trust OpenAI with their data, and they don't work on tokens for any of the protein or chemical modeling.
There will undoubtedly be tons of deep nets used by pharma, with many $1-10k buys replacing more expensive physical assays, but it won't be through OpenAI, and it won't be as big as a consumer business.
Of course there may be other new markets opened up but current pharma is not big enough to move the needle in a major way for a company with an OpenAI valuation.
My claim is that there will exist some company which pharma is willing to trust for AI research…they presumably trust Microsoft with their email today.
But my bigger claim is that ~half the Fortune 500 will be able to profitably deploy AI with spends in the tens or hundreds of millions per year quite soon. Not that pharma itself is a major contributor to that effect.
But for 'AI' to be a winner-take-all market, it seems that the winner would have to be using customer data to improve the 'AI'. Not only do you have to believe that one of these (relatively) under-capitalized upstarts can corral the money, but also that they can convince (enterprise) customers to 'fork over' their proprietary data to only one provider, and also that the provider can then charge a monopoly rent.
Those all seem possible, but I wouldn't assign greater than a 50% probability to any of them, and the valuations seem to imply near-certainty.
When I'm building out a new feature, I can churn through millions of tokens in Claude code. And that's just me... Now think about Claude code but integrated with Excel or datadog, or whatever app could be improved through LLM integration. Think about the millions of office workers, beyond just software engineers, who will be running hundreds of thousands or millions of tokens per day through these tools.
Let's estimate 200 million office workers globally as TAM running an average of 250k tokens. That's 50 trillion tokens DAILY. Not sure what model provider profit per token is, but let's say it's .001 cents.
I find it absurd to pay for tokens I cant control, predict or even check in any reasonable way. It is literally amounts to "pay whatever random money the company asks you to pay" kind of contract.
> “an airline wants its front-line customer service fully managed by LLM.”
This has been experimented on before by many companies over the recent years, most notably Klarna which was among the earliest guinea pigs for it and had to later on backtrack on this "novel" idea when the results came out.
But if I'm a pharma lab, I don't want to rely on a statistical engine that makes mistakes to answer those questions, I want to query a database that is deterministic.
Sam Altman says that he thinks scientific research is a huge opportunity for AI to contribute as it more fully develops and I think he’s right.
Since I’m not a scientific researcher, I have no idea if he’s just blowing smoke but I think it’s reasonable to think of a purpose-built system which has an LLM component being used by a team to do something useful.
This. LLM is NOT the tool for a pharma lab - properly trained ML is the right tool. Heck, English is probably not even the right LANGUAGE to use for discussing chemical interactions.
- Lead Time vs. TCO vs. IRS Asset Deprecation: The moment you get it fully built, it's already obsolete. Thus from a CapEx point of view, if you can lease your compute (including GPU) and optimize the rest of the inputs for similar then your CapEx overall is much lower and tied to the real estate - not the technology. The rest is cost of doing business and deductible in and of itself.
- The "X" factor: Someone mentioned TPU/ASIC but then there is the DeepSeek factor - what if we figure out a better way of doing the work that can shortcut the workflow?
- AGI partnerships: Right now, you see a lot of Mega X giving billions to Mega Y because all of them are trying to get their version of Linux or Apache or whatever at parity with the rest. Once AGI is settled and confirmed, then most all of these partnerships will be severed because it then becomes which company is going to get their AI model into that high prestige Montessori school and into the right ivy league schools - like any other rich parent would for their "bot" offspring.
So what will it look like when it crashes? A bunch of bland empty "warehouses" with mobile PDU's once filling all their parking lot space gone. Whatever "paradise" that was there may come back... once you bulldoze all that concrete and steel. The money will do something else like a Don McLean song.
You're not quite thinking things through there man. Once the elites who built these follies have gone, the mob will go shopping for building materials. I wouldn't be surprised even if people end up living in these datacentres once they become derelict. They have AC after all.
Welp Gemini got me. Using G3 to improve what I write, generate specific images, and use NotebookLM to dive into some research materials. Tried to do a bit each day with my free credits, but hit the limit too often. G2.5 was not nearly as useful. So I upgraded my baselevel Google workspace plan. Recently spoke to someone who is also using G3 a lot with good results. YMMV re: G3, but Google hooked me, and now I pay more. However I think it is worth it for what I do. G3 is my helpful, nerdy work mate. I never plan to use agenic AI. Not using ChatGPT much if any at all anymore. Sorry Sam.
I am still a little skeptical about utilisation rates. If demand is so extreme, wouldn't we see rental prices for H100/A100 prices go up or maintain? Wouldn't the cost for such a gpu still be high (you can get em 3k used).
On "runpod community cloud" renting a 5090 costs $0.69/hour [1] and it consumes about $0.10/hour electricity, if running at full power and paying $0.20/kWh.
On Amazon, buying a 5090 costs $3000 [2]
That's a payback time of 212 days. And Runpod is one of the cheaper cloud providers; for the GPUs I compared, EC2 was twice the price for an on-demand instance.
Yes or no conclusions aside (and despite its title, the article deserves better than that), the key point is I think this one: “But unlike telecoms, that overcapacity would likely get absorbed.”
Stylistically, this smells like it was copy and pasted from straight out Deep Research. Substantively, I could use additional emphasis on the mismatch between expectations and reality with regards to telco debt-repayment schedule.
I'm curious about the telecom collapse. How was it not in the pipeline that much better use of the fibres was around the corner? Surely they would have known that people were looking into it with some promise?
There's an enormous amount of unused, abandoned fiber. All sorts of fiber was run to last mile locations, across most cities in the US, and a shocking amount effectively got abandoned in the frenzy of mergers and acquisitions. 2 trillion seems like a reasonable estimate.
Giant telecoms bought big regional telecoms which came about from local telecoms merging and acquiring other local telecoms. A whole bunch of them were construction companies that rode the wave, put in resources to run dark fiber all over the place. Local energy companies and the like sometimes participated.
There were no standard ways of documenting runs, and it was beneficial to keep things relatively secret, since if you could provide fiber capabilities in a key region, but your competition was rolling out DSL and investing lots of money, you could pounce and make them waste resources, and so on. This led to enormous waste and fraud, and we're now on the outer edge of usability for most of the fiber that was laid - 29-30 years after it was run, most of it will never be used, or ever have been used.
I so desperately wish it weren't abandoned. I hate that it's almost 2026 and I still can't get a fiber connection to my apartment in a dense part of San Diego. I've moved several times throughout the years and it has never been an option despite the fact that it always seems to be "in the neighborhood".
That has nothing to do with fiber, it’s all about politics and a regulatory environment where nobody is incented to act. Basically, the states can’t fully regulate internet and the Federal government only wants to fund buildouts on a pork barrel basis. Most recently rural.
At the local level, there is generally a cable provider with existing rights of way. To get a fiber provider, there’s 4 possible outcomes: universal service with subsidy (funded by direct subsidy), cherry-picked service (they install where convenient), universal service (capitalized by the telco) and “fuck you”, where they refuse to operate. (ie. Verizon in urban areas)
The private capitalized card was played out by cable operators in the 80s (they were innovators then, and AT&T was just broken up and in chaos). They have franchise agreements whose exclusivity was used as loan collateral.
Forget about San Diego, there are neighborhoods in Manhattan with the highest population density in the country where Verizon claims it’s unprofitable to operate.
I served on a city commission where the mayor and county were very interested in getting our city wired, especially as legacy telco services are on the way out and cable costs are escalating and will accelerate as the merger agreement that formed Spectrum expires. The idea was to capitalize last mile with public funds and create an authority that operated both the urban network and the rural broadband in the county funded by the Federal legislation. With the capital raised with grants and low cost bonding (public authority bonds are cheap and backed by revenue and other assets), it would raise a moderate amount of income in <10 years.
We had the ability to get the financing in place, but we would have needed legislation passed to get access to rights of way. Utilities have lots of ancient rights and laws that make disruption difficult. The politicians behind it turned over before that could be changed.
The worst part is it'd probably cost less than $100 of fiber and labor to splice something into your building, maybe $200-400 of gear to light it up, and you'd have a 10 gbps pipe back to some colo. It's more economical to run new fiber in most places these days, even if the local ISP knows exactly where all the old abandoned legacy lines are run, because of subsidization and basically scamming. The big companies like Lumen keep their knowledge regionally compartmented, legally shielded, and deliberately obfuscated, because if it became known that existing fiber was already run to a place they claim they can't serve, they can't get access to yet more funding for their eternal "service for the underserved" government money grift.
I stumbled on old maps that showed a complete coverage of fiber in my municipality, paperwork from a company that was acquired, and which in turn merged, then was bought out by one of the big 5 ISPs. When local officials requested information regarding existing fiber, this ISP refused and said any such information was proprietary. They later bid on and won contracts to run new fiber (parallel to existing lines which they owned, which still had more than a decade of service life left in them at that point).
I estimated that only around 10-15% of the funding went toward actual labor and materials, the remainder was pure profit. The local government considered it a major victory, money well spent.
For infrastructure, central planning and state-run systems make a lot of sense - this after all is how the USA's interstate highway system was built. The important caveat is that system components and necessary tools should be provided by the competitive private sector through transparent bidding processes - eg, you don't have state-run factories for making switches, fiber cable, road graders, steel rebar, etc. There are all kinds of debatable issues, eg should system maintenance be contracted out to specialized providers, or kept in-house, etc.
The GDP 1995-2000 (inclusive) was about $52T. So that assertion would mean that about %3.8 of the US' economic activity was laying fiber. That seems like a lot, but in my ignorance doesn't sound totally impossible.
The 2001 telecoms crash drove benefits for companies that came later in the availability of inexpensive dark fiber after the bubble popped. WorldCom, ICG, Williams sold off to Verizon, Level 3, Teleglobe, and others. That in turn helped future Internet companies gain access to plentiful and inexpensive bandwidth.
Cable telephony companies such as Cablevision Systems, Comcast, Cox Communications, and Time Warner, used the existing coaxial connections into the home to launch voice services.
Rail and fiber deprecates on multiple decade timescales. AI data centers close to tulips. Even assuming we manage to make data center stretch to 10 years, these assets won't be around long enough to support ecosystem of new companies if the economics stops making sense. Ultimately the only durable thing is any power infra that gets built, vs rail and fiber where inheritance isn't just rail networks or fiber but like 1000s of kilometers of earthwork projects to build out massive physical networks.
Data centers last decades. Many or the current AI hosting vendors such as Coreweave have crypto origin. Their data centers were built out in 2010s, early 2020s.
Many of legacy systems still running today are IBM or Solaris servers at 20, 30 year old. No reason to believe GPU won’t still be in use in some capacity (e.g. interference) a decade from now.
This is indeed true, but doesn't fiber have a far longer lifetime than GPU heavy data centers? The major cost center is the hardware, which has a fairly short shelf life.
Well you still get the establishment of 1) large industrial buildings 2) water/electricity distribution 3) trained employees who know how to manage a data center
Even if all of the GPUs inside burn out and you want to put something else entirely inside of the building, that's all still ready to go.
Although there is the possibility they all become dilapidated buildings, like abandoned factories
The building and electrical infrastructure are far cheaper than the hardware. So much so that the electricity is a small cost of the data center build out, but a major cost for the grid.
Of the most valuable part is quickly depreciating and goes unused within the first few years, it won't have a chance for long term value like fiber. If data centers become, I don't know, battery grid storage, it will be very very expensive grid storage.
Which is to say that while there was an early salivation for fiber that was eventually useful, overallocation of capital to GPUs goes to pure waste.
Don’t think looking at power consumption of b200s is a good measure of anything. Could well be an indication of higher density rather than hitting limits and cranking voltage to compensate
Yes, one of NVidia's selling points for the b200 is that performance per watt is better than before. High power consumption without controlling for performance means nothing.
Is there a way in which this is good for a segment of consumers? When the current gen of GPUs are too old, will the market be flooded with cheap GPUs that benefit researchers and hobbyists who therwis would not afford them?
GPUs age surprisingly gracefully. If a GPU isn't cutting edge, you just tie two or more of them together for a bit more power consumption to get more or less the same result as the next generation GPU.
if there's ever a glut in GPUs that formula might change but it sure hasn't happened yet. Also, people deeply underestimate how long it would take a competing technology to displace them. It took GPUs nearly a decade and the fortunate occurrence of the AI boom to displace CPUs in the first place despite bountiful evidence in HPC that they were already a big deal.
source: Nvidia Shields still shipping with Maxwell GPUs from 2014 and many university labs still running on Pascal GPUs from 2016. But stupid facts don't matter, this one hit you wrong in the feels, amIRight?
* The GPUs in use in data centers typically aren’t built for consumer workloads, power systems, or enclosures.
* Data Centers often shred their hardware for security purposes, to ensure any residual data is definitively destroyed
* Tax incentives and corporate structures make it cheaper/more profitable to write-off the kit entirely via disposal than attempt to sell it after the fact or run it at a discount to recoup some costs
* The Hyperscalers will have use for the kit inside even if AI goes bust, especially the CPUs, memory, and storage for added capacity
That’s my read, anyway. They learned a lot from the telecoms crash and adjusted business models accordingly to protect themselves in the event of a bubble crash.
We will not benefit from this failure, but they will benefit regardless of its success.
Why not? It’s 40A at 240V, or 25% of the continuous load rating of a 200A 240V single-phase service.
If someone can afford an 8 GPU server, they should be able to afford some #6 wire, a 50A 2P breaker, and a 50A receptacle. It has the same exact power requirements as an L2 EV charger.
In the US, you don’t need a neutral for a 240V 50A circuit on a residential single-phase service, there are two line conductors (120V to ground) both connected to a 2-pole single-phase breaker, line-to-line between the two is 240V.
You would need a neutral if it was a 208/120V three-phase service.
Neutrals and grounds are sized per the NEC, neutrals are the same size as the line conductors and equipment grounds are sized off of a table.
#6 conductors and #10 ground is what the NEC calls for.
I just haven't seen server rooms that don't demand a doubly sized neutral when one is required.
I live the NYC 208V life doing mostly resi, though.
Quick search of the spec for that is 6 power supplies, 2 of which are redundant. Looks to use a neutral to me. Says it uses C19/C20 connectors
edit: wait most ranges use 14-50R outlets and need a neutral ran. I am calling your statement into question. Surely harmonics and 120V internal draws cause non-zero neutral current. And I'm sure GPUs have harmonics being semiconductor flavored.
I just put in a dedicated 50 amp circuit myself and my car charges from ~25% to full in about 6-7 hours. But I wanted to present the lazy worst case scenario. The warrior's FUD here is that there isn't enough easily available lithium for everyone in just California alone to have one.
Simultaneous claims that 'agentic' models are dramatically less efficient, but also forecasts efficiency improvements? We're in full-on tea-leaves-reading mode.
No, because at least dark fiber is useful. AI GPUs will be shipped off to developing nations to be dissolved for rare earth metals once the third act of this clown show is over.
> You can already use Claude Code for non engineering tasks in professional services and get very impressive results without any industry specific modifications
After clicking on the link, and finding that Claude Code failed to accurately answer the single example tax question given, very impressive results! After all, why pay a professional to get something right when you can use Claude Code to get it wrong?
This seems to be either LLM AI slop or a person working very hard to imitate LLM writing style:
The key dynamic: X were Y while A was merely B. While C needed to be built, there was enormous overbuilding that D ...
Why Forecasting Is Nearly Impossible
Here's where I think the comparison to telecoms becomes both interesting and concerning.
[lists exactly three difficulties with forecasting, the first two of which consist of exactly three bullet points]
...
What About a Short-Term Correction?
Could there still be a short-term crash? Absolutely.
Scenarios that could trigger a correction:
1. Agent adoption hits a wall ...
[continues to list exactly three "scenarios"]
The Key Difference From S:
Even if there's a correction, the underlying dynamics are different. E did F, then watched G. The result: H.
If we do I and only get J, that's not K - that's just L.
A correction might mean M, N, and O as P. But that's fundamentally different from Q while R. ...
The key insight people miss ...
If it's not AI slop, it's a human who doesn't know what they're talking about: "enormous strides were made on the optical transceivers, allowing the same fibre to carry 100,000x more traffic over the following decade. Just one example is WDM multiplexing..." when in fact wavelength division multiplexing multiplexing is the entirety of those enormous strides.
Although it constantly uses the "rule of three" and the "negative parallelisms" I've quoted above, it completely avoids most of the overused AI words (other than "key", which occurs six times in only 2257 words, all six times as adjectival puffery), and it substitutes single hyphens for em dashes even when em dashes were obviously meant (in 20 separate places—more often than even I use em dashes), so I think it's been run through a simple filter to conceal its origin.
Remember we have about 20 years of poorly written articles along with a few well written ones for the LLM to be trained on. I'm confident that attempting to tell LLM from human writing is a waste of time now that the year is almost over.
Other than that I'd rather choose a comprehensive article than a summary.
I agree, and it feels like an allergy by now to that style specifically. This is doubly annoying because it ruins the reading experience and just makes me question myself constantly because you often can't be quite certain especially for shorter posts/comments.
On topic: It is always quite easy to be the cynical skeptic, but a better question in my view: Is the current AI boom closer to telecoms in 2000 or to video hosting in 2005? Because parallels are strong to both, and the outcomes vastly different (Cisco barely recovered by now compared to 1999 while youtube is printing money).
No, because the datacenters will get used. The demand side exists, whether it’s LLM AIs or something completely different that isn’t AI related. That’s very different from a crash where there is absolutely nothing valuable/useable/demanded underneath the bubble.
‘LLM tuned GPUs’ are just GPUs. The tuning refers to the models and how they use something like CUDA or whatever. There was a GPU shortage even before LLMs properly burst onto the scene, back with crypto mining. Now, it’s possible TPUs might add a wrinkle to later demand side issues when there is a crash but that will depend on how useful TPUs actually end up being to those outside Google. But GPUs will remain useful, whether it’s for gaming, machine learning (not the AI slop variety of this, but more categorising like for self-driving cars or medical imaging etc), or for the next crypto scam. Surely you agree that powerful computing capacity, independent of AI scams, is here to stay, right?
My main point in arguing that now isn’t like 2000 is that unlike in 2000 we have actual hardware and physical assets underpinning this bubble. In 2000 the assets were literally just imaginary. Yes there is speculation now but it is underpinned by silicon that will still be worth decent money even after LLMs are exposed as a hallucinatory mirage.
> This is the opposite of what happened in telecoms. We're not seeing exponential efficiency gains that make existing infrastructure obsolete. Instead, we're seeing semiconductor physics hitting fundamental limits.
What about the possibility of improvements in training and inference algorithms? Or do we know we won't get any better than grad descent/hessians/etc ?
Hardware growth is slow and predictable, but one breakthrough algorithm completely undercuts any finance hypothesis premised on compute not flowing out of the cloud and back to the edges and into the phones.
This is a kind of risk that finance people are completely blind to. Open AI won't tell them because it keeps capital cheap. Startups that must take a chance on hardware capability remaining centralized won't even bother analyzing the possibility. With so many actors incentivized to not know or not bother asking the question, there's the biggest systematic risk.
The real whiplash will come from extrapolation. If an algorithm advance shows up promising to halve hardware requirements, finance heads will reason that we haven't hit the floor yet. A lot of capital will eventually re-deploy, but in the meantime, a great deal of it will slow down, stop, or reverse gears and get un-deployed.
AI had a kind of Jevons paradox approach to efficiency improvements, unfortunately - if you halve the compute requirements with an algorithmic advance, you can run a model twice as big.
The large SOTA models have hit very diminishing returns on further scaling, I think. So you’d rather double the number of models you can run in parallel.
The article claims that AI services are currently over-utilised. Well isn't that because customers are being undercharged for services? A car when in neutral will rev up easily if the accelerator pedal is pushed even very slightly, because there's no load on the engine. But in gear the same engine will rev up much less when the accelerator is pushed the same amount. Will there be the same overutilisation occurring if users have to financially support the infrastructure, either through subscriptions or intrusive advertising?
I doubt it.
And what if the technology to locally run these systems without reliance on the cloud becomes commonplace, as it now is with open source models? The expensive part is in the training of these models more than the inference.
> Will there be the same overutilisation occurring if users have to financially support the infrastructure, either through subscriptions or intrusive advertising? > I doubt it.
I agree. Right now a lot of AI tools are underpriced to get customers hooked, then they'll jack up the prices later. The flaw is that AI does not have the ubiquitous utility internet access has, and a lot of people are not happy with the performance per dollar TODAY, much less when prices rise 80%. We already see companies like Google raising prices stating it's for "AI" and we customers can't opt out of AI and not pay the fee.
At my company we've already decided to leave Google Workspace in the spring. GW is a terrible product with no advanced features, garbage admin tools, uncompetitive pricing, and now AI shoved in everywhere and no way to granularly opt out of a lot of it. Want spell check? Guess what, you need to leave Gemini enabled! Shove off, Google.
Yes, where are you moving to from GW? We're looking for options...
Where are you moving to?
Best Comment
> The article claims that AI services are currently over-utilised. Well isn't that because customers are being undercharged for services?
Absolutely, not only are most AI services free but even the paid portion is coming from executives mandating that their employees use AI services. It's a heavily distorted market.
On the flip side, there are many indications that a huge part of corporate AI usage is "Shadow AI", i.e. where workers use GenAI without their employer's blessing or knowledge. E.g. https://cacm.acm.org/news/the-real-significant-threat-of-sha...
And a majority of those workers do not reveal their AI usage, so they either take credit for the faster work or use the extra time for other activities, which further confounds the impact of AI.
This is also distorting the market, but in other ways.
Yes, over-utilization is a natural response to being undercharged. And being undercharged is a natural result when investors are throwing money at you. During bubbles, Silicon Valley often goes to "lose money, make it up with scale". With the vague idea that after you get to scale, THEN you can figure out how to make money. And fairly consistently, their idea for how to make money is "sell ads".
Past successes like Google encourage hope in this strategy. Sure, it mostly doesn't work. Most of of everything that VCs do doesn't work. Returns follow a power law, and a handful of successes in the tail drive the whole portfolio.
The key problem here doesn't lie in the fact that this strategy is being pursued. The key problem is that it is rare for first mover advantages to last with new technologies. That's why Netscape and Yahoo! aren't among the FAANGs today. The long-term wins go to whoever successfully create a sufficient moat for themselves to protect lasting excess returns. And the capabilities of each generation of AI leapfrogs the last so well that nobody has figured out how to create such a moat.
Today, 3 years after launching the first LLM chatbot, OpenAI is nowhere near as dominant as Netscape was in late 1997, 3 years after launching Netscape Navigator. I see no reason to expect that 30 years from now OpenAI will be any more dominant than Netscape is today.
Right now companies are pouring money into their candidates to win the AI race. But if the history of browsers repeats itself, the company that wins in the long-term would launch in about a year from now, focused on applications on top of AI. And its entrant into the AI wars wouldn't get launched until a decade after that! (Yes, that is the right timeline for the launch of Google, and Google's launch of Chrome.)
Investing in silicon valley is like buying a positive EV lottery ticket. An awful lot of people are going to be reminded the hard way that it is wiser to buy a lot of lottery tickets, than it is to sink a fortune into a single big one.
> Today, 3 years after launching the first LLM chatbot, OpenAI is nowhere near as dominant as Netscape was in late 1997.
Incorrect. There were about 150 millions Internet users in 1998, or 3.5% of total population. The number grew 10 times by 2008 [0]. Netwcape had about 50% of the browser market at the time [1]. In other words, Netscape dominated a small base and couldn’t keep it up.
ChatGPT has about 800 millions monthly users, or already 10% of total current population. Granted, not exclusively. ChatGPT is already a household name. Outside of early internet adopters, very few people knew who Netscape or what Navigator was.
[0] https://archive.globalpolicy.org/component/content/article/1...
[1] https://www.wired.com/1999/06/microsoft-leading-browser-war/...
I was not addressing the size of the market. But the share.
According to https://en.wikipedia.org/wiki/Usage_share_of_web_browsers, Netscape had 60-70% market share. According to https://firstpagesage.com/reports/top-generative-ai-chatbots..., ChatGPT currently has a 60% market share.
But I consider the enterprise market a better indicator of where things are going. As https://menlovc.com/perspective/2025-mid-year-llm-market-upd... shows, OpenAI is one of a pack of significant competitors - and Anthropic is leading the pack.
Furthermore my point that the early market leaders are seldom the lasting winners is something that you can see across a large number of past financial bubbles through history. You'll find the same thing in, for example, trains, automobiles, planes, and semiconductors. The planes example is particularly interesting. Airline companies not only don't have a good competitive moat, but the mechanics of chapter 11 mean that they keep driving each other bankrupt. It is a successful industry, and yet it has destroyed tons of investment capital!
Despite your quibbles over the early browser market, my broader point stands. It's early days. AI companies do not have a competitive moat. And it is way to premature to reliably pick a winner.
We're talking miraculous level of improvement for a SOA LLM to run on a phone without crushing battery life this decade.
People are missing the forest for the trees here. Being the go to consumer Gen AI is a trillion+ dollar business. How many 10s of billions you waste on building unnecessary data centers is a rounding error. The important number is your odds of becoming that default provider in the minds of consumers.
In fact it's apparently $5.2 trillion by 2030 [0] (out of $6.7T total data center spend; meaning all of "traditional IT needs" are less than a quarter of the total). That's the total if you add up all of the firms chasing this opportunity.
I do wonder, if you (and the commenter you replied to) think this is a good thing, will you be OK with a data center springing up in your neighbourhood, driving up water or power prices, emitting CO2? Then if SOTA LLMs become efficient enough to run on a smartphone will you be OK with a data center bailout coming from your tax dollars?
[0]: https://www.mckinsey.com/industries/technology-media-and-tel...
> The important number is your odds of becoming that default provider in the minds of consumers.
I haven't seen any evidence that any Gen AI provider will be able to build a moat that allows for this.
Some are better than others at certain things over certain time periods, but they are all relatively interchangeable for most practical uses and the small differences are becoming less pronounced, not more.
I use LLMs fairly frequently now and I just bounce around between them to stay within their free tiers. Short of some actual large breakthrough I never need to commit to one, and I can take advantage of their own massive spends and wait it out a couple of years until I'm running a local model self-hosted with a cloudflare tunnel if I need to access it on my phone.
And yes, most people won't do that, but there will be a lot of opportunity for cheap providers to offer that as a service with some data center spend, but nowhere near the massive amounts OpenAI, Google, Meta, et al are burning now.
The moat is the chat history and the flywheel of user feedback improving the product.
Given how often smaller LLM companies train on the output of bigger LLM companies, it's not much of a moat.
LLMs complete text. Every query they answer is giving away the secret ingredient in the shape of tokens.
It’s extremely easy to switch.
I used ChatGPT for every day stuff, but in my experience their responses got worse and I had to wait much longer to get them. I switched to Gemini and their answers were better and were much faster.
I don’t have any loyalty to Gemini though. If it gets slow or another provider gives better answers, I’ll change. They all have the same UI and they all work the same (from a user’s perspective).
There is no moat for consumer genAI. And did I mention I’m not paying for any of it?
It’s like quick commerce, sure it’s easy to get users by offering them something expensive off of VC money. The second they raise prices or offer degraded experience to make the service profitable, the users will leave for another alternative.
Just build more tulip fields/railways/websites no matter the cost - the golden age is inevitable! ;-)
How big does an LLM need to be to support natural language queries with RAG?
My hot (maybe just warm these days) take is, the problem with voice assistants on phones is they have to be able to have reasonable responses to a long tail or users will learn not to use them, since the use cases aren’t discoverable and the primarily value is talking to it like a person.
So voice assistants backed by very large LLMs over the network are going to win even if we solve the (substantial) battery usage issue.
Why even bother with the text generation then? You could just make a phone call to an LLM with a TTS frontend. Like with directory enquiries back in the day. Which can be set up as easily as a BBS if you have a home server rack like Jeff Geerling makes youtube videos about.
Of all the players, I'd argue Google certainly knows how to give away a product for free and still make money.
The local open source argument doesn't hold water for me -- why does anyone buy Windows, Dropbox, etc when there's free alternatives?
Yes, it seems to me their strategy is to watch the OpenAI, Anthropics, etc of the world bleed themselves to death.
Not many people buy Windows, they buy laptops that happen to have Windows installed. IMO this is a worthwhile distinction because most people don’t really care about operating systems anyway, and would happily (I suspect, at least) use an Open Source one if it came installed and configured on a device that they got in a store.
Installing an OS is seen as a hard/technical task still. Installing a local program, not so much. I suspect people install LLM programs from app stores without knowing if they are calling out to the internet or running locally.
Besides the fact that this article is obviously AI generated (and not even well, why is there mismatches in british/american english? I can only assume that the few parts in british english are the human author's writing or edits), yes "overutilization" is not a real thing. There is a level of utilization at every price point. If something is "overutilizated" that actually means it's just being offered at a low price, which is good for consumers. It's a nice scare word though and there's endless appetite at the moment for ai-doomer articles.
Author here, I mix up American and British English all the time. It's pretty common for us Brits to do that imo.
See also how all (?) Brits pronounce Gen Z in the American way (ie zee, not zed).
Brit here… I say Gen Zed!
Sorry but it's highly suspect to be spelling the same word multiple different ways across paragraphs. You switch between using centre/center or utilization/utilisation? It is a very weird mistake to make for a human.
> why is there mismatches in british/american english
You sometimes see this with real live humans who have lived in multiple counties.
> multiple counties
Pay no attention to those fopheads from Kent. We speak proper British English here in Essex
I do this because I'm a non-native english speaker. My preference varies from word to word. I write color, but i also write aliminium.
> why is there mismatches in british/american english
Some people are not from usa or England.
By this logic loss leaders to drive out competition are good gor the consumer, no?
To be honest it doesn't feel manually edited.
Bullet points hell, a table that feels it came straight out of grok.
Will the OpenRouter marketplace of M clouds X N models die if the investor money stops? I believe its a free and profitable service, offered completely pay as you go.
> I doubt it.
I don't. This is simply the "drug dealer" model where the first hit is free. They know that once people are addicted, they will keep coming back.
The question of course is, will they keep coming back? I think they very much will. There are indications that GenAI adoption is already increasing labor producitivity labor improvements at a national scale, which is quite astounding for a technology just 3 years old: https://news.ycombinator.com/item?id=46061369
Imagine a magic box where you put in some money and get more productivity back. There is no chance Capitalism (with a capital "C") is going to let such a powerful growth machine wither on the vine. This mad AI rush is all about that.
OpenAI has 800,000,000 weekly users but only 20,000,000 are paying while 780,000,000 are free riding. Should they by accident under provision then they could simply remove the freebee and raise the prices for the paying clients. But that is not what they want.
IMHO the investors are betting on a winner-takes-it-all market and that some magic AGI will be coming out of OpenAI or Anthropic.
The questions are:
How much money can they make by integrating advertising and/or selling user profiles?
What is the model competition going to be?
What is the future AI hardware going to be - TPUs, ASICs?
Will more people have powerful laptops/desktops to run a mid-sized models locally and be happy with it?
The internet didn't stop after the dotcom crash and the AI wont stop either should there be a market correction.
>OpenAI has 800,000,000 weekly users but only 20,000,000 are paying while 780,000,000 are free riding.
By itself, this doesn't tell us much.
The more interesting metric would be token use comparison across free users, paid users, API use, and Azure/Bedrock.
I'm not sure if these numbers are available anywhere. It's very possible B2B use could be a much bigger market than direct B2C (and the free users are currently providing value in terms of training data).
800M people are not using chatGPT every week. Be a little less credulous.
I just thought about it, and honestly, from my surroundings of people aged 12-70s, across multiple continents, I can’t think of anyone who isn’t using some sort of LLM once a week.
The thing that makes AI investment hard to reason about for individuals is that our expectations are mostly driven by a single person’s usage, just like many of the numbers reported in the article.
But the AI providers are betting, correctly in my opinion, that many companies will find uses for LLM’s which are in the trillions of tokens per day.
Think less of “a bunch of people want to get recipe ideas.”
Think more of “a pharma lab wants to explore all possible interactions for a particular drug” or “an airline wants its front-line customer service fully managed by LLM.”
It’s unusual that individuals and industry get access to basically similar tools at the same time, but we should think of tools like ChatGPT and similar as “foot in the door” products which create appetite and room to explore exponentially larger token use in industry.
> Think more of “a pharma lab wants to explore all possible interactions for a particular drug”
Pharma does not trust OpenAI with their data, and they don't work on tokens for any of the protein or chemical modeling.
There will undoubtedly be tons of deep nets used by pharma, with many $1-10k buys replacing more expensive physical assays, but it won't be through OpenAI, and it won't be as big as a consumer business.
Of course there may be other new markets opened up but current pharma is not big enough to move the needle in a major way for a company with an OpenAI valuation.
My claim is that there will exist some company which pharma is willing to trust for AI research…they presumably trust Microsoft with their email today.
But my bigger claim is that ~half the Fortune 500 will be able to profitably deploy AI with spends in the tens or hundreds of millions per year quite soon. Not that pharma itself is a major contributor to that effect.
But for 'AI' to be a winner-take-all market, it seems that the winner would have to be using customer data to improve the 'AI'. Not only do you have to believe that one of these (relatively) under-capitalized upstarts can corral the money, but also that they can convince (enterprise) customers to 'fork over' their proprietary data to only one provider, and also that the provider can then charge a monopoly rent.
Those all seem possible, but I wouldn't assign greater than a 50% probability to any of them, and the valuations seem to imply near-certainty.
When I'm building out a new feature, I can churn through millions of tokens in Claude code. And that's just me... Now think about Claude code but integrated with Excel or datadog, or whatever app could be improved through LLM integration. Think about the millions of office workers, beyond just software engineers, who will be running hundreds of thousands or millions of tokens per day through these tools.
Let's estimate 200 million office workers globally as TAM running an average of 250k tokens. That's 50 trillion tokens DAILY. Not sure what model provider profit per token is, but let's say it's .001 cents.
Thats $500M per day in profit.
I’m with you on the Claude Code example —- it matches my experience.
But I do think the important thing to look forward to is AI work which is totally detached from human intervention.
I find it absurd to pay for tokens I cant control, predict or even check in any reasonable way. It is literally amounts to "pay whatever random money the company asks you to pay" kind of contract.
Currently there is no profit per token, quite a bit of loss per token, that's the problem. Your not going to make it up in volume.
Do you have a source for that? I'm especially interested in a source for Anthropic.
https://www.wsj.com/tech/ai/openai-anthropic-profitability-e...
Anthropic expects to break even in 2028. They’re all unprofitable now.
> “an airline wants its front-line customer service fully managed by LLM.”
This has been experimented on before by many companies over the recent years, most notably Klarna which was among the earliest guinea pigs for it and had to later on backtrack on this "novel" idea when the results came out.
Does neither mean it can't work nor that it can't work with LLMs. Just that hacking together some RAG chatbot is probably not it.
But if I'm a pharma lab, I don't want to rely on a statistical engine that makes mistakes to answer those questions, I want to query a database that is deterministic.
Sam Altman says that he thinks scientific research is a huge opportunity for AI to contribute as it more fully develops and I think he’s right.
Since I’m not a scientific researcher, I have no idea if he’s just blowing smoke but I think it’s reasonable to think of a purpose-built system which has an LLM component being used by a team to do something useful.
This. LLM is NOT the tool for a pharma lab - properly trained ML is the right tool. Heck, English is probably not even the right LANGUAGE to use for discussing chemical interactions.
Would the proper language be math, or another human language?
I could see things like "nitrate" and "nitrite" possibly being a stumbling block for an LLM.
After reading the article and the comments, here are a few points people are missing from their analysis:
- OverUtilized/UnderCharged: doesn't matter because...
- Lead Time vs. TCO vs. IRS Asset Deprecation: The moment you get it fully built, it's already obsolete. Thus from a CapEx point of view, if you can lease your compute (including GPU) and optimize the rest of the inputs for similar then your CapEx overall is much lower and tied to the real estate - not the technology. The rest is cost of doing business and deductible in and of itself.
- The "X" factor: Someone mentioned TPU/ASIC but then there is the DeepSeek factor - what if we figure out a better way of doing the work that can shortcut the workflow?
- AGI partnerships: Right now, you see a lot of Mega X giving billions to Mega Y because all of them are trying to get their version of Linux or Apache or whatever at parity with the rest. Once AGI is settled and confirmed, then most all of these partnerships will be severed because it then becomes which company is going to get their AI model into that high prestige Montessori school and into the right ivy league schools - like any other rich parent would for their "bot" offspring.
So what will it look like when it crashes? A bunch of bland empty "warehouses" with mobile PDU's once filling all their parking lot space gone. Whatever "paradise" that was there may come back... once you bulldoze all that concrete and steel. The money will do something else like a Don McLean song.
>once you bulldoze all that concrete and steel
You're not quite thinking things through there man. Once the elites who built these follies have gone, the mob will go shopping for building materials. I wouldn't be surprised even if people end up living in these datacentres once they become derelict. They have AC after all.
Welp Gemini got me. Using G3 to improve what I write, generate specific images, and use NotebookLM to dive into some research materials. Tried to do a bit each day with my free credits, but hit the limit too often. G2.5 was not nearly as useful. So I upgraded my baselevel Google workspace plan. Recently spoke to someone who is also using G3 a lot with good results. YMMV re: G3, but Google hooked me, and now I pay more. However I think it is worth it for what I do. G3 is my helpful, nerdy work mate. I never plan to use agenic AI. Not using ChatGPT much if any at all anymore. Sorry Sam.
I am still a little skeptical about utilisation rates. If demand is so extreme, wouldn't we see rental prices for H100/A100 prices go up or maintain? Wouldn't the cost for such a gpu still be high (you can get em 3k used).
On "runpod community cloud" renting a 5090 costs $0.69/hour [1] and it consumes about $0.10/hour electricity, if running at full power and paying $0.20/kWh.
On Amazon, buying a 5090 costs $3000 [2]
That's a payback time of 212 days. And Runpod is one of the cheaper cloud providers; for the GPUs I compared, EC2 was twice the price for an on-demand instance.
Rental prices for GPUs are pretty darn high.
[1] https://www.runpod.io/pricing [2] https://www.amazon.com/GIGABYTE-Graphics-WINDFORCE-GV-N5090G...
A 5090 gaming card is a different beast to the 80gb ai cards. That one was 40k usd so for renting that to hit 1.50 dollar per hour is interesting.
Telco capex was $100 billion at the peak of the IT bubble, give or take. There's going to be $400 billion investments in AI in 2025. Just saying.
Yes or no conclusions aside (and despite its title, the article deserves better than that), the key point is I think this one: “But unlike telecoms, that overcapacity would likely get absorbed.”
Telecom (dark fiber) capacity got absorbed too. Eventually. After a ton of bankruptcies.
Stylistically, this smells like it was copy and pasted from straight out Deep Research. Substantively, I could use additional emphasis on the mismatch between expectations and reality with regards to telco debt-repayment schedule.
I'm curious about the telecom collapse. How was it not in the pipeline that much better use of the fibres was around the corner? Surely they would have known that people were looking into it with some promise?
It's human nature not to admit the obvious until it is way way way too late.
Some of the utilization comparisons are interesting, but the article says 2 trillion was spent on laying fiber but that seems suspicious.
There's an enormous amount of unused, abandoned fiber. All sorts of fiber was run to last mile locations, across most cities in the US, and a shocking amount effectively got abandoned in the frenzy of mergers and acquisitions. 2 trillion seems like a reasonable estimate.
Giant telecoms bought big regional telecoms which came about from local telecoms merging and acquiring other local telecoms. A whole bunch of them were construction companies that rode the wave, put in resources to run dark fiber all over the place. Local energy companies and the like sometimes participated.
There were no standard ways of documenting runs, and it was beneficial to keep things relatively secret, since if you could provide fiber capabilities in a key region, but your competition was rolling out DSL and investing lots of money, you could pounce and make them waste resources, and so on. This led to enormous waste and fraud, and we're now on the outer edge of usability for most of the fiber that was laid - 29-30 years after it was run, most of it will never be used, or ever have been used.
The 90s and early 2000's were nuts.
I so desperately wish it weren't abandoned. I hate that it's almost 2026 and I still can't get a fiber connection to my apartment in a dense part of San Diego. I've moved several times throughout the years and it has never been an option despite the fact that it always seems to be "in the neighborhood".
That has nothing to do with fiber, it’s all about politics and a regulatory environment where nobody is incented to act. Basically, the states can’t fully regulate internet and the Federal government only wants to fund buildouts on a pork barrel basis. Most recently rural.
At the local level, there is generally a cable provider with existing rights of way. To get a fiber provider, there’s 4 possible outcomes: universal service with subsidy (funded by direct subsidy), cherry-picked service (they install where convenient), universal service (capitalized by the telco) and “fuck you”, where they refuse to operate. (ie. Verizon in urban areas)
The private capitalized card was played out by cable operators in the 80s (they were innovators then, and AT&T was just broken up and in chaos). They have franchise agreements whose exclusivity was used as loan collateral.
Forget about San Diego, there are neighborhoods in Manhattan with the highest population density in the country where Verizon claims it’s unprofitable to operate.
I served on a city commission where the mayor and county were very interested in getting our city wired, especially as legacy telco services are on the way out and cable costs are escalating and will accelerate as the merger agreement that formed Spectrum expires. The idea was to capitalize last mile with public funds and create an authority that operated both the urban network and the rural broadband in the county funded by the Federal legislation. With the capital raised with grants and low cost bonding (public authority bonds are cheap and backed by revenue and other assets), it would raise a moderate amount of income in <10 years.
We had the ability to get the financing in place, but we would have needed legislation passed to get access to rights of way. Utilities have lots of ancient rights and laws that make disruption difficult. The politicians behind it turned over before that could be changed.
The worst part is it'd probably cost less than $100 of fiber and labor to splice something into your building, maybe $200-400 of gear to light it up, and you'd have a 10 gbps pipe back to some colo. It's more economical to run new fiber in most places these days, even if the local ISP knows exactly where all the old abandoned legacy lines are run, because of subsidization and basically scamming. The big companies like Lumen keep their knowledge regionally compartmented, legally shielded, and deliberately obfuscated, because if it became known that existing fiber was already run to a place they claim they can't serve, they can't get access to yet more funding for their eternal "service for the underserved" government money grift.
I stumbled on old maps that showed a complete coverage of fiber in my municipality, paperwork from a company that was acquired, and which in turn merged, then was bought out by one of the big 5 ISPs. When local officials requested information regarding existing fiber, this ISP refused and said any such information was proprietary. They later bid on and won contracts to run new fiber (parallel to existing lines which they owned, which still had more than a decade of service life left in them at that point).
I estimated that only around 10-15% of the funding went toward actual labor and materials, the remainder was pure profit. The local government considered it a major victory, money well spent.
For infrastructure, central planning and state-run systems make a lot of sense - this after all is how the USA's interstate highway system was built. The important caveat is that system components and necessary tools should be provided by the competitive private sector through transparent bidding processes - eg, you don't have state-run factories for making switches, fiber cable, road graders, steel rebar, etc. There are all kinds of debatable issues, eg should system maintenance be contracted out to specialized providers, or kept in-house, etc.
The GDP 1995-2000 (inclusive) was about $52T. So that assertion would mean that about %3.8 of the US' economic activity was laying fiber. That seems like a lot, but in my ignorance doesn't sound totally impossible.
The 2001 telecoms crash drove benefits for companies that came later in the availability of inexpensive dark fiber after the bubble popped. WorldCom, ICG, Williams sold off to Verizon, Level 3, Teleglobe, and others. That in turn helped future Internet companies gain access to plentiful and inexpensive bandwidth. Cable telephony companies such as Cablevision Systems, Comcast, Cox Communications, and Time Warner, used the existing coaxial connections into the home to launch voice services.
Rail and fiber deprecates on multiple decade timescales. AI data centers close to tulips. Even assuming we manage to make data center stretch to 10 years, these assets won't be around long enough to support ecosystem of new companies if the economics stops making sense. Ultimately the only durable thing is any power infra that gets built, vs rail and fiber where inheritance isn't just rail networks or fiber but like 1000s of kilometers of earthwork projects to build out massive physical networks.
Is this a question of the GPU chips dying due to be being warm semiconductors, or of them becoming outdated in relation to new chips?
Data centers last decades. Many or the current AI hosting vendors such as Coreweave have crypto origin. Their data centers were built out in 2010s, early 2020s.
Many of legacy systems still running today are IBM or Solaris servers at 20, 30 year old. No reason to believe GPU won’t still be in use in some capacity (e.g. interference) a decade from now.
Used to last decades, the world didn't move at this speed before.
This is indeed true, but doesn't fiber have a far longer lifetime than GPU heavy data centers? The major cost center is the hardware, which has a fairly short shelf life.
Well you still get the establishment of 1) large industrial buildings 2) water/electricity distribution 3) trained employees who know how to manage a data center
Even if all of the GPUs inside burn out and you want to put something else entirely inside of the building, that's all still ready to go.
Although there is the possibility they all become dilapidated buildings, like abandoned factories
The building and electrical infrastructure are far cheaper than the hardware. So much so that the electricity is a small cost of the data center build out, but a major cost for the grid.
Of the most valuable part is quickly depreciating and goes unused within the first few years, it won't have a chance for long term value like fiber. If data centers become, I don't know, battery grid storage, it will be very very expensive grid storage.
Which is to say that while there was an early salivation for fiber that was eventually useful, overallocation of capital to GPUs goes to pure waste.
Don’t think looking at power consumption of b200s is a good measure of anything. Could well be an indication of higher density rather than hitting limits and cranking voltage to compensate
Yes, one of NVidia's selling points for the b200 is that performance per watt is better than before. High power consumption without controlling for performance means nothing.
Is there a way in which this is good for a segment of consumers? When the current gen of GPUs are too old, will the market be flooded with cheap GPUs that benefit researchers and hobbyists who therwis would not afford them?
GPUs age surprisingly gracefully. If a GPU isn't cutting edge, you just tie two or more of them together for a bit more power consumption to get more or less the same result as the next generation GPU.
if there's ever a glut in GPUs that formula might change but it sure hasn't happened yet. Also, people deeply underestimate how long it would take a competing technology to displace them. It took GPUs nearly a decade and the fortunate occurrence of the AI boom to displace CPUs in the first place despite bountiful evidence in HPC that they were already a big deal.
>GPUs age surprisingly gracefully. source: it was revealed to me in a dream
source: Nvidia Shields still shipping with Maxwell GPUs from 2014 and many university labs still running on Pascal GPUs from 2016. But stupid facts don't matter, this one hit you wrong in the feels, amIRight?
Unlikely, for a few reasons:
* The GPUs in use in data centers typically aren’t built for consumer workloads, power systems, or enclosures.
* Data Centers often shred their hardware for security purposes, to ensure any residual data is definitively destroyed
* Tax incentives and corporate structures make it cheaper/more profitable to write-off the kit entirely via disposal than attempt to sell it after the fact or run it at a discount to recoup some costs
* The Hyperscalers will have use for the kit inside even if AI goes bust, especially the CPUs, memory, and storage for added capacity
That’s my read, anyway. They learned a lot from the telecoms crash and adjusted business models accordingly to protect themselves in the event of a bubble crash.
We will not benefit from this failure, but they will benefit regardless of its success.
Some of them will probably be starving, homeless, or bedridden by the time that happens but yes they can get cheap GPUs
Many researchers and hobbyists cannot even plug in a 10 KW 8 GPU DGX server.
Why not? It’s 40A at 240V, or 25% of the continuous load rating of a 200A 240V single-phase service.
If someone can afford an 8 GPU server, they should be able to afford some #6 wire, a 50A 2P breaker, and a 50A receptacle. It has the same exact power requirements as an L2 EV charger.
You probably run a bigger (perhaps double) neutral and care about a stronger ground. But yeah, the $12 is rounding error at this scale
In the US, you don’t need a neutral for a 240V 50A circuit on a residential single-phase service, there are two line conductors (120V to ground) both connected to a 2-pole single-phase breaker, line-to-line between the two is 240V.
You would need a neutral if it was a 208/120V three-phase service.
Neutrals and grounds are sized per the NEC, neutrals are the same size as the line conductors and equipment grounds are sized off of a table.
#6 conductors and #10 ground is what the NEC calls for.
I just haven't seen server rooms that don't demand a doubly sized neutral when one is required.
I live the NYC 208V life doing mostly resi, though.
Quick search of the spec for that is 6 power supplies, 2 of which are redundant. Looks to use a neutral to me. Says it uses C19/C20 connectors
edit: wait most ranges use 14-50R outlets and need a neutral ran. I am calling your statement into question. Surely harmonics and 120V internal draws cause non-zero neutral current. And I'm sure GPUs have harmonics being semiconductor flavored.
That doesn't exactly bode well for the EV revolution, then, does it?
The average commute in the United States is about 24 miles a day round trip. That's about 10 kWH. That's enough to charge overnight on a 15A circuit.
You can drive around with 24 miles of charge, but ... I'll pass, thanks.
In reality, if you have a dryer outlet, you have a good fraction of 10 kW available.
I just put in a dedicated 50 amp circuit myself and my car charges from ~25% to full in about 6-7 hours. But I wanted to present the lazy worst case scenario. The warrior's FUD here is that there isn't enough easily available lithium for everyone in just California alone to have one.
Almost all home EV charging is <=10kW.
wut?
Simultaneous claims that 'agentic' models are dramatically less efficient, but also forecasts efficiency improvements? We're in full-on tea-leaves-reading mode.
Yes
No, because at least dark fiber is useful. AI GPUs will be shipped off to developing nations to be dissolved for rare earth metals once the third act of this clown show is over.
Amazing article, I found it fascinating.
> You can already use Claude Code for non engineering tasks in professional services and get very impressive results without any industry specific modifications
After clicking on the link, and finding that Claude Code failed to accurately answer the single example tax question given, very impressive results! After all, why pay a professional to get something right when you can use Claude Code to get it wrong?
This seems to be either LLM AI slop or a person working very hard to imitate LLM writing style:
The key dynamic: X were Y while A was merely B. While C needed to be built, there was enormous overbuilding that D ...
Why Forecasting Is Nearly Impossible
Here's where I think the comparison to telecoms becomes both interesting and concerning.
[lists exactly three difficulties with forecasting, the first two of which consist of exactly three bullet points]
...
What About a Short-Term Correction?
Could there still be a short-term crash? Absolutely.
Scenarios that could trigger a correction:
1. Agent adoption hits a wall ...
[continues to list exactly three "scenarios"]
The Key Difference From S:
Even if there's a correction, the underlying dynamics are different. E did F, then watched G. The result: H.
If we do I and only get J, that's not K - that's just L.
A correction might mean M, N, and O as P. But that's fundamentally different from Q while R. ...
The key insight people miss ...
If it's not AI slop, it's a human who doesn't know what they're talking about: "enormous strides were made on the optical transceivers, allowing the same fibre to carry 100,000x more traffic over the following decade. Just one example is WDM multiplexing..." when in fact wavelength division multiplexing multiplexing is the entirety of those enormous strides.
Although it constantly uses the "rule of three" and the "negative parallelisms" I've quoted above, it completely avoids most of the overused AI words (other than "key", which occurs six times in only 2257 words, all six times as adjectival puffery), and it substitutes single hyphens for em dashes even when em dashes were obviously meant (in 20 separate places—more often than even I use em dashes), so I think it's been run through a simple filter to conceal its origin.
Remember we have about 20 years of poorly written articles along with a few well written ones for the LLM to be trained on. I'm confident that attempting to tell LLM from human writing is a waste of time now that the year is almost over.
Other than that I'd rather choose a comprehensive article than a summary.
Attempting to tell LLM writing from human writing is usually a waste of time, but apparently not in this case.
Did you see the reply from the author to similar statements?
You mean https://news.ycombinator.com/item?id=46138663? He said that he mixed up British and American English all the time, but didn't try to deny that the article was AI slop.
I agree, and it feels like an allergy by now to that style specifically. This is doubly annoying because it ruins the reading experience and just makes me question myself constantly because you often can't be quite certain especially for shorter posts/comments.
On topic: It is always quite easy to be the cynical skeptic, but a better question in my view: Is the current AI boom closer to telecoms in 2000 or to video hosting in 2005? Because parallels are strong to both, and the outcomes vastly different (Cisco barely recovered by now compared to 1999 while youtube is printing money).
You guys make me laugh.
No, because the datacenters will get used. The demand side exists, whether it’s LLM AIs or something completely different that isn’t AI related. That’s very different from a crash where there is absolutely nothing valuable/useable/demanded underneath the bubble.
> The demand side exists, whether it’s LLM AIs or something completely different that isn’t AI related.
What do you think LLM tuned GPUs or TPUs are going to be used for that is completely different and not AI related?
‘LLM tuned GPUs’ are just GPUs. The tuning refers to the models and how they use something like CUDA or whatever. There was a GPU shortage even before LLMs properly burst onto the scene, back with crypto mining. Now, it’s possible TPUs might add a wrinkle to later demand side issues when there is a crash but that will depend on how useful TPUs actually end up being to those outside Google. But GPUs will remain useful, whether it’s for gaming, machine learning (not the AI slop variety of this, but more categorising like for self-driving cars or medical imaging etc), or for the next crypto scam. Surely you agree that powerful computing capacity, independent of AI scams, is here to stay, right?
My main point in arguing that now isn’t like 2000 is that unlike in 2000 we have actual hardware and physical assets underpinning this bubble. In 2000 the assets were literally just imaginary. Yes there is speculation now but it is underpinned by silicon that will still be worth decent money even after LLMs are exposed as a hallucinatory mirage.
Nice article; far from bullet-proof, but it brings up some interesting points. HN comments are vicious on the topic of AI non-bubbles.
> This is the opposite of what happened in telecoms. We're not seeing exponential efficiency gains that make existing infrastructure obsolete. Instead, we're seeing semiconductor physics hitting fundamental limits.
What about the possibility of improvements in training and inference algorithms? Or do we know we won't get any better than grad descent/hessians/etc ?
No. AI data center, or any data center is designed with incorrect data structure resulting in over utilisation of computing resource.
Hardware growth is slow and predictable, but one breakthrough algorithm completely undercuts any finance hypothesis premised on compute not flowing out of the cloud and back to the edges and into the phones.
This is a kind of risk that finance people are completely blind to. Open AI won't tell them because it keeps capital cheap. Startups that must take a chance on hardware capability remaining centralized won't even bother analyzing the possibility. With so many actors incentivized to not know or not bother asking the question, there's the biggest systematic risk.
The real whiplash will come from extrapolation. If an algorithm advance shows up promising to halve hardware requirements, finance heads will reason that we haven't hit the floor yet. A lot of capital will eventually re-deploy, but in the meantime, a great deal of it will slow down, stop, or reverse gears and get un-deployed.
AI had a kind of Jevons paradox approach to efficiency improvements, unfortunately - if you halve the compute requirements with an algorithmic advance, you can run a model twice as big.
The large SOTA models have hit very diminishing returns on further scaling, I think. So you’d rather double the number of models you can run in parallel.
Holly cow, we've found an exception to Betteridge's Law of Headlines! Talk about burying the lede!
If you read the article, then this is not an exception to the law
If you read the article _and_ agree with the author's conclusions. I did the former, but not the later.