This narrative isn't consistent with reality. Cloud computing is much older than 10 years ago. We were doing production grade cloud computing over 16 years ago. The OSS revolution dates to the late 90's. The author seems to think that SWE was in a golden age 10 years ago. Sorry, but it's about the same. We have the same problems in SWE as we had 40 years ago. The scale and ambition of the problems we are solving have increased as compute capabilities have increased.
The timeline is very confused, but the arc is mostly right.
1. Consolidating internet-dependent projects to a few winning cloud vendors and a dozen winning SaaS providers, made all of those project have coincident failure points. Which is great for corporate accountability ("it's Amazon's issue, not ours!") but bad for end users ("shit, EVERYTHING is down today")
2. The movement to migrate fat local-first projects towards thin internet-dependent or server-hosted ones made almost all software share those same coincident failure paths. This was good for entrenching a subscription business model and brought some end user conveniences, but meant everything being down meant everything being down.
3. Anxious and lazy dependency-first development culture ("you could never write that as well as some github rando") meant projects now had an order of magnitude more contributors but only reviewed and owned code for a surface level, making all these projects extremely vulnerable to supply chain failures and subject to constant maintenance churn.
4. Employment and team tenure continually fell so that most projects no longer have experienced owners and instead just temporary custodians.
You can find the seeds of many those problems in the 1990's and earlier and you can even argue that some of them were inevitable (and even worthwhile) destinations for the industry as it evolved without an engineering guild to slow it down and insist on rigor or evidenced safety.
But anybody who's been around for a few decades can attest to how these risky patterns rapidly grew into norms over the past 20 years and shifted the industry from one where any given software had some idiosyncratic bugs that users could learn to work around to one where all software had lots of bugs, many of them correlated across apps, whete failures would wash through and shut down (or make vulnerable) whole industries for hours or days, with effecrively nobody in particular to hold to account for any of it.
Software has never been in a pristine "golden age", but the last 20 years built almost everything on one giant, shared house of cards that's now teetering on the brink of collapse. Better or wose, it's indisputably different than it was before.
> Anxious and lazy dependency-first development culture
I think this is the completely wrong takeaway.
Ober the last ~15 years I have probably used around ~10 web frameworks/libraries
_not a single one was even close to being complete or mostly correct, even just for standard HTTP ~1.1 handling_!!! (The routing, path, header handling. Not any of the complex stuff in HTTP2, Quick etc.)
the problem isn't using dependencies, that is the solution (not reinventing the wheel again).
And preferring reusing solutions which are more complete, practical testet and have more resources to maintain/improve basic functionality is (in most situations) not "anxious" development, but (should be) common sense.
The amount of hidden complexity in even somewhat basic stuff like HTTP is enourmous, rewriting everything all the time is wasting billions across the industry all the time and (in general) increases the chance for bugs.
The problem isn't "using dependencies" or small dependencies, but:
- availability (or lack) of high quality well maintained dependencies
- "no one" wanting to pay for software dependencies
- misguided tendencies for 'clever' & overengineered solution in one extreme (e.g. the stuff which lead to log4shell)
- misguided obsession with deceptive simplicity in artificial examples which lead to 1001 incomplete solutions because of the obsessiv simplicity doesn't stop at cutting out over-engineering but also cuts out fundamental parts
- lack of widely adopted, effective, tooling & infrastructure to trust dependencies.
- lack of tooling to more efficient reuse and compose working parts, the way we do programming seems just very fundamental broken.
- also a lot of legacy tech we are stuck with making many things unnecessary complicated
Also as a side note, number of packages != number of entities to trust. Weather you use 10 or 1 package from the same source, maintained and published in the same way often makes little to no difference for supply chain security.
Powerful. I feel like this is emblematic of the economy as a whole since the 80s. Consolidation and outsourcing makes hyper-efficient but brittle businesses.
With the advent of individual computers/disks/networking equipment not breaking as much, we optimised for Efficient+Cheap software. This results in systems prone to cascading failures.
The most resilient systems I've worked (mostly telecom) on we built from crappy parts which broke all the time, and they sure weren't cheap.
> Anxious and lazy dependency-first development culture ("you could never write that as well as some github rando")
I’ll die on this hill. You should pin your dependencies so you don’t get pwned, and then use dependencies for everything mildly complicated that isn’t core to your product.
I bet 90% of people reading this work somewhere that has built a model router in the last year.
I understand the sentiment. You work at a sparkling water company, and building a durable workflow engine is much more interesting than building a website.
But without that tech being your core product, maintaining and evolving it will lose priority. If it requires barely no maintenance, it’s even worse, because when it inevitably does, you don’t know if deploying a new version still works.
Use stuff that exists. If the dependencies don’t do what you need, fork them, and consider contributing a patch.
Pinning deps only works if you subscribe to security release notifications from every one AND requires that they distribute those ~anywhere. Otherwise you’re missing patches.
And if you’re allowing patches via SemVer, you’re relying on authors using SemVer correctly, which they won’t. For example, the common practice of ~never having a 1.0 so the minor release is basically the major and the patch is effectively the minor.
Tracking version changes and breaking api changes in dependencies and cves/vulnerabilities has a non zero cost, it’s simply dishonest to pretend it doesn’t.
This is a common take, but it's most attractive for its cynicism, it doesn't fully represent the reality.
Cloud vendors are also great for allowing corporations to implement systems that would be prohibitive for them to implement, operate, and maintain as reliably on their own. Even big enterprises often struggle to implement everything they need, that a cloud provider can provide out of the box.
I worked at a multi-billion $ subsidiary of a global Fortune 500 company that tried, as part of its move off mainframes, to "go its own way", with their own hardware in Equinix data centers. The entire executive level of the IT team responsible for that effort - about five or six people - were eventually fired, along with much acrimony, because they couldn't pull it off. There were plenty of warning signs, but the final straw came when there was a major security breach, which was unacceptable because this company had federal customers and were subject to all sort of federal regulations, including CFIUS because the parent company was non-US. (They once had to rewrite an entire major system from scratch because of a breach of CFIUS regulations that made the government consider their code to be potentially compromised.)
That may be an unusual case, but the point is that if even large, well-funded, established companies can get a lot of benefit from cloud systems, that goes double or more for smaller companies, companies in less attractive industries, etc. etc.
(Pointing this out sometimes triggers people who maintain servers themselves, especially here on HN, but the point is those people don't work at the companies in question, and essentially don't scale - i.e. it's difficult to recruit enough of them and manage them and put together a durable system with enterprise-grade reliability that depends on having 10x sysadmins.)
And yes, "shit, EVERYTHING is down today" is a consequence of all this, but the alternative is generally much worse for individual companies.
IMHO it’s a skill issue. Nothing wrong with bare metal.
My friend runs an OSS project with 10s of thousands of users on a fairly realtime and latency sensitive application and it works great and has been very reliable. He does storage, compute, networking, all of it.
I run a FinTech startup on bare metal and do it for 1/10th the price of AWS (In two Geographic Regions). Reliability has been 4-5 9s over the last year. I’ve also worked for a few F500s. I’ve been the guy responsible for buying, racking, networking, and coding. I’ve also worked at AWS and am quite familiar with the pros and cons and nothing about my opinion has really changed, the cloud is a money printer that serves to make certain people extremely rich. Companies and individuals standing up their own hardware and learning how it works is a threat to their business model.
> Companies and individuals standing up their own hardware and learning how it works is a threat to their [cloud/big tech] business model.
This point never occurred to me but it explains so much. In addition to the points made in this thread, I have seen a dismantling of tech education. For example, I created the two-year degree program for Florida in Linux system administration 20+ years ago. I later learned the program's requirements were gutted by a committee; the took a Windows administration degree document and s/Windows|Microsoft/Linux/g. The new requirements included adding users using Linux active directory and other absurdities. Programming degree knowledge requirements were also dumbed down.
All this because educated tech workers are a threat to someone's bottom line.
This is exactly what I was referring to in my parenthetical.
You're thinking of this from the perspective of your own personal expertise, not from a business perspective. The problem is that you don't scale. There are millions of businesses using cloud services. You're not working for any of them. Do you want to?
And are there millions of people with your experience and skills? Doubtful. (If there were, you'd be a lot less valuable.) And a company of any significant size can't rely on one person to operate their systems, so you need a team, and you need to manage that team, and the managers have to be good as well.
Also, 10s of thousands of users is tiny in the modern world. Instagram scaled to 30 million users on AWS with something like 13 employees, before it was acquired by Facebook. It would have been much different if they'd had to do that on their own hardware. They're an outlier, but many business are able to use the cloud to scale to levels that would require a massive internal effort otherwise, especially if they have to meet compliance requirements, integrate with other enterprises, etc.
> Companies and individuals standing up their own hardware and learning how it works is a threat to their business model.
They don't care much about this "threat", because executives look at the situation and choose cloud because it makes business sense.
This reminds me of an old boss of mine who wondered why we had to spend so much effort on securing our systems. I pointed out to him that we didn't used to be connected to the internet.
"Before the cloud was trendy", small to medium businesses didn't have millions of customers using their online systems.
> I worked at a multi-billion $ subsidiary of a global Fortune 500 company that tried, as part of its move off mainframes, to "go its own way", with their own hardware in Equinix data centers. The entire executive level of the IT team responsible for that effort - about five or six people - were eventually fired, along with much acrimony, because they couldn't pull it off.
> ...
> And yes, "shit, EVERYTHING is down today" is a consequence of all this, but the alternative is generally much worse for individual companies.
IIRC, AWS and cloud infrastructure didn't get big until the mid-2010s. Fortune 500 companies (and everyone else) were able to "go their own way" with their own hardware before that, they had to. That was just ten years ago.
If "[going your] own way", with [your] own hardware" is too hard to do now, I'd guess that's not because it's impossible, but because there's been a combination of skill/capability loss (similar to what's been seeing in manufacturing) and places like AWS sucking up much of the remaining talent.
And now add vibe slop onto that, in both the dependencies and the implementations and the infrastructure management, and you really have a recipe for cascading failures from which it’s difficult to recover.
I’d like to add that the perpetual gold rush during this period has created a culture of self-serving that few other professions can compete with - basically institutionalised low-level corruption.
Decisions are made on the basis of trend chasing, what’s fun, and what’s good for the devs’ resume. And it’s completely accepted, hence the world is overflowing with Rube-Goldberg machines that basically defrauded whoever paid for it. Try doing that as a carpenter and see how long you’d stay in business.
The PHP2Shell bug existed more than 15 years ago, spanning releases as early as 5.x. There was no AI coding back then. So it's very hard to blame AI. There was a coding mistake, and AI found it in 2026. Probably a good thing before a malicious party does the same.
OSS definitely flourished way before 10-15 years ago. I'm not sure what the author suggests as "knowing the details". Should we all rebuild our own web scripting language to know "all the details": e.g. PHP? Probably not practical. A line has to be drawn on where your knowledge and expertise stop. THe line can be vague and consistently changing based on needs. But there will always be a line. We will always stand on shoulders of of our predecessors one way or another
I understand the author's trying to say the overall situation is unstable and trending negative but I am not sure the writing's logical flow is holding up very well.
A better way to put it is we've written a lot of code, and a lot of it had bugs hiding in plain sight. AI made the whole bill come due at once. And will this AI code explosion create another bill we won't know about until 20 years later? Who knows.
I don't think the point was to blame AI for PHP2Shell. It is that we've built up gigantic systems on shaky foundations: "a highly complex system made of low quality parts, where a single error can compromise everything"
Of course the shaky foundations were there before AI. And before the widespread proliferation of NPM packages etc... But those increase the ease and pace at which we can build on top of the shaky foundations, they increase the rate of so-called decoherence. And there are economic and organizational incentives to keep increasing that ease and pace. That makes it more and more likely that a problem in one of the shaky foundations (whether AI-created or not) causes a system-wide failure, and at the same time results in a more and more generalist tech workforce, making it harder and harder for an organization to respond appropriately when that happens:
> I believe that tech companies risk losing their “organisational cognition” - their ability to think and respond as a coherent unit.
> This is the real threat: a scenario where nothing in the system is done with intent, and nobody in the system has control. We generate complexity faster than we can tame it.
The major problem IMHO is that AI massively reduces the required skill to exploit. PHP2Shell? Even the best minds working on PHP were not able to come up with a way to exploit the vulnerability, which is why it was shelved aside in the first place.
But now? I member someone on Twitter claiming a reproduction cost them less than a dollar worth of tokens.
That isn't just tech debt coming due, it's a bunch of mobsters armed with the equivalent of gatling guns. But unlike powerful guns that governments can somewhat regulate, there just is no way to get the AI genie back into the bottle. People run abliterated models on their own hardware now, the equivalent of 3d printed guns, just so much more powerful.
AWS is 20 years old, before that people did rent machines from places like Rackspace. Maven Central (Java) is 25 years old, RubyGems is 22 years old, CPAN (Perl) is 31.
I find the arguments about decoherence and the sort of snowball-of-complexity more compelling than this timeline.
The first widely used cloud computing platforms probably were the PHP/MySQL hosting services of the late 90s. FTP your PHP, done. No ssh, nothing. Only your FTP folder with the files you uploaded.
I get the impression that when the author says "open source" they mean "rampant use of dependencies". And while the terminology is confusing and the time frame maybe not quite right, there certainly has been a shift where many projects went from depending on none or a few big reputable open source project (MySQL, Apache, Linux, etc.) into depending on hundreds of language packages made by random people.
This article is pure nonsense. Yes there may be more and more failures that are getting more difficult to catch and remedy in time but that is the nature of the kind of systems that we build nowadays and that is distributed systems at massive scale. When an error occurs in such systems the effect may show up in a far different component or part of the system and in multiple places effectively making it difficult to troubleshoot.
AI is great at following patterns. AI is great at writing tests. AI is great at doing work that humans would consider drudgery and then applying "enough" judgement to it.
The problem is that most software teams and most engineers are very, very bad at the thing that AI needs to succeed: technical writing. Documenting the patterns, the rules, the heuristics, the SOP, the judgement; developers could barely be bothered to write comments before (which in the AI age are free, infrastructure-less memory that is always "line-of-sight" for the agent and can be used to enrich with keywords).
It's not that AI is the problem, it's that we humans are good at resolving these constraints in the real world at a human pace. AI makes that pace impossible for a human to keep up as a trailing activity. Rather teams need to invest more in technical writing and producing the corpus that AI relies on to learn the operational constraints and rules.
I'm actually quite good at building robust systems, cleaning up messes, documenting things well.
As a result I get:
1) The enmity of my lead dev who prefers to do things more fast and loose.
2) Not promoted, due to the first reason and due to the fact that I don't have "impactful" metrics.
I'm not saying I'm perfect, there are definitely things I could have done better. But the fact still remains that the industry grossly undervalues maintenance and optimization that doesn't lead to easily measurable numbers that impact the bottom line.
That in turn is because it's hard to tell if SWEs are actually paying down critical debt or just wasting time. But that's the core problem, not that we're not good at it.
The good news is I can get AI agents to do this kind of work so I can now chase after metrics.
I feel similarly: I am slow but solid. The industry values those who move fast, break things, and make a mess.
Which makes sense: those in a position to promote others generally don't understand the technical part. What they see is how fast they get a new toy to play with. And if it breaks too soon, they will still be able to blame the developers.
I think this is a real alignment issue - nobody has excess time/money to make the software robust/secure.
This is possibly due to the high level of economic inequality, meaning less resources in the middle of the economy.
In a decade I think we will have few people who can competently write or even read computer code without AI assistance.. and I think we might say the same about math and science and medicine.
AI assisted learning could lead to a renaissance of human capability .. but thats not where we are headed, we are outsourcing all the hard thinking skills. In future, only a few crazy retro nerds will learn assembler or multivariable calculus, just as only a few crazy nerds now make mechanical watches as art objects.
University means knowledge work, and knowledge work is already getting fucked over by AI. The clankers can now design rudimentary PCBs. The only university degrees that will not be compromised by AI are the ones that deal with humans in the meatspace - think medicine, psychology or education, although I think we might be getting a revisit of MOOCs augmented by AI to drastically reduce the need for teachers.
Being good at one of the trades however? That's a pretty safe bet. It will be quite the time until physical robot "bodies" are capable of the dexterity, size and endurance of a human. And by the time robots get that capability, we're either living in a Star Trek utopia or in Matrix style neo-feudalism.
While you do that, look for opportunities in niches where "fast and loose" is unacceptable, because while they might not market themselves as loudly as most software development organisations they do actually exist.
Typically these places are associated with compliance, audits and regulation with teeth. It might be adjacent to lawyers, in public institutions or weird consultancies. At least that's where I've found such work. Insurance and banking might be decent bets, and they sometimes also come with oddball tech stacks, perhaps mainframes, perhaps Smalltalk.
I absolutely agree, as a child of automotive industry I am looking on software development through the lenses of kaizen loop. If you can loop the development cycle so it can correct itself through tests, then errors, either human or AI made are never going to be a problem, they will get self corrected by the testing.
Where do you get the correct test results? First from the specifications, then gradually from the edge cases found by testers and eventually users. It is not one shot process it is a big loop which will get repeated over and over again until users will run out of things to break.
The software is breaking because nobody is bothered to write down what is software actually supposed to do, and from there to create a set of tests which will anchor the behavior of software.
Nobody bothers to write down what it's supposed to do because nobody thought about it enough for a full spec to be conceivable, let alone written. In a lot of modern development, you have a committee of non-technical product people who might, at best, own a single flow, but they sure don't own, and don't care, about the interactions across flows. And then you have development teams that live and die by KPI, and thus draw the edge of their responsibilities in the narrowest form possible. I was told by a subscription team in a certain very large company that they just own the subscription platform, not anything about the data in it, or the relationships mapped in the data. Bundling? Data. Billing? data. They are just 40+ people whose responsibility is to keep a database up, and therefore no outcomes other than inability to retrieve the data was ever their problem.
So who owns outcomes? Usability across the board? General customer experience? Nobody. The actual software that people pay for, is orphaned, and anyone trying to track down problems is just asking to be blamed for them. Thus, most of what happens is undefined behavior, like if we were writing an old C standard. Having someone that has the responsibility to make the product as a whole do something, and thus maybe be able to write the spec eventually, would already be an improvement.
> The problem is that most software teams and most engineers are very, very bad at the thing that AI needs to succeed: technical writing.
Software engineers were at least pretty good at technical writing using the artificial languages we created to enable clear, unambiguous technical writing... programming languages. Seems like software engineers might do better using AI if they remembered this and started using pseudo-code as their language for spec writing (as many have recommended).
The urge to one-shot everything isn't helping. Getting people to push back on assumptions the model is making, and to have an actual conversation with it about the solution, would lead to more maintainable output.
Maybe part of it is laziness but for sure part of is low morale and hopelessness.
Part of it is the hype cycle too. Everywhere you look online, all the easy examples are people one-shotting this and that. Strangely it seems like the 'standard' way for people on YT to 'test' a model is to just one-shot some random game idea. It's meaningless, but it creates this unconscious expectation (even amongst experienced devs who might not be as experienced in AI use) that the models _should_ be able to derive exactly what's needed from a one-shot.
That's fundamentally impossible though -- if the information isn't there, the model is just going to move toward the average and fill in the blanks.
On another note, I'm also not sure where I stand yet on the thing I see where software devs end up falling down the rabbit hole of building little mini software factories, with agents reviewing agents, coordinating agents and having agents cosplay human roles -- essentially 'orchestration porn'. I'm yet to dive into that rabbit hole (I likely will at some point, so I'm not judging), but from the outside it looks like you end up worsening your understanding of the system, while spending tokens unnecessarily and creating a closed-loop mini model collapse.
I've actually run into the situation at work where I've asked people whether the libraries they've made can do a particular thing and sometimes the answer has been "I don't know" (I'm talking just _features_, not even low-level architecture). At that point, something has gone wrong.
I'm starting to get really interested in "Okay, if human code review is fundamentally impossible given the volume of code, what can we do?". We have a tool that can essentially build tools for almost free, so I'm trying to approach it from that angle. Let's see what sticks.
> it creates this unconscious expectation (even amongst experienced devs who might not be as experienced in AI use) that the models _should_ be able to derive exactly what's needed from a one-shot.
It also creates an expectation that anything done with AI as part of the process was done in one shot, and therefore the person had no meaningful impact on the output.
I saw a Reddit post of someone asking essentially “I’ve spent the last 7-8 months working with Claude on a game, am I really making something?” and I just wanted to reply “of course you f****ng are, don’t let people tell you otherwise!”
For this reason, I keep a "how it started / how it's going" log for everything I work with, to show how it looked after the first prompt, versus how it ended up after extensive work. Just to make clear how my input actually mattered to the output.
I think the correct way to solve this problem is completely at odds with AI.
Write less code. Reuse things that have already been written and thus were already reviewed and tested by someone else. Do not reinvent the wheel. KISS.
With AI everyone is implemented a dozen libraries from scratch in their project instead of using something someone else made. Why use a library for 2D geometry when ChatGPT can write one in 10 minutes?
I have to think part of this is model training, encouraging the model to code-up something new instead of looking for library code that can fit in the project. One of the things I ask Claude repeatedly is something like "can Spring do this already, or is someone working on this already?"
That just follows from the fact that people can learn. Human organizations always have to balance between tacit knowledge and explicit documentation. Document too little, and onboarding new people will be more difficult, and you will lose information when people quit. Document too much, and you are just wasting time without doing anything productive, because people would have figured out that extra information in less time than it took to write it down.
The fundamental problem is that current AI tools don't learn. They make the same mistakes over and over again, unless you give them explicit instructions to avoid those mistakes. Maybe we need to accept that all work is now done by new hires, who have read all the documentation but have no experience in anything. Or maybe AI tools will get better, and all that tedious documentation effort was just a waste of time.
Public documentation can help to mitigate the issue, but only with information that can be released to the general public, and only if it can be easily put into words. But as long as there is a separation between training and inference, the divergence between what the model knows and what it should know only grows wider over time.
I've been one of the organizers of a relatively large RPG convention for ~25 years. If I had to write down all I know about that convention, it would easily take a year or two. Because most of that knowledge is not in a form that can be easily expressed in words. And in my day job as a scientist, a similar task would easily take 5–10 years.
> The fundamental problem is that current AI tools don't learn
They don't learn, but they are very, very good at searching. That is why writing things down is important, but so is getting that into some sort of agent accessible index.
The problem is not the input (documentation) it's the evaluation of the output. Managing an AI is like managing a team of specialists, whose subjects you are not a master of. Most programmers did not have the problem of managing experts - we were the experts. There are those that do, but several of the levers you have over a team of humans (incentives, responsibility, even hiring and firing) are not available to control an AI, and the new levers ( fine tuning, training, reinforcement learning) are not in the hands of those relying on it. This is not solved by any amount of documentation.
You hand the same documents over to AI agents that evaluate the output.
The same rules that you, an expert would apply, should be documented and used by an agent to apply those heuristics.
There are obviously some areas where there's spot judgement, deviations, and exceptions. But by and large, you have some heuristic that you use when evaluating output. Document it and give it to an agent.
You're asking ICs to become managers of a bunch of mid-level developers (agents). It's a skill that has to be developed like any other, and takes time.
It does have to be this way, unfortunately, because AI bros will press to have as much software produced at the absolute minimal cost possible.
I've seen it first hand. 70 people fired, 20 people left. The 20 that are left take on all the work that was done before, plus some extra, because every company now has a new AI product, obviously. AI agents make us so productive. Every review is a 30 second LGTM on 500 new lines of code. Confluence filling up with AI slop.
A few weeks back I saw, first hand, a DevOps lead getting ready to ship a big update to performance-critical product where Opus 5.5 suggested they completely turn off JIT everywhere because they were running into a HotSpot C2 optimizer bug that would make the first request after a restart fail with a 500. Actual fix was a one-line adjustment somewhere in a library.
People are scared of being replaced, so they need to appear as productive as possible.
You can be productive and guide AI to operate in a more compliant way: by writing down basically everything you know, every heuristic you have, documenting the correct patterns, documenting the framework level approach, explicitly spelling out what's good code, writing down detailed specs for an agent to work on, writing the test plan, etc.
A lot of devs still think their job is to produce code; it's actually to produce all of the guidance and structural scaffolding for the agents ("the harness") to guide its behavior and output.
A concrete example: I wrote a small library[0] (MIT, OSS) that improves how agents write unit and integration tests. This isn't product code, but this makes the agent better at writing the tests it uses to verify its work. (Tests become denser, more semantic, less context, type-checked setup, etc.). You can't expect an agent to do this by itself; the human needs to produce the scaffolding.
agree- this is how i name variables, this is how we handle third party APIs, this is how we handle caching and here are some heuristics for when we want to cache and when we want to avoid it, etc. etc. all coded up in a nice documentation tree with links in the agents.md that say "when integrating with third party APIs, read docs/integrating-with-third-party-apis.md".
i have written alot of this type of documentation as an architect for human teams, and i find it works for agents as well.
> A lot of devs still think their job is to produce code; it's actually to produce all of the guidance and structural scaffolding for the agents ("the harness") to guide its behavior and output.
This basic reasoning is wrong and leads to “orchestration porn”. Thinking that you can laid down rules to automate software production like it’s a factory pumping out products.
What you forget is that software are grown iteratively, not guided down some assembly line. Every patch is a revision of what the software should do, not the next step of a workflow.
So stuff like patterns, architecture, tests,… are helpers for communicating the purpose of the software. They’re not really relevant to the software itself as the computer would be happy with just a single file.
You have a software that’s needed to fulfill some needs, you have a team of people where each brings expertise to build it. Workflows are built so that everyone how to contribute, what they can depends on, and what others need. When there’s a failure, we add the missing information. When there’s need to do some changes, we inform everyone impacted.
Everything is information and communication related. Information about the system and communication between everyone that interacts with the latter.
So devs do not think their job is to produce code. It is to take requirements written in ambiguous natural language, sometimes incomplete and contradictory, and to design a solution that is formally written while taking care that all statements are coherent.
Making things coherent (aka correct) is what takes the majority of our time. The other is gathering requirements. The actual typing of code is pretty much instinctive, just like you don’t check the orthography of words like “concrete” or “guide”, and rely on the spellchecker for more difficult one.
The “orchestration porn” part is that there’s no mention anywhere on how you think correctness and be improved or even maintained as part of your tool.
I suspect the majority of reviews had been a rubber stamped LGTM. Often the minority with substantive feedback were heeded and changes made; I'm not sure that is happening anymore.
You have to stop thinking this way. It's our job to do what our management wants us to do, which is often to produce as much software at the lowest cost possible.
There are legitimate criticisms about AI use and how AI has changed the profession, but you have to learn which battles are worth fighting.
The refusal to take responsibility for the consequences of one's work is one part of a vast complex of machines that normalize the horrific. We have met the shoggoths and they are us.
> It's our job to do what our management wants us to do, which is often to produce as much software at the lowest cost possible.
Have you ever been in a situation where you did what the management wanted you to do for years, regularly warning them that you could not write good code while doing what they wanted but being dismissed because they knew better? And have you ever been in a situation where you finally got your "I told you so" moment after years, when they actually realise it won't work?
I have. You know what happens then? It's not their fault. Best case, they explain why you couldn't have known better because it was hard. Worst case, they fire you because... well you built that mess, remember?
What's the winning strategy for a dev in that situation? IMO it's adversarial: do whatever works for you: work less and pretend it's hard, or chase a better salary, don't get attached, and prepare to jump ship whenever needed.
Doing what management wants you to do blindly is an easy path to ruining the company and losing your job. We used to joke that the true mark of a senior engineer is being able to tell management when their ideas are stupid, will not work, and "no."
Just like crypto bros a few years back, where we "do what our management wants us to do", built stupid-ass blockchain into every every which app that didn't need it, and when it doesn't sell get blamed for it.
Personally, I was asking "but what's the use case" in every meeting. "Why would somebody pay for this". "Who's the customer". Right now the vibes are quite similar to back then.
Dude, you just didn't write down the rules and the patterns effectively. Just write down "don't completely turn off JIT" /s
Damn if only there was a way to write strict rules and patterns in the english language that could be converted into strict instructions for the computers and ways to verify those instructions...
SOP and documentation is everything. You don’t want it going out of bounds and coming up with its own solutions, which are often just inefficient facsimiles of random stuff it finds that seems to makes sense. You also run the risk of not understanding the solution it puts in front of you. With an ironclad SOP and quality documentation, you know when it is straying from the best paths, because they are laid out for both of you.
Plus your documentation can often work very well alongside whatever skills you have created over time, because you basically have two sources it is drawing on telling it to do the same thing. I find it leads to less off book behavior.
I’ve always been perplexed by how people see AI as this creative tool or something to innovate with. That’s not how I see it at all. I see it as a device that can follow a template very well if you lay good tracks. They can archive and check hundreds of pieces of media way faster than I can. But I would never trust it to craft a story
Well after five years of miserably keeping our SOP tight and templatizing (is that a word?) our edits aggressively, we are finally seeing some dividends with resolve’s recent Claude integration. It was a painful process but the amount of time we are now saving is well worth it. Also takes a lot of the drudgery out of editing. Anyone who has been a DIT before knows what I mean lol it’s nice to sit down to a prepared timeline that is already roughed in with multicam clips and effects all applied.
"AI is great at following patterns", sure but if some attackers can "inject attack patterns" at a high rate in your training set (check "dataset poisoning") then it's going to be an issue...
Software is not really being more broken today than it was 10 or 20 years ago. It actually has gotten better.
I've seen the codebase of a dozen companies and worked with hundreds of peers.
There is plenty of horrific bugs and bad software outside. The reasons are plentiful too and include everything like budget constraints, management failures and engineering skill issues.
Today, the average code that even a product manager or junior engineer submits is actually excellent overall - if your processes, harness and CI etc are well designed. It is actually super easy to author software well now. Whether you build something that is actually needed and is tasteful is another matter.
99% of the security vulnerabilities being found and exploited today were created by people. Cutting most of them out of the technical details is actually good.
Even if individual code is of higher quality, if there are more pieces of code talking to each other, there are many, many more failure modes because software often fails at its interfaces. Nancy Leveson from MIT has written well on this subject.
Perhaps a bad example, but isolated code will often fail less than networked code that has to interact with a bunch of stuff. An old “dumb” tv never seems to have software issues, my new “smart” tv consistently has issues. It’s like having a dozen people write a paragraph each or a single person write a paper. Even if the single persons writing isn’t the better than everyone else’s, you’ll often get a more coherent paper.
Imo: the way we make software is breaking down at a systemic level.
The harsh reality is that some people/systems are adapting just fine to the new way of doing things, others are making a mess of things by rushing via vibe coding and not checking the work (either by choice or by circumstance at work). The results won't be (and shouldn't be expected to be) linear.
The 2010s brought on a very groupthinky dynamic to software dev. This led to a culture where the majority of developers just ask "what is everyone else doing." Now that we're past the blog era of development, that shared reality appears to be breaking down. Now the answer really is "whatever makes sense for/to you," but I think (anecdotally) that's freaking a lot of people and teams out.
For the first time in a long time, a "best practice" doesn't really exist.
Your last sentence nails it from my perspective and experience. I run a software dev agency, and for the last like 10 years, us and all of our competitors have pretty much done the same thing. Followed best practices, approached software dev from a "normal" viewpoint. We competed on price and expertise.
Now, almost all of our competitors and us have taken a fundamentally different approach to how we use the AI tools for our work, and it's been really fascinating and fun to compare and contrast with them. We're finally different, and to me that's a great thing.
I'm sure approaches will converge into "best practices" again but for now I'm really enjoying that we don't all have to be the same.
> This changed the balance of power between workers and management.
I think this is the core insight that I keep coming back to: technological advances always entail a transfer of political power.
We would all be better off, when evaluating new technologies and listening to their evangelists, thinking hard about who gains power and who loses power as a result of adopting that technology.
No, we'd be worse off. It's almost impossible to vet technology and only adopt the good. People are already extremely biased toward the status quo; the last thing we need is one more reason to avoid adopting new technologies
That's not necessarily the moral of the story though.
If we hadn't spent the last decade circle jerking about how we're all 10x devs and everyone else is a pleb there's a chance we could have built unions for ourselves.
That would mean we'd be able to actually wield political power collectively and establish standards for quality and technology adoption like all the other critical industries have.
We don't need to say "we're not adopting tech", we need to say "we're going to follow our process for adopting tech" which may be slightly slower, but the banks would still be online today. Further, it doesn't even necessarily need to be process, it just needs to be something we have a say in instead of a hand full of the craziest narcissists to ever walk the earth.
Unions that have "wielded power collectively to establish standards" are generally bad, and have mostly made the world worse. That's the last thing software needs.
Software wasn't exactly hard 12 years ago either, the barriers to entry were being lowered at a rapid pace. 2014 was the dawn of the software bootcamp where JS became more and more pronounced in web architecture.
I feel this piece is either more alarmist than it deserves to be, or it is late to the party.
It uses various incidents in 2026 to prove a point while dismissing those that did take place in the decade or so previous. For example, heartbleed, the log4j RCE, Rails' mass assignment DOS, Cloudflare updating a WAF rule and breaking the internet, Atlassian deleting almost 1000 of its customers' accounts and data, the Ashley Madison breach, Crowdstrike bricking their customers computers, Okta getting owned multiple times, etc.
My own point is that software became like this when we started becoming pragmatic. Nothing to do with AI or management really, we just realised, especially with web tech, that you can release something and then fix it. In the old age of getting CDs or floppies printed and distributed, that was not even remotely feasible. A similar incident in the 90s would involve a product recall or a way to get updated media from a retailer with the fix. Very expensive.
Standards have dropped and continue to do so, because we can also fix fast. It's not suddenly breaking down. If anything, it's just too centralised.
Software has always been broken at a systemic level. It’s a complicated continuously changing web of abstractions and abstraction breaking hacks attempting to survive prolonged contact with reality.
A decade ago there was a huge culture around having 100% uptime, nowadays it seems that not only is lower uptime acceptable, but services need to have some downtime and break when they are not profitable. Filtering out the undesirable users and upgrading them to plans with better economics is a new requirement in this age that wasn't present before during the growth stage and freemium VC backed startups
It was less broken before open source allowed an architectural explosion, because the dependencies were too expensive. It took a lot of arguing to get management to splurge on a single library or other tool. A lot of astronaut-architect wannabes were constantly grumpy about mgmt that wouldn't pay for the latest whiz-bang thingie.
The actual programming logic has certainly always been complex, but the integrations weren't so bad. You could learn your very small toolset quickly and focus on ideas. Now, you have an ultra-massive toolchain to come to terms with, made up of free software and "cloud" architecture that starts off cheap and gets expensive later on.
Dependencies have always been the problem, but that problem has expanded in every direction, at every layer, approaching our own limits of comprehension. AI has allowed us to catch up, temporarily, but given the collective penchant for upping the ante, I expect the real supernova is yet to come.
Is the problem open source? I haven't see these kinds of widespread problems coming from Java dependencies (25 years old) or Perl (maybe 30 years old)? Okay, maybe people aren't using Perl like they used to but Java is still a thing.
Part of it is the inclination of the developer to make the thing more robust. Part of it is lowering the barrier of entry, maybe to the point it's _too_ easy to submit a package. Part of it is bad decisions made by the package manager, like running arbitrary scripts.
I don’t doubt you would find a trove of issues in CPAN if you went looking. It’s not an interesting target, though, because it’s all but dead.
Java via maven central is similar. It’s not a popular language. We know those libraries are riddled with issues - log4j’a big vulnerability isn’t that old - and simply requiring a GPG signature on a package is a thin guard against compromised uploads.
The NPM ecosystem is uniquely suited to supply chain attacks though. It has a massive rate of churn, which makes it impossible to maintain dependencies safely. You either lock in and hope there’s no big issues or ride the wave and hope you don’t get hit by zero days.
I agree that CPAN is not as active as it once was. Maven is hardly dead, it's pretty active (as is Java the language). NPM is very busy but they also have a very real bot problem.
Yeah, I also question if this has anything to do with open source. Open source solutions have been around for decades, heck React has been open source for 13 years.
AI has allowed us to scale development effort massively - and shows the same weaknesses as scaling human teams too quickly - planning, verification/testing and code fragmentation become issues almost immediately.
Well, yes, but the proportion of systemic level broken stuffs might have change a bit. Just like, ok, there was always rule34 lurking to make dirty everything innocent that remained of the untainted child memory one had.
Now it’s somehow guarantied that the volume of dirtied quickly prompted version of anything anyone ever cherished far exceed the scarce outcomes of the original.
I do embedded. I’ve always marveled at how ludicrous that “real software” is. I have truths. I have registers that will never be or act differently, I can make pretty damn good assumptions that no one will put this code somewhere else’s etc.
In the past few years though, it’s been creeping into my domain too.
>I have registers that will never be or act differently
Even this needs a qualifier. Radiation exposure can cause bit flips. Certain aerospace applications must address this, and even some earth-bound applications at high altitude.
Sure, but RAM isn’t the only place a bit can flip. Registers are a thing. IBM had error correction in Power architecture CPU registers, not sure where AMD, Intel, or Apple (lol) are on that issue. IIRC register bit flips first came up with Sun SPARC hardware, which Sun mitigated, but of course SPARC isn’t used too much these days…
A quick AI-assisted search informs me that neither AMD nor Intel document any on-CPU error correction, though perhaps there is some. Apple currently has zero systems that even use ECC, which seems insane to me given they’re shipping systems with up to 512 GB of RAM.
This seems to be an issue that’s falling through the cracks again. I guess nothing will be done until there’s a critical failure due to a bit flip, and verifying that was the cause for a given problem is difficult.
On ECC, Apple needs to Think Different (tm). Maybe it should introduce actual “Pro” machines…
Besides, chip bugs are famous since the Pentium, and unreliable circuits are the normal way to describe circuits, you have to add notice when you get a reliable one.
That's why we do requirement gathering and analysis. Engineering is always about tradeoffs and things that matters in certain context don't in others. The issue is that in a lot of orgs, while there's a record of decisions, we don't usually track assumptions and the overall system is full of contradictions. Also there's no overall keeper of knowledge (either one person or a team) that periodically ensure that contradictions are kept at minimum.
I don't get this thing where people are allergic to the idea that change can happen.
No matter what's trending worse there's always this thought terminating phantom law of conservation of everything. The fact that something is not an entirely new bespoke phenomenon always turns into denial that it could be happening more often or in a newly qualitatively worse way.
This feels a little hyperbolic, the grounding was a result of a cable swap and the ssh cert incident they cite was so human caused it’s not even funny. I think things are a little more shaky, but not massively so. I had the town I was living in during college lose its internet and phone lines due to a cable being cut on a mountain pass and it was fine. If your business is sensitive to an email outage, prioritize high reliability over integration advantages that outlook offers. The php2shell vuln is bad, but we also had log4j years ago.
I’d like to analogize the situation with the Kessler syndrome: the increasing density of satellites in low Earth orbit increases the probability that a cascading failure occurs, which would deny us access to LEO for centuries. The inciting incident could be intentional or could be accidental, but the result is the same.
The analogous situation is brewing. The depth and complexity of our computational infrastructure is increasing, and the impact of an incident is less and less likely to be contained solely to the affected system. Eg, a router replacement goes awry, and the entirety of the world’s aviation industry is broken for weeks.
Intentionality isn’t necessary: this is a systemic problem.
When the official cited story (put forth by a known boot-licker) is contested by people on the front lines, I would defer my conclusions until more is known.
There are some decent points, but this is collating a lot of separate incidents with wildly different causes that just happened to occur in a close timeframe.
I don't even know if there are decent points. This flails around in a way that's just...not super useful to spend the time trying to digest.
Some things happened. AI is also happening, therefore AI is to blame (in the way that some people really believe a problem isn't a problem unless you test for it) and here's a lesson about organizational coherence or something.
These essays are the real AI slop of 2026. Not authored by AI, but always trying to get attention by tangentially being related to AI and getting the anxious upvotes of the "can I turn back the clock?" crowd. And there's a humorous commonality to all of these pieces that they all start with some sort of "don't think I'm not with it. I'm with it! I hand slung an LLM off the shoulder of Orion. I watched C-beams glitter in the dark near the Tannhäuser Gate." call to authority, then pivoting to the "everything has gone to shit" spiel.
We've always had a tenuously connected, barely operated network of systems. We've always had bugs and engineers doing shitty things. Every organization goes through periods of upheaval where all of the institutional knowledge is invalidated or discarded regardless.
Most folks here didn't even know, but a Bitrise security outage hobbled our entire CI this weekend. Friday to Sunday night.
The security issue motivated all Bitrise customers rotating their secrets. this was a big deal for mobile deployments but didn't get much attention on HN..
Imho, the problem with AI is that management thinks that engineers can now do 10X (or do one item of work in 1/x)… and so there’s so much pressure to get things done. That’s how (tons of) bad software gets out in production
Imagine using AI but still producing the same amount of output/outcome. That would be delightful, great software could be possible… but it’s not “productive”
IMO control has been lost because we decided that moving fast is the most important thing and outsourcing everything that is not directly related to business (sometimes even business itself...) is a great idea.
The seed was there since at least a decade, AI just accelerated everything.
This is the usual reaction when people run into an uncomfortable truth. We're all bad at objectivity - we preferentially look for evidence that supports the outcome we want (or the one that's least uncomfortable).
In this case it's pretty obvious that AI is better at coding than humans. Lower failure rate per task is better code. We're just shipping more of it.
Nobody's claiming AI code is bug-free. Just that it's better and faster than how we used to do it.
> Just that it's better and faster than how we used to do it.
I think only one of those is true. It takes a lot of effort to get quality architecture and not-function-soup code (at least in Python) from LLMs. I can't imagine the pile of crap code bases these software factory type solutions are producing. Well, you can see a lot of them on GitHub I guess, so no need to imagine.
Further, one of the negative impacts I've seen from being able to go a lot of faster is that it translates to product teams being able to ship faster which is not always better. We've lost the "back pressure" we used to have in software development which forced product teams to consider, then consider again, then consider again as each new sprint requires reevaluating priorities. This resulted in a lot of requests stalling out and never being built that should have never been suggested in the first place. Or, to be more charitable, the delay gave the product team time to get over the initial excitement of a new idea and really consider if it was warranted.
Now we are building all-the-things and I think that's only better in some cases. I think it's proving worse in most other cases. Especially when you consider that every line of code that ships has liability attached to it.
That results in large bloated mess projects that no human can reasonably understand. That means we are dependent on LLMs to understand it and, the larger the code base, the more LLM tokens are required and the worse the LLMs are at understanding all the relevant parts and making wise architecture decisions. This creates a downward spiral that is harder and harder to manage and means features take longer and longer to ship (although, still maybe less time than pre-LLM dev...we'll see).
Do we want “faster” development? Like I couldn’t care less if my bank app or netflix didn’t ship a single new feature in the next 12 months. Users don’t need constant stream of features. But faster development is “better”. Progress and all they say. Like yeah, why I would like a car if my horse is just fine
I agree with the premise that things are breaking down, but I'm not sold yet that the problem is a systemic one.
At least some of these issues are ones that already existed, but are now being discovered by AI.
If that is the case, we would not see the exponential increase that the article mentions, but should instead see something like a temporary blip where discoveries in existing code spike, but eventually all get discovered and fixed.
I would love to see some more data that specifically looks into the number of issues caused by pre-ai code versus post-ai.
That being said, the comments about decoherence strongly resonate with me. Things are moving so fast in so many different directions right now. I don't think we know the right way to build things in this new future yet.
There have been parts of the software industry that have been mission critical from day one and have built processes, languages, and tools to make sure things are 100% reliable. Building software that way is slower and more expensive and in the past has been an overkill for most systems.
The problem software engineers are facing is that now that software is ubiquitous, almost everything needs to be 100% reliable (or close to it). This is coming into conflict with the move fast and break stuff everything needs to be done this quarter at minimal cost mentality.
I’ve had AI running to auto upgrade our infrastructure and it’s successful twice as much as humans. Failure rate of a human managed upgrade is 1/10, failure rate of an AI agent is 1/20 in my small set data.
Problem is I’ve had 5x more incidents. AI is operating at 10x the speed of a human so even though it’s failing less, it’s causing more incidents per month.
I wonder if people are ready for this kind of paradox where using AI may be safer and faster but less reliable for “human time”.
It feels like we are currently experiencing a soft version of the sewing machine riot.
Perhaps no one is intentionally destroying anything in this case, but malicious compliance and simple negligence can go a long way when it comes to disruption of complex systems. Destroying a sewing machine requires actual violence. An air traffic control system is an abstract concept. You don't need to touch it to break it.
Things are breaking down because they were weak to begin with, and demolition is needed in order to rebuild better, stronger. This is just a period of trial by fire: iron has to be heated and beaten numerous times (with carbon) before it becomes steel. All that's happening is AI is accelerating the evolution of industries into something which all eventually have the strength of steel.
My impression is it's always been a mixed bag. The level of bugginess has been relatively constant. In the meantime, scope and capabilities have gone way up. I hate Teams and Outlook and the virus scanner on my company's laptop though. Those are keeping me from doing my work. Windows + anti-virus makes file access and process spawning 10x to 100x slower compared to a Linux machine.
I think the article has a point that the if you have a very complicated legacy system, then if you just let an LLM run on this, there is a significant chance that some parts of the system will be become inconsistent with each other, and the whole thing will be more of a mess and more likely to break.
In my view, however, what is more likely to happen soon is usage of LLM to transform various parts of code into the form with much more clearly defined interfaces/rules/boundaries that can be essentially proven to work correctly. That in turn (in my view) will enable much more robust systems.
To talk abstractly about all software, you'd also need to analyze all the software that stays working. This article is just cherry picking a few recent incidents and try's to make some conclusions without really looking at causes and whether that can be generalized to "Software".
Thanks to the author. It's a good read and interesting HN comments. Nothing wrong with having concerns, and I'm sure many people share those concerns. It's scary sometimes. But, the view of the author is somewhat on the negative side. I feel that AI development is still in early phases and people or companies are discovering what works and what doesn't. AI empowers us in new ways with new problems, and new trade-offs. The old way of doing things had different hurdles. Interesting times. It's not going anyway any time soon I expect ;)
There's a lot of writing going around that we are all headed for some catastrophe, like everything has been building to this grand moment where society just crumbles and AI is the final piece.
I highly doubt this is going to happen, guys. The world has changed and software is never going to be the same. The same battle against complexity and hard things will continue to happen, but the front has moved.
Open source as we know it is probably dead. Tech companies that would create SaaS for X and get billion dollar valuations are also likely dead. DevOps is dead. Other stuff will take all these things places in time but it's going to take probably a decade. Get used to it and let's get on with it.
I agree. In my life time there hasn't been a minute of software reliability. There's nothing special about this year or even this month.
I remember the first time I installed Linux in the early 2000s and was and owned within 5 minutes of putting it on the Internet. Desktop computers of any OS crashed regularly. Servers crashed and needed total rebuilds.
If anything software is way more reliable than it appears because the scale of computers and software is thousands of times larger than it was 20 years ago. More things can fail because there are just more things.
I've been using Linux since 1997, and I've never seen a Linux box being "owned within 5 minutes of putting it on the Internet". These things simply didn't happen in the early 2000s (it did happen with Windows 95/98/XP, though). The Linux kernel and Linux distributions had matured enough already at that time and surely didn't crash regularly, neither as desktop OS nor as server.
Things were less convenient and generally harder to set up back in the days (we had to fight with XFree86 configurations, desktop environments were immature, a lot of hardware didn't work due to lack of driver support, application software for many tasks were unavailable, there were barely any commercial games), but the kernel had excellent network support and scaled very well across multiple CPUs already (unlike Windows NT's SMP support, let alone the single-CPU-only DOS ancestors). During the 2000s, the Top 500 list of supercomputers was slowly being taken over by Linux clusters, and that wouldn't have happened if Linux would have been unstable. For developers, Linux was (and arguably still is) heaven on earth, and developers actually made sure that "their" system worked well for them. And it did.
There were many difficulties with software in the past, but Linux was rarely one of them.
> I've been using Linux since 1997, and I've never seen a Linux box being "owned within 5 minutes of putting it on the Internet".
Absolutely happened.
But then I went on to do Linux hosting and co-location for the next decade.
I don't see why you felt the need to respond. Linux is almost certainly more reliable and secure now than it has been historically. But software continues to grow and get new features, so there will always be new issues. Linux is not a special snowflake in either direction; it's software made by humans.
I write code with AI daily and it got much faster. If it brakes - it gets fixed then. We face new vulnerabilities, but we find them faster and have better instruments to investigate and fix. The current period is transitional, but for me it's obvious that it AI is a win for software development
The author seems to have glossed over the viruses of the early 2000s,or the Y2K panic, and other incidents from before. Building software has always been hard and it's always been abstractions on more abstractions.
The one compelling part is that now we might be forced to pay for some of the rickety towers that we've erected because AI has made exploits pretty cheap to try.
I think the author is pointing to something fundamentally different than software being hard.
Like before, individuals in an organization had various interests, motivations, connections, knowledge, context, that allowed our organizational brain to function properly. Now, everyone is (or will be in the future) a meat proxy, isolated, and disengaged which produces an organization with neurons but no neural connections: apparently healthy but, as the author put it, decoherent.
> You can find programmers who know how to manage the complexity, but they want to be paid lots of money.
This wasn't always the case, by the way. It started with the "dot-com" mania and VC investment in the late 1990s. Before that, a programming job paid decently well but it was on the level of other cost-center professional jobs such as accounting. A comfortable living, but nothing extravagant.
I mean all of the failures that happened have happened in the exact same way many times before, default passwords not changed keys signed with 5 and 7, we have seen all of this before, and the FAA issue, was it even software, I have not followed the coverage, but its systems setup 20ish years ago and forgotten.
Sure the MS thing might be vibe coded horri, but it's not like we have not seen outages like that in the past.
If anything AI is bringing all of the bad configs and legacy issues to the fore front, because "it" has the time to look for them.
Software isn't breaking down, people just haven't figured out how to use AI yet.
It reminds me when I first started work on a codebase with proper unit tests: I thought I could ship whatever and the tests would catch it. That was incorrect; while unit tests are a guardrail, I still had to properly test my code.
Similarly, people need to learn how to use AI and make sure they don't introduce bugs or slop into the codebase.
But this brings up a larger point for me: most people are grieving the loss of what their jobs used to be, and denial is a core phase of grief.
I went through it too. I tried to find ways to say AI was silly or wouldn't work. But now it's inescapably here and we need to work in the new paradigm. There are plenty of valid criticisms, but it's already very good, it's getting better, and you have to learn how to use it as best you can.
> About ten years ago, two small revolutions happened in our industry ... And then, we had the Open Source boom.
This seems a bit myopic. Or naive. Open Source "happened" a lot longer than 10 years ago. Microsoft didn't shit itself about Linux just 10 bare years ago. And it's not like prior to 10 years ago, the narrow focus of npm world wasn't cobbling web sites together with bits and pieces garnered from wherever.
That said...
> But AI is both a cause and a symptom
This I agree with 100%. The rise of AI and necessity of AI to continue to make progress is a massive indictment against an industry that prioritized more features and uptime over solid first principles for years, taking out promisory note after note in complexity, in hopes that the stuff papered over would quietly just fade away.
> Most SWEs now are unable to tackle engineering problems without some form of AI assistance. That’s not because they’re incompetent - they just can’t keep on top of the complexity. Even if they could once, they now face a tsunami of AI generated code.
That's called outsourcing, which inevitably enables "vibe" effect of the coding: Your AI goes out of its constraints and does totally irrelevant shit you didn't ask for or does something barely passable you could approve, without knowing further consequences because you outsourced your thinking and control over the project.
To try to find a remedy for this shortcoming, they invented "skills", which totally negates the purpose of an AI (LLMs, genAI, whatever you call it to describe the current tech) IMHO.
It's also funny reading these comments. Most say "well, it was already broken" and they continue trusting AI more. Well, AI is not exempt from this "brokenness".
People keep believing that the big LLM players with somehow disprove Rice's Theorem. "Somehow we'll be able to get AI to check if a program does X or Y." "We'll be able to have the machines verify things automatically." But this is is of course as hopeless as solving the Halting Problem. Theoretically, we know this is impossible.
Empirically, we can observe the breaking effects of this broken thinking. Code quality goes down, and apps go from being 99% to 95% reliable. But that's not a big deal, they say, that's good enough... But when the quality of all the building blocks further down get degraded in the same way, you can't keep track of which parts are broken, and which parts are dependable, and the whole thing comes crumbling down.
People keep saying, "I can build an app with AI, therefore we can build everything with AI." This article talks about some potential anecdotes that could help to show that doesn't work.
As the sibling says, the problem is not really Rice's Theorem. We were doing some great progress into software reliability before the LLM-crazy destroyed the industry.
The problem for LLMs is less formal and way more wicked. We don't have any good theory of software reliability. At all. All the reliability we have is artisanal and designed ad-hock.
Rice's theorem is across arbitrary turing machines. We intentionally limit the kinds of programs we emit with type systems, knowing they will prevent us from writing some programs, but we'll be able to analyze the ones we can write much better.
We prove properties of programs all the time, that's what a type system is for.
Can we prove all properties about our program that we're interested in? Maybe, maybe not. But Rice's theorem is not the tool for that argument.
Are vibe-coded apps being being written in Rocq programs that perfectly match dependent types to every possible behavior? Is anyone anywhere writing software with type theory at the level where it perfectly limits things and saves our programs from semantic mistakes?
Of course the point is not whether we can now do this with AI, but whether AI can help us get the same quality we used to get, faster.
Also, consider that it should rarely be the goal to enumerate and judge every possible behavior of a program, with exceptions for life-saving or otherwise critical software.
1. go fast and break things became the norm not the rare exception, mostly due to under-regulation when it comes to responsibility and other much more complex side effects of what you can describe as the US forgoing a fair free marked in favor of out of control monopoly markets they can use as soft power against other countries
2. software being really hard and most tries to fix that being stuck dead before maturing due to them being unfamiliar and often complex to get started with. There being a huge problem with opposite bad ends of "misguided simplicity" and "programmers loving 'clever' & overengineered" didn't help exactly either.
> It wasn’t always like this.
no it was, yes the 2010 felt like a high time. A lot of 90th nonsense has finally been left behind (but would still lead to security issues years later, e.g. log4shell in 2021); Tools for analysis had gotten better; Compute allowed expansive test suites and "go fast and break things" hadn't yet been spread to widely to critical infrastructure (at least kinda, _a lot of the issues with Azure today root back to decisions done in the 2010th_!!).
Most importantly while having well educated highly skilled Devs is needed, _it isn't sufficient_. They still are human. Blaming the huge flood of CVEs on AI generated code is simple, but wrong. Many of the CVE in recent years are human caused, some older then usable AI code gen and at least in case of the Linux kernel much of the code of the past had been written and reviewed by pretty professional, skilled, well educated developers and still AI found way way to many CVEs in it.
So yeah IMHO there is a long standing fundamental issue with how to handle software production, starting with the very first step of requirements engineering (which often isn't done, or at least not properly) and ending with how we fundamentally compose software components.
And the problem I see with AI it makes it worse. We already where approaching a breaking point with software dev and I hoped that this would create pressure to finally find a working way to reinvent it from scratch. But now AI allows us to apply the same failing approaches even more large scale, faster, cheaper and sure with more bugs, but also with AI making many bug fixes far faster and cheaper (if correctly used, it often isn't).
When I first started my career we used to ship like once a month and that was considered good, everything was extremely convoluted and worse than it is today, whole segments of the industry didn't even use Version Control, half the government websites you used stored passwords in the clear and emailed them to you for resets, e-commerce was a cgi script that sent an email for fulfilment and it took weeks to get your shipment, sites went down at night and weekends for maintenance regularly, servers fell over with traffic that would run on a raspberry pi now, but for some reason everyone keeps saying how bad things are now. They're delusional or don't actually remember the past.
Not everyone is using AI to make shittier software. Most big important software projects I know of are frantically fixing an avalanche of AI-found bugs. Important software is getting more robust and resistant to attacks, while still adding functionality.
There's a lot of slop, and it will almost certainly grow quicker than non-slop, but there is a big opportunity for those of us who still work at the bottom to make the pillars more solid, and take on re-engineering projects that would be infeasible otherwise.
Case in point: Before Claude a port of my compiler to a new architecture might take anywhere from months to a year. After AI, now that there are multiple backends as examples and an even larger--massive even--test suite, this can take days or less. It took Claude about 3 days to come up with an arm64 backend for Virgil (around April time frame). Then in August it took just 3 hours to build a complete RiscV backend. Both of them pass the entire test suite. And they shook out other bugs elsewhere. Overall, more targets, and more robust.
Things can have rocket boosters, you just gotta know where to place them.
> I believe that tech companies risk losing their “organisational cognition” - their ability to think and respond as a coherent unit.
LMAO. They can't lose something they never had in the first place.
This article has other pearls like this:
> Reddit and Bluesky speculated that even MSFT engineers had no real handle on the complexity of this outage.
Like, with all due respect, what the fuck do they know?
Also this:
>It’s the software that controls your flights, your banking, your petrol pumps, your TV, your payroll.
Dude, how are you gonna sneak TV software into the same sentence as flights and banking? :D
This is not a serious article, and I don't recommend reading it. But to finish on a positive note, since I've already (regretfully) read it, here's a sentence I like:
> This is the real threat: a scenario where nothing in the system is done with intent, and nobody in the system has control. We generate complexity faster than we can tame it.
I would just add that this was the case long before AI. And also, just as AI generates complexity at such a great pace, it can just as easily help you fix it and fight that complexity.
All software has bugs, and some software is critical
Which means all critical software has bugs!
There's also more of it! Even more once you finish reading this sentence. If security incidents scale sublinearly vs the growth of critical software, we should be proud.
Crappy work is not universal either. There are engineers out there, guiding agents diligently, doing a good job. Never letting an agent go wild unattended, writing the same code they'd have written themselves, but with more tests and more better assurances and more docs and better infra and and and.
The problem that leads to blog posts like this is that good, diligent work, like critical software that does its job, are invisible.
Because no one notices a building that's not on fire. They just walk right by.
"Decoherence: the accumulation of locally valid but mutually incompatible or redundant models within a system, such that the cost of change is dominated by translating among those models and preventing unintended side-effects."
This one is interesting. Not on its own merit which is more like a mother of facts, prime phenomenon, but that it was mentioned at AI. I had this feeling of decoherence with all the various techniques and approaches that existed out there, from programming languages through data management through composing instructions through user interactions, into the way orchestration of all of these, way before AI. Somehow none were matching the specific composition of products needs completely so had to tape together available parts at loose couplings, hammering things a bit if needed. Or choosing those that fit together perfectly on the cost of giving up on requirements/functionality for the product. Lots of good libraries not good enough for some details so a new one emerged or the existing one tried to be so many things that it reached digital schizophrenia stemming in complexity (allowing flame or even religious wars between developers about best/sacred approaches/techniques/mentality/...).
With AI I feel different: the promise of purpose made things fitting together well.
Most people here are dismissing this article just because it doesn't use variations of 'fuck' 50 times or throw in emojis to make it 'fun to read.' And as always, decent comments only appear at the end of the page.
If you want high quality results, you need skilled developers who know their business. It's that simple.
I could stop writing at this point because that's all you need to know to understand why "software is breaking down".
Anyway, the problem is just that there were never enough skilled developers while the demand for software was ever growing. That's the reason why the software industry has always been crowded with unskilled amateurs doing grunt work. Amateurs were tolerated simply because of the lack of highly skilled developers: one good developer can easily replace 5 untalented code monkeys, but it's better to have 5 untalented code monkeys than 0 good developers so that at least something gets done. Programming languages like PHP or JavaScript lowered the bar, attracting even more, even less talented people. It really started going downhill during the dotcom bubble, well before cloud computing was a thing, and it never stopped going downhill.
In such an environment, if you want quality results, the amateur work requires supervision and strict quality gateways so that good code can be distilled from the bad. That is, code monkeys must be managed and supervised by more disciplined, more skilled, more talented developers who are actually capable of understanding and managing the complexity of software.
These days, however, everybody is being dumbed down by AI. Everybody even seems to enjoy being dumbed down by AI, even those who actually are (or rather, used to be) good developers. Instead of doing proper code reviews by a professional, an AI is consulted to do the review. Instead of planning a software architecture, people are vibe-coding. Instead of doing a simple change that actually works, an AI is asked to generate spaghetti code (and you don't recognize the spaghetti because you don't understand your architecture anymore). Good practices that served as safety net against low software quality are ignored because "AI doesn't need them".
Of course software is breaking down, now much faster than ever before! It is because we (as an industry) are moving at neck-breaking speed and have stopped thinking, even those few of us whose job is to keep the quality at least at an acceptable level.
Spaghetti Claude is killing quality faster than most can realize, and it shows already. Your amateur code monkey crowd has been replaced by even less talented, but much faster AI with a knack for convincing language. Your code monkey crowd is still there, but it is now instructing the AI to produce results the monkeys don't understand and push to production. Quality gateways have been dumbed down or simply removed to keep up with the speed.
It’s basically free to black/grey/whitehat penetration test anything and everything without having to even know what ghidra or metasploit so everything gets penetrated
There is one root cause for all major software system issues: business doesn't give a shit.
There isn't a software system in the world for which we can't predict the failure modes and make solutions for. You simply ask two questions: 1) "How can I tell for sure this thing is working?", 2) "If it doesn't work, how do I fix it so it does work?". You might not know the answers at first, and that's fine. The business's job is to ask the questions, do all the work to figure out the answers, and then test & implement the answers. If the business does this, they prevent most of the failures, and have a quick deterministic solution for when eventual failure happens. If the business doesn't do these things, they are choosing to let it fail in an unpredictable and likely very expensive way (they don't give a shit).
This process has a bunch of names. Failure Mode and Effects Analysis, Business Continuity Planning, Risk Management, etc. No matter how you slice it, it's the same basic work: find out what could go wrong (at a high level) and have a plan to fix it when it does. It's not always easy, but you can always figure out some kind of plan that works.
For all the security and reliability issues we've seen lately, there are known methods to either prevent those problems or fix them quickly. The incidents aren't happening because something new is going on. They're happening because businesses stopped giving a shit a long time ago, and the chickens are coming home to roost.
the 4 events (Php, emudhra , MS outage etc) were coincidental. Two of them were mostly theoretical . In the PHP case, despite the tremendous potential threat, the recorded damage has been relatively minimal. Everyone expected to see broad and severe compromises of PHP assets, akin to Y2K’s potential – and very little has happened.
A better narrative is that software quality has been very low for some time, and we are lucky that the threat actors have not compromised the assets as broadly as they could have been .
We have a great opportunity with AI to patch up the duct tape and popsicle sticks. Moreover, to realize that consolidating authority into central management with synchronous coordination is terribly brittle.
In other words, we can fix software by dismantling some of the dependencies.
Yeah, yeah, it's all misunderstood and the timeline is wrong and it's always been this way and and nitpick this and nitpick that but: I believe this is the way the "vibe" is shifting. The average person is going to guess that AI caused all the weird outages in the last couple weeks, corporate and Trump admin denials notwithstanding, and I suspect something like that is what the history books will record. If you are all in on this technology and surprised at these attitudes (or at the "blue wave" in the US, or at scandals "suddenly" dropping on Jensen Huang) it is a good time to look around.
I’d argue this is a consequence of a lot that’s gone wrong in the industry. Maybe there was never a golden age, and I didn’t live through whatever there was, but there used to be real power in the craft of software engineering itself. Over time that power was eroded, co-opted, and sold off. There’s no single event to point to, just an accumulation that’s left us with top companies that operate like cults, people entering the field to chase status, and software that’s less the product of the craft and more a side-effect of dealmaking, politics, and greed. It isn’t true everywhere, but watching most of the talks given in the field now feels like touring the circles of hell. Everybody just seems blind now.
Ultimately, it was inevitable. What I'm saying is just the status quo for centuries and it was a matter of time before software was brought into the fold. There is nothing intrinsically moral about the field, just that respect used to be necessary before structures were created to launder that respect. The good stuff, like most things in life, are on the edges, but the edges don't stay edges forever i.e. see this forum.
It's nice to say, "Oh we did it better in our day! You kids lack discipline!"
But look, the reality is we copy-pasted just as much slop we didn't really understand to get over the finish line back then as kids to today.
Deadlines were missed. There were bugs. Things broke. That's just life.
Now, do I wish we had TDD for everything? Yes. Do I wish there wasn't quite as much emphasis on AI? Also yes.
But there's a literal mountain of slop code that was written over the last 40+ years. Without AI, it sort of just sits there... people are afraid to touch it.
With AI, and I mean me using AI, I'm comfortable diving into a massive legacy code base and refactoring pretty much anything at will. And it's pretty cool when you stop looking at code as "I don't want to be the last one to touch it, because then I'll get blamed if it breaks."
And instead take the approach, "OK, without proper maintenance, this dies. How can I give it proper maintenance?" And look, if something breaks along the way, "Meh." I can fix it. Those old code bases were often just as buggy and duct-tapped as anything AI created... worse... years upon years of multiple developers doing patches. Convoluted AF.
I still like to think I have good discipline. I have a repo that has all my coding standards, and that's a good starting point for most of my projects. I weave in a bunch of rules as I go, things I learn, and that repo becomes the guide for the AI to do work on the project I'm working on. Then... when I'm not the one doing the work, it'll still be there to explain to the next person (or rather AI being managed by a person) why I did things, and what the goals were, and how shit works.
Most things that break... it's just because it wasn't documented in the first place. I find a lot of that... maybe they had discipline, and maybe the legacy systems that had the documentation were lost to time. Sure. Let's give them the benefit of the doubt. But... I'm guessing they did quite a few patches with just someone calling, "Hey Swanson, make the button blue, and make it say 2+2=7 to impress this client!" Right, so there's still a lot of that today too.
> At its worst, AI turns tech workers into “meat proxies” who spend their days massaging prompts and letting Claude figure out if the work is actually complete.
I am going to claim, AI is at its worst. Software companies are doing mass layoffs, they are decreasing the quality of software and reeking the profits, making sure only upper management (or worse, shareholders) are benefiting from software becoming so much cheaper and shittier.
And software workers are not loosing our jobs equally. Those that don‘t want to use AI are being fired much faster than those that do. And those that remain are de-skilling them selves, fast, letting AI do all the work for them.
I want to blame AI, and I want to blame capitalism and the profit motive for making sure that AI would inevitably cause as much damage as it possible could.
It's because amateurism has been hailed as the second coming of Christ.
Leftpad, mentioned in TFA, is a case in point: not because of leftpad itself but because of all those defending leftpadism.
"People can't write a proper leftpad".
Yeah, well... If a dev cannot write a proper leftpad, then it should never have been a dev in the first place.
Someone who cannot write leftpad is not qualified to use leftpad as a dependency (and, no, leftpad is neither the Linux kernel nor a cryptographic libray: it's fucking leftpad, which is the entire point).
We've had "devs" whose only qualification was writing glorified "punch the monkey" ads in JavaScript: the JavaScript surveillance/ad complex. Devs who only ever knew the mono-threaded JavaScript ecosystem and suffered from heavy leftpadism.
And those very same surveillance/ad complex bros are now generating countless crap with LLMs, in languages they don't understand.
They're just mediocre.
That's what happened: the veneration of mediocrity.
You're repeating points that were brought up in the discussions about leftpad a decade ago, but the root cause wasn't venerating amateurism (even though the everyone can code / software bootcamp trends of the time was a by-product of it); it was the veneration of velocity. Which, I suspect, is inevitable in software because it's so powerful, low-resource intensive, and lucrative compared to other engineering disciplines. This technology was always going to create an industry focused on speed above everything else.
I think the exact opposite has happened: needless gatekeeping and NIH-syndrome has created this idea that software development is a skill that measures intelligence.
Because of this, battle-scars from the rough edges of software are seen as part of software-competence, not as a failure of industry. Novices get met with 'you're holding it wrong', but at the same time veterans lose the plot and spend most their time doing technical onanism, or worse, fighting meaningless complexity.
Until the stack exceeds any individuals understanding, and the cracks start to show. Leaky abstraction upon leaky abstraction upon layers and layers of ducttape, under the guise of HA, distributed system mantras, robustness, you name it.
Every part needing babysitting because every part is half the solution to half of your problem, but admitting that is seen as a skill issue.
Yes my DB dies because it caches stuff and (under obscure circumstances) never frees it: 'thats just MySql, deal with it', yes of course my frontend framework needs to be kept up to date every week because everyone is evalling strings all the time, 'don't you sanitize your input?', multiplied by every layer of the stack.
Don't you let your static analysis automatically merge security PRs so you can stay on top of this giant heap of string evals? No? That's best practice man!
This narrative isn't consistent with reality. Cloud computing is much older than 10 years ago. We were doing production grade cloud computing over 16 years ago. The OSS revolution dates to the late 90's. The author seems to think that SWE was in a golden age 10 years ago. Sorry, but it's about the same. We have the same problems in SWE as we had 40 years ago. The scale and ambition of the problems we are solving have increased as compute capabilities have increased.
The timeline is very confused, but the arc is mostly right.
1. Consolidating internet-dependent projects to a few winning cloud vendors and a dozen winning SaaS providers, made all of those project have coincident failure points. Which is great for corporate accountability ("it's Amazon's issue, not ours!") but bad for end users ("shit, EVERYTHING is down today")
2. The movement to migrate fat local-first projects towards thin internet-dependent or server-hosted ones made almost all software share those same coincident failure paths. This was good for entrenching a subscription business model and brought some end user conveniences, but meant everything being down meant everything being down.
3. Anxious and lazy dependency-first development culture ("you could never write that as well as some github rando") meant projects now had an order of magnitude more contributors but only reviewed and owned code for a surface level, making all these projects extremely vulnerable to supply chain failures and subject to constant maintenance churn.
4. Employment and team tenure continually fell so that most projects no longer have experienced owners and instead just temporary custodians.
You can find the seeds of many those problems in the 1990's and earlier and you can even argue that some of them were inevitable (and even worthwhile) destinations for the industry as it evolved without an engineering guild to slow it down and insist on rigor or evidenced safety.
But anybody who's been around for a few decades can attest to how these risky patterns rapidly grew into norms over the past 20 years and shifted the industry from one where any given software had some idiosyncratic bugs that users could learn to work around to one where all software had lots of bugs, many of them correlated across apps, whete failures would wash through and shut down (or make vulnerable) whole industries for hours or days, with effecrively nobody in particular to hold to account for any of it.
Software has never been in a pristine "golden age", but the last 20 years built almost everything on one giant, shared house of cards that's now teetering on the brink of collapse. Better or wose, it's indisputably different than it was before.
> Anxious and lazy dependency-first development culture
I think this is the completely wrong takeaway.
Ober the last ~15 years I have probably used around ~10 web frameworks/libraries
_not a single one was even close to being complete or mostly correct, even just for standard HTTP ~1.1 handling_!!! (The routing, path, header handling. Not any of the complex stuff in HTTP2, Quick etc.)
the problem isn't using dependencies, that is the solution (not reinventing the wheel again).
And preferring reusing solutions which are more complete, practical testet and have more resources to maintain/improve basic functionality is (in most situations) not "anxious" development, but (should be) common sense.
The amount of hidden complexity in even somewhat basic stuff like HTTP is enourmous, rewriting everything all the time is wasting billions across the industry all the time and (in general) increases the chance for bugs.
The problem isn't "using dependencies" or small dependencies, but:
- availability (or lack) of high quality well maintained dependencies
- "no one" wanting to pay for software dependencies
- misguided tendencies for 'clever' & overengineered solution in one extreme (e.g. the stuff which lead to log4shell)
- misguided obsession with deceptive simplicity in artificial examples which lead to 1001 incomplete solutions because of the obsessiv simplicity doesn't stop at cutting out over-engineering but also cuts out fundamental parts
- lack of widely adopted, effective, tooling & infrastructure to trust dependencies.
- lack of tooling to more efficient reuse and compose working parts, the way we do programming seems just very fundamental broken.
- also a lot of legacy tech we are stuck with making many things unnecessary complicated
Also as a side note, number of packages != number of entities to trust. Weather you use 10 or 1 package from the same source, maintained and published in the same way often makes little to no difference for supply chain security.
Powerful. I feel like this is emblematic of the economy as a whole since the 80s. Consolidation and outsourcing makes hyper-efficient but brittle businesses.
* Efficient
* Resilient
* Cheap
Pick 2
With the advent of individual computers/disks/networking equipment not breaking as much, we optimised for Efficient+Cheap software. This results in systems prone to cascading failures.
The most resilient systems I've worked (mostly telecom) on we built from crappy parts which broke all the time, and they sure weren't cheap.
> Anxious and lazy dependency-first development culture ("you could never write that as well as some github rando")
I’ll die on this hill. You should pin your dependencies so you don’t get pwned, and then use dependencies for everything mildly complicated that isn’t core to your product.
I bet 90% of people reading this work somewhere that has built a model router in the last year.
I understand the sentiment. You work at a sparkling water company, and building a durable workflow engine is much more interesting than building a website.
But without that tech being your core product, maintaining and evolving it will lose priority. If it requires barely no maintenance, it’s even worse, because when it inevitably does, you don’t know if deploying a new version still works.
Use stuff that exists. If the dependencies don’t do what you need, fork them, and consider contributing a patch.
Pinning deps only works if you subscribe to security release notifications from every one AND requires that they distribute those ~anywhere. Otherwise you’re missing patches.
And if you’re allowing patches via SemVer, you’re relying on authors using SemVer correctly, which they won’t. For example, the common practice of ~never having a 1.0 so the minor release is basically the major and the patch is effectively the minor.
It’s viable. There are dozens of tools that automatically track dependency updates, and even let you specify a custom policy for pre-1.0 software.
With floating dependencies you still need to track them all to know when you need a new deploy just for the version bumps.
Tracking version changes and breaking api changes in dependencies and cves/vulnerabilities has a non zero cost, it’s simply dishonest to pretend it doesn’t.
> Which is great for corporate accountability
This is a common take, but it's most attractive for its cynicism, it doesn't fully represent the reality.
Cloud vendors are also great for allowing corporations to implement systems that would be prohibitive for them to implement, operate, and maintain as reliably on their own. Even big enterprises often struggle to implement everything they need, that a cloud provider can provide out of the box.
I worked at a multi-billion $ subsidiary of a global Fortune 500 company that tried, as part of its move off mainframes, to "go its own way", with their own hardware in Equinix data centers. The entire executive level of the IT team responsible for that effort - about five or six people - were eventually fired, along with much acrimony, because they couldn't pull it off. There were plenty of warning signs, but the final straw came when there was a major security breach, which was unacceptable because this company had federal customers and were subject to all sort of federal regulations, including CFIUS because the parent company was non-US. (They once had to rewrite an entire major system from scratch because of a breach of CFIUS regulations that made the government consider their code to be potentially compromised.)
That may be an unusual case, but the point is that if even large, well-funded, established companies can get a lot of benefit from cloud systems, that goes double or more for smaller companies, companies in less attractive industries, etc. etc.
(Pointing this out sometimes triggers people who maintain servers themselves, especially here on HN, but the point is those people don't work at the companies in question, and essentially don't scale - i.e. it's difficult to recruit enough of them and manage them and put together a durable system with enterprise-grade reliability that depends on having 10x sysadmins.)
And yes, "shit, EVERYTHING is down today" is a consequence of all this, but the alternative is generally much worse for individual companies.
IMHO it’s a skill issue. Nothing wrong with bare metal.
My friend runs an OSS project with 10s of thousands of users on a fairly realtime and latency sensitive application and it works great and has been very reliable. He does storage, compute, networking, all of it.
I run a FinTech startup on bare metal and do it for 1/10th the price of AWS (In two Geographic Regions). Reliability has been 4-5 9s over the last year. I’ve also worked for a few F500s. I’ve been the guy responsible for buying, racking, networking, and coding. I’ve also worked at AWS and am quite familiar with the pros and cons and nothing about my opinion has really changed, the cloud is a money printer that serves to make certain people extremely rich. Companies and individuals standing up their own hardware and learning how it works is a threat to their business model.
> Companies and individuals standing up their own hardware and learning how it works is a threat to their [cloud/big tech] business model.
This point never occurred to me but it explains so much. In addition to the points made in this thread, I have seen a dismantling of tech education. For example, I created the two-year degree program for Florida in Linux system administration 20+ years ago. I later learned the program's requirements were gutted by a committee; the took a Windows administration degree document and s/Windows|Microsoft/Linux/g. The new requirements included adding users using Linux active directory and other absurdities. Programming degree knowledge requirements were also dumbed down.
All this because educated tech workers are a threat to someone's bottom line.
This is exactly what I was referring to in my parenthetical.
You're thinking of this from the perspective of your own personal expertise, not from a business perspective. The problem is that you don't scale. There are millions of businesses using cloud services. You're not working for any of them. Do you want to?
And are there millions of people with your experience and skills? Doubtful. (If there were, you'd be a lot less valuable.) And a company of any significant size can't rely on one person to operate their systems, so you need a team, and you need to manage that team, and the managers have to be good as well.
Also, 10s of thousands of users is tiny in the modern world. Instagram scaled to 30 million users on AWS with something like 13 employees, before it was acquired by Facebook. It would have been much different if they'd had to do that on their own hardware. They're an outlier, but many business are able to use the cloud to scale to levels that would require a massive internal effort otherwise, especially if they have to meet compliance requirements, integrate with other enterprises, etc.
> Companies and individuals standing up their own hardware and learning how it works is a threat to their business model.
They don't care much about this "threat", because executives look at the situation and choose cloud because it makes business sense.
It is very much a skill issue. It's not some arcane art to keep servers online. We did so for years before the cloud was trendy.
This reminds me of an old boss of mine who wondered why we had to spend so much effort on securing our systems. I pointed out to him that we didn't used to be connected to the internet.
"Before the cloud was trendy", small to medium businesses didn't have millions of customers using their online systems.
> I worked at a multi-billion $ subsidiary of a global Fortune 500 company that tried, as part of its move off mainframes, to "go its own way", with their own hardware in Equinix data centers. The entire executive level of the IT team responsible for that effort - about five or six people - were eventually fired, along with much acrimony, because they couldn't pull it off.
> ...
> And yes, "shit, EVERYTHING is down today" is a consequence of all this, but the alternative is generally much worse for individual companies.
IIRC, AWS and cloud infrastructure didn't get big until the mid-2010s. Fortune 500 companies (and everyone else) were able to "go their own way" with their own hardware before that, they had to. That was just ten years ago.
If "[going your] own way", with [your] own hardware" is too hard to do now, I'd guess that's not because it's impossible, but because there's been a combination of skill/capability loss (similar to what's been seeing in manufacturing) and places like AWS sucking up much of the remaining talent.
And now add vibe slop onto that, in both the dependencies and the implementations and the infrastructure management, and you really have a recipe for cascading failures from which it’s difficult to recover.
I’d like to add that the perpetual gold rush during this period has created a culture of self-serving that few other professions can compete with - basically institutionalised low-level corruption.
Decisions are made on the basis of trend chasing, what’s fun, and what’s good for the devs’ resume. And it’s completely accepted, hence the world is overflowing with Rube-Goldberg machines that basically defrauded whoever paid for it. Try doing that as a carpenter and see how long you’d stay in business.
Yes, there's too much to pick at with this piece.
The PHP2Shell bug existed more than 15 years ago, spanning releases as early as 5.x. There was no AI coding back then. So it's very hard to blame AI. There was a coding mistake, and AI found it in 2026. Probably a good thing before a malicious party does the same.
OSS definitely flourished way before 10-15 years ago. I'm not sure what the author suggests as "knowing the details". Should we all rebuild our own web scripting language to know "all the details": e.g. PHP? Probably not practical. A line has to be drawn on where your knowledge and expertise stop. THe line can be vague and consistently changing based on needs. But there will always be a line. We will always stand on shoulders of of our predecessors one way or another
I understand the author's trying to say the overall situation is unstable and trending negative but I am not sure the writing's logical flow is holding up very well.
A better way to put it is we've written a lot of code, and a lot of it had bugs hiding in plain sight. AI made the whole bill come due at once. And will this AI code explosion create another bill we won't know about until 20 years later? Who knows.
I don't think the point was to blame AI for PHP2Shell. It is that we've built up gigantic systems on shaky foundations: "a highly complex system made of low quality parts, where a single error can compromise everything"
Of course the shaky foundations were there before AI. And before the widespread proliferation of NPM packages etc... But those increase the ease and pace at which we can build on top of the shaky foundations, they increase the rate of so-called decoherence. And there are economic and organizational incentives to keep increasing that ease and pace. That makes it more and more likely that a problem in one of the shaky foundations (whether AI-created or not) causes a system-wide failure, and at the same time results in a more and more generalist tech workforce, making it harder and harder for an organization to respond appropriately when that happens:
> I believe that tech companies risk losing their “organisational cognition” - their ability to think and respond as a coherent unit.
> This is the real threat: a scenario where nothing in the system is done with intent, and nobody in the system has control. We generate complexity faster than we can tame it.
The major problem IMHO is that AI massively reduces the required skill to exploit. PHP2Shell? Even the best minds working on PHP were not able to come up with a way to exploit the vulnerability, which is why it was shelved aside in the first place.
But now? I member someone on Twitter claiming a reproduction cost them less than a dollar worth of tokens.
That isn't just tech debt coming due, it's a bunch of mobsters armed with the equivalent of gatling guns. But unlike powerful guns that governments can somewhat regulate, there just is no way to get the AI genie back into the bottle. People run abliterated models on their own hardware now, the equivalent of 3d printed guns, just so much more powerful.
We are getting to pay for all of the tech debt from the past 30 years all at once.
AI is basically at this point a super auditor that doesnt fall asleep.
AWS is 20 years old, before that people did rent machines from places like Rackspace. Maven Central (Java) is 25 years old, RubyGems is 22 years old, CPAN (Perl) is 31.
I find the arguments about decoherence and the sort of snowball-of-complexity more compelling than this timeline.
The first widely used cloud computing platforms probably were the PHP/MySQL hosting services of the late 90s. FTP your PHP, done. No ssh, nothing. Only your FTP folder with the files you uploaded.
I agree with you. I can see how we might be accelerating decoherence with AI, but maybe it won't ultimately matter. I just don't know.
I get the impression that when the author says "open source" they mean "rampant use of dependencies". And while the terminology is confusing and the time frame maybe not quite right, there certainly has been a shift where many projects went from depending on none or a few big reputable open source project (MySQL, Apache, Linux, etc.) into depending on hundreds of language packages made by random people.
Yeah when I read that the blooming of open source was in 2014 with NPM, I thought, huh, I sure remember using bundler before that and GitHub existing.
Reminded me of this tweet: “This has been talked about extensively you’re just 21”
Just because it is/was new to you at that time, doesn’t mean it is New.
Thoughts on the inter-connected bank outages today?
This article is pure nonsense. Yes there may be more and more failures that are getting more difficult to catch and remedy in time but that is the nature of the kind of systems that we build nowadays and that is distributed systems at massive scale. When an error occurs in such systems the effect may show up in a far different component or part of the system and in multiple places effectively making it difficult to troubleshoot.
Yep.
It doesn't have to be this way.
AI is great at following patterns. AI is great at writing tests. AI is great at doing work that humans would consider drudgery and then applying "enough" judgement to it.
The problem is that most software teams and most engineers are very, very bad at the thing that AI needs to succeed: technical writing. Documenting the patterns, the rules, the heuristics, the SOP, the judgement; developers could barely be bothered to write comments before (which in the AI age are free, infrastructure-less memory that is always "line-of-sight" for the agent and can be used to enrich with keywords).
It's not that AI is the problem, it's that we humans are good at resolving these constraints in the real world at a human pace. AI makes that pace impossible for a human to keep up as a trailing activity. Rather teams need to invest more in technical writing and producing the corpus that AI relies on to learn the operational constraints and rules.
I'm actually quite good at building robust systems, cleaning up messes, documenting things well.
As a result I get:
1) The enmity of my lead dev who prefers to do things more fast and loose.
2) Not promoted, due to the first reason and due to the fact that I don't have "impactful" metrics.
I'm not saying I'm perfect, there are definitely things I could have done better. But the fact still remains that the industry grossly undervalues maintenance and optimization that doesn't lead to easily measurable numbers that impact the bottom line.
That in turn is because it's hard to tell if SWEs are actually paying down critical debt or just wasting time. But that's the core problem, not that we're not good at it.
The good news is I can get AI agents to do this kind of work so I can now chase after metrics.
I feel similarly: I am slow but solid. The industry values those who move fast, break things, and make a mess.
Which makes sense: those in a position to promote others generally don't understand the technical part. What they see is how fast they get a new toy to play with. And if it breaks too soon, they will still be able to blame the developers.
I think this is a real alignment issue - nobody has excess time/money to make the software robust/secure.
This is possibly due to the high level of economic inequality, meaning less resources in the middle of the economy.
In a decade I think we will have few people who can competently write or even read computer code without AI assistance.. and I think we might say the same about math and science and medicine.
AI assisted learning could lead to a renaissance of human capability .. but thats not where we are headed, we are outsourcing all the hard thinking skills. In future, only a few crazy retro nerds will learn assembler or multivariable calculus, just as only a few crazy nerds now make mechanical watches as art objects.
In my own social circles I hear of capable young people who want to forego university and just take up a trade, because it avoids debt.
Excessive inequality means we wont be training young scientists, and young couples will defer having children until its biologically too late.
High inequality + LLM AIs + climate change is not a recipe for human flourishing.
University means knowledge work, and knowledge work is already getting fucked over by AI. The clankers can now design rudimentary PCBs. The only university degrees that will not be compromised by AI are the ones that deal with humans in the meatspace - think medicine, psychology or education, although I think we might be getting a revisit of MOOCs augmented by AI to drastically reduce the need for teachers.
Being good at one of the trades however? That's a pretty safe bet. It will be quite the time until physical robot "bodies" are capable of the dexterity, size and endurance of a human. And by the time robots get that capability, we're either living in a Star Trek utopia or in Matrix style neo-feudalism.
> This is possibly due to the high level of economic inequality,
Friend, I work for Google. The high level of inequality works completely in our favor, yet we are incentived to churn out slop that hits metrics.
While you do that, look for opportunities in niches where "fast and loose" is unacceptable, because while they might not market themselves as loudly as most software development organisations they do actually exist.
Typically these places are associated with compliance, audits and regulation with teeth. It might be adjacent to lawyers, in public institutions or weird consultancies. At least that's where I've found such work. Insurance and banking might be decent bets, and they sometimes also come with oddball tech stacks, perhaps mainframes, perhaps Smalltalk.
I absolutely agree, as a child of automotive industry I am looking on software development through the lenses of kaizen loop. If you can loop the development cycle so it can correct itself through tests, then errors, either human or AI made are never going to be a problem, they will get self corrected by the testing.
Where do you get the correct test results? First from the specifications, then gradually from the edge cases found by testers and eventually users. It is not one shot process it is a big loop which will get repeated over and over again until users will run out of things to break.
The software is breaking because nobody is bothered to write down what is software actually supposed to do, and from there to create a set of tests which will anchor the behavior of software.
Nobody bothers to write down what it's supposed to do because nobody thought about it enough for a full spec to be conceivable, let alone written. In a lot of modern development, you have a committee of non-technical product people who might, at best, own a single flow, but they sure don't own, and don't care, about the interactions across flows. And then you have development teams that live and die by KPI, and thus draw the edge of their responsibilities in the narrowest form possible. I was told by a subscription team in a certain very large company that they just own the subscription platform, not anything about the data in it, or the relationships mapped in the data. Bundling? Data. Billing? data. They are just 40+ people whose responsibility is to keep a database up, and therefore no outcomes other than inability to retrieve the data was ever their problem.
So who owns outcomes? Usability across the board? General customer experience? Nobody. The actual software that people pay for, is orphaned, and anyone trying to track down problems is just asking to be blamed for them. Thus, most of what happens is undefined behavior, like if we were writing an old C standard. Having someone that has the responsibility to make the product as a whole do something, and thus maybe be able to write the spec eventually, would already be an improvement.
Feature churn moves faster than this sort of feedback loop does.
> The problem is that most software teams and most engineers are very, very bad at the thing that AI needs to succeed: technical writing.
Software engineers were at least pretty good at technical writing using the artificial languages we created to enable clear, unambiguous technical writing... programming languages. Seems like software engineers might do better using AI if they remembered this and started using pseudo-code as their language for spec writing (as many have recommended).
Some large percentage of the dialogue around LLM prompting boils down to arguments we should be speaking to them in Haskell.
The urge to one-shot everything isn't helping. Getting people to push back on assumptions the model is making, and to have an actual conversation with it about the solution, would lead to more maintainable output.
Maybe part of it is laziness but for sure part of is low morale and hopelessness.
Part of it is the hype cycle too. Everywhere you look online, all the easy examples are people one-shotting this and that. Strangely it seems like the 'standard' way for people on YT to 'test' a model is to just one-shot some random game idea. It's meaningless, but it creates this unconscious expectation (even amongst experienced devs who might not be as experienced in AI use) that the models _should_ be able to derive exactly what's needed from a one-shot.
That's fundamentally impossible though -- if the information isn't there, the model is just going to move toward the average and fill in the blanks.
On another note, I'm also not sure where I stand yet on the thing I see where software devs end up falling down the rabbit hole of building little mini software factories, with agents reviewing agents, coordinating agents and having agents cosplay human roles -- essentially 'orchestration porn'. I'm yet to dive into that rabbit hole (I likely will at some point, so I'm not judging), but from the outside it looks like you end up worsening your understanding of the system, while spending tokens unnecessarily and creating a closed-loop mini model collapse.
I've actually run into the situation at work where I've asked people whether the libraries they've made can do a particular thing and sometimes the answer has been "I don't know" (I'm talking just _features_, not even low-level architecture). At that point, something has gone wrong.
I'm starting to get really interested in "Okay, if human code review is fundamentally impossible given the volume of code, what can we do?". We have a tool that can essentially build tools for almost free, so I'm trying to approach it from that angle. Let's see what sticks.
> it creates this unconscious expectation (even amongst experienced devs who might not be as experienced in AI use) that the models _should_ be able to derive exactly what's needed from a one-shot.
It also creates an expectation that anything done with AI as part of the process was done in one shot, and therefore the person had no meaningful impact on the output.
Oh this so much.
I saw a Reddit post of someone asking essentially “I’ve spent the last 7-8 months working with Claude on a game, am I really making something?” and I just wanted to reply “of course you f****ng are, don’t let people tell you otherwise!”
For this reason, I keep a "how it started / how it's going" log for everything I work with, to show how it looked after the first prompt, versus how it ended up after extensive work. Just to make clear how my input actually mattered to the output.
I think the correct way to solve this problem is completely at odds with AI.
Write less code. Reuse things that have already been written and thus were already reviewed and tested by someone else. Do not reinvent the wheel. KISS.
With AI everyone is implemented a dozen libraries from scratch in their project instead of using something someone else made. Why use a library for 2D geometry when ChatGPT can write one in 10 minutes?
I have to think part of this is model training, encouraging the model to code-up something new instead of looking for library code that can fit in the project. One of the things I ask Claude repeatedly is something like "can Spring do this already, or is someone working on this already?"
More tokens, more money.
That just follows from the fact that people can learn. Human organizations always have to balance between tacit knowledge and explicit documentation. Document too little, and onboarding new people will be more difficult, and you will lose information when people quit. Document too much, and you are just wasting time without doing anything productive, because people would have figured out that extra information in less time than it took to write it down.
The fundamental problem is that current AI tools don't learn. They make the same mistakes over and over again, unless you give them explicit instructions to avoid those mistakes. Maybe we need to accept that all work is now done by new hires, who have read all the documentation but have no experience in anything. Or maybe AI tools will get better, and all that tedious documentation effort was just a waste of time.
>current AI tools don't learn
and
>and all that tedious documentation effort was just a waste of time.
Seeing where the AI is failing and then putting out good public documentation is a good way to ensure the next released model learns.
Public documentation can help to mitigate the issue, but only with information that can be released to the general public, and only if it can be easily put into words. But as long as there is a separation between training and inference, the divergence between what the model knows and what it should know only grows wider over time.
I've been one of the organizers of a relatively large RPG convention for ~25 years. If I had to write down all I know about that convention, it would easily take a year or two. Because most of that knowledge is not in a form that can be easily expressed in words. And in my day job as a scientist, a similar task would easily take 5–10 years.
The problem is not the input (documentation) it's the evaluation of the output. Managing an AI is like managing a team of specialists, whose subjects you are not a master of. Most programmers did not have the problem of managing experts - we were the experts. There are those that do, but several of the levers you have over a team of humans (incentives, responsibility, even hiring and firing) are not available to control an AI, and the new levers ( fine tuning, training, reinforcement learning) are not in the hands of those relying on it. This is not solved by any amount of documentation.
You hand the same documents over to AI agents that evaluate the output.
The same rules that you, an expert would apply, should be documented and used by an agent to apply those heuristics.
There are obviously some areas where there's spot judgement, deviations, and exceptions. But by and large, you have some heuristic that you use when evaluating output. Document it and give it to an agent.
This suffices for merely random error (as long as the error is demonstrable) but it's not a general solution to the problem.
You're asking ICs to become managers of a bunch of mid-level developers (agents). It's a skill that has to be developed like any other, and takes time.
And in my experience in the software industry, most managers are bad (presumably because they didn't develop that skill either).
It was like that before AI, it won't get better now that we need even more people to pretend they know how to manage developers.
It does have to be this way, unfortunately, because AI bros will press to have as much software produced at the absolute minimal cost possible.
I've seen it first hand. 70 people fired, 20 people left. The 20 that are left take on all the work that was done before, plus some extra, because every company now has a new AI product, obviously. AI agents make us so productive. Every review is a 30 second LGTM on 500 new lines of code. Confluence filling up with AI slop.
A few weeks back I saw, first hand, a DevOps lead getting ready to ship a big update to performance-critical product where Opus 5.5 suggested they completely turn off JIT everywhere because they were running into a HotSpot C2 optimizer bug that would make the first request after a restart fail with a 500. Actual fix was a one-line adjustment somewhere in a library.
People are scared of being replaced, so they need to appear as productive as possible.
You can be productive and guide AI to operate in a more compliant way: by writing down basically everything you know, every heuristic you have, documenting the correct patterns, documenting the framework level approach, explicitly spelling out what's good code, writing down detailed specs for an agent to work on, writing the test plan, etc.
A lot of devs still think their job is to produce code; it's actually to produce all of the guidance and structural scaffolding for the agents ("the harness") to guide its behavior and output.
A concrete example: I wrote a small library[0] (MIT, OSS) that improves how agents write unit and integration tests. This isn't product code, but this makes the agent better at writing the tests it uses to verify its work. (Tests become denser, more semantic, less context, type-checked setup, etc.). You can't expect an agent to do this by itself; the human needs to produce the scaffolding.
[0] https://github.com/zeeq-ai/zeeq-biome
Do you have eval testing? Benchmarks? Deterministic model checkpoints? Deterministic harness versioning?
Building "orchestration" without being able to deterministically measure improvements is not engineering, it's cargo cult.
agree- this is how i name variables, this is how we handle third party APIs, this is how we handle caching and here are some heuristics for when we want to cache and when we want to avoid it, etc. etc. all coded up in a nice documentation tree with links in the agents.md that say "when integrating with third party APIs, read docs/integrating-with-third-party-apis.md".
i have written alot of this type of documentation as an architect for human teams, and i find it works for agents as well.
> A lot of devs still think their job is to produce code; it's actually to produce all of the guidance and structural scaffolding for the agents ("the harness") to guide its behavior and output.
This basic reasoning is wrong and leads to “orchestration porn”. Thinking that you can laid down rules to automate software production like it’s a factory pumping out products.
What you forget is that software are grown iteratively, not guided down some assembly line. Every patch is a revision of what the software should do, not the next step of a workflow.
So stuff like patterns, architecture, tests,… are helpers for communicating the purpose of the software. They’re not really relevant to the software itself as the computer would be happy with just a single file.
You have a software that’s needed to fulfill some needs, you have a team of people where each brings expertise to build it. Workflows are built so that everyone how to contribute, what they can depends on, and what others need. When there’s a failure, we add the missing information. When there’s need to do some changes, we inform everyone impacted.
Everything is information and communication related. Information about the system and communication between everyone that interacts with the latter.
So devs do not think their job is to produce code. It is to take requirements written in ambiguous natural language, sometimes incomplete and contradictory, and to design a solution that is formally written while taking care that all statements are coherent.
Making things coherent (aka correct) is what takes the majority of our time. The other is gathering requirements. The actual typing of code is pretty much instinctive, just like you don’t check the orthography of words like “concrete” or “guide”, and rely on the spellchecker for more difficult one.
The “orchestration porn” part is that there’s no mention anywhere on how you think correctness and be improved or even maintained as part of your tool.
I suspect the majority of reviews had been a rubber stamped LGTM. Often the minority with substantive feedback were heeded and changes made; I'm not sure that is happening anymore.
> AI bros
You have to stop thinking this way. It's our job to do what our management wants us to do, which is often to produce as much software at the lowest cost possible.
There are legitimate criticisms about AI use and how AI has changed the profession, but you have to learn which battles are worth fighting.
We were just following orders!
The refusal to take responsibility for the consequences of one's work is one part of a vast complex of machines that normalize the horrific. We have met the shoggoths and they are us.
> It's our job to do what our management wants us to do, which is often to produce as much software at the lowest cost possible.
Have you ever been in a situation where you did what the management wanted you to do for years, regularly warning them that you could not write good code while doing what they wanted but being dismissed because they knew better? And have you ever been in a situation where you finally got your "I told you so" moment after years, when they actually realise it won't work?
I have. You know what happens then? It's not their fault. Best case, they explain why you couldn't have known better because it was hard. Worst case, they fire you because... well you built that mess, remember?
What's the winning strategy for a dev in that situation? IMO it's adversarial: do whatever works for you: work less and pretend it's hard, or chase a better salary, don't get attached, and prepare to jump ship whenever needed.
Is it sad? Yes, but they start it.
Doing what management wants you to do blindly is an easy path to ruining the company and losing your job. We used to joke that the true mark of a senior engineer is being able to tell management when their ideas are stupid, will not work, and "no."
Just like crypto bros a few years back, where we "do what our management wants us to do", built stupid-ass blockchain into every every which app that didn't need it, and when it doesn't sell get blamed for it.
Personally, I was asking "but what's the use case" in every meeting. "Why would somebody pay for this". "Who's the customer". Right now the vibes are quite similar to back then.
Management are AI bros. Bro.
Dude, you just didn't write down the rules and the patterns effectively. Just write down "don't completely turn off JIT" /s
Damn if only there was a way to write strict rules and patterns in the english language that could be converted into strict instructions for the computers and ways to verify those instructions...
This is hilarious. Just "document the patterns and the rules" guys, then AI can do the rest.
I agree that humans are the problem and their gullibility towards token prediction algorithms and CEOs blathering.
SOP and documentation is everything. You don’t want it going out of bounds and coming up with its own solutions, which are often just inefficient facsimiles of random stuff it finds that seems to makes sense. You also run the risk of not understanding the solution it puts in front of you. With an ironclad SOP and quality documentation, you know when it is straying from the best paths, because they are laid out for both of you.
Plus your documentation can often work very well alongside whatever skills you have created over time, because you basically have two sources it is drawing on telling it to do the same thing. I find it leads to less off book behavior.
I’ve always been perplexed by how people see AI as this creative tool or something to innovate with. That’s not how I see it at all. I see it as a device that can follow a template very well if you lay good tracks. They can archive and check hundreds of pieces of media way faster than I can. But I would never trust it to craft a story
IMHO, most places are pretty terrible and documentation. I work with people today that seem to think someone else is going to do it for them.
Well after five years of miserably keeping our SOP tight and templatizing (is that a word?) our edits aggressively, we are finally seeing some dividends with resolve’s recent Claude integration. It was a painful process but the amount of time we are now saving is well worth it. Also takes a lot of the drudgery out of editing. Anyone who has been a DIT before knows what I mean lol it’s nice to sit down to a prepared timeline that is already roughed in with multicam clips and effects all applied.
"AI is great at following patterns", sure but if some attackers can "inject attack patterns" at a high rate in your training set (check "dataset poisoning") then it's going to be an issue...
I completely agree, and I’m building a tool to solve exactly that challenge.
Pathfinder lets you understand your product’s building blocks. So you and your team pilot your agents at the product level, not the code level.
It’s augmented product engineering: https://pathfinder.com.ai
Software is not really being more broken today than it was 10 or 20 years ago. It actually has gotten better. I've seen the codebase of a dozen companies and worked with hundreds of peers. There is plenty of horrific bugs and bad software outside. The reasons are plentiful too and include everything like budget constraints, management failures and engineering skill issues.
Today, the average code that even a product manager or junior engineer submits is actually excellent overall - if your processes, harness and CI etc are well designed. It is actually super easy to author software well now. Whether you build something that is actually needed and is tasteful is another matter.
99% of the security vulnerabilities being found and exploited today were created by people. Cutting most of them out of the technical details is actually good.
Let me play devils advocate:
Even if individual code is of higher quality, if there are more pieces of code talking to each other, there are many, many more failure modes because software often fails at its interfaces. Nancy Leveson from MIT has written well on this subject.
Perhaps a bad example, but isolated code will often fail less than networked code that has to interact with a bunch of stuff. An old “dumb” tv never seems to have software issues, my new “smart” tv consistently has issues. It’s like having a dozen people write a paragraph each or a single person write a paper. Even if the single persons writing isn’t the better than everyone else’s, you’ll often get a more coherent paper.
Imo: the way we make software is breaking down at a systemic level.
The harsh reality is that some people/systems are adapting just fine to the new way of doing things, others are making a mess of things by rushing via vibe coding and not checking the work (either by choice or by circumstance at work). The results won't be (and shouldn't be expected to be) linear.
The 2010s brought on a very groupthinky dynamic to software dev. This led to a culture where the majority of developers just ask "what is everyone else doing." Now that we're past the blog era of development, that shared reality appears to be breaking down. Now the answer really is "whatever makes sense for/to you," but I think (anecdotally) that's freaking a lot of people and teams out.
For the first time in a long time, a "best practice" doesn't really exist.
Your last sentence nails it from my perspective and experience. I run a software dev agency, and for the last like 10 years, us and all of our competitors have pretty much done the same thing. Followed best practices, approached software dev from a "normal" viewpoint. We competed on price and expertise.
Now, almost all of our competitors and us have taken a fundamentally different approach to how we use the AI tools for our work, and it's been really fascinating and fun to compare and contrast with them. We're finally different, and to me that's a great thing.
I'm sure approaches will converge into "best practices" again but for now I'm really enjoying that we don't all have to be the same.
> This changed the balance of power between workers and management.
I think this is the core insight that I keep coming back to: technological advances always entail a transfer of political power.
We would all be better off, when evaluating new technologies and listening to their evangelists, thinking hard about who gains power and who loses power as a result of adopting that technology.
No, we'd be worse off. It's almost impossible to vet technology and only adopt the good. People are already extremely biased toward the status quo; the last thing we need is one more reason to avoid adopting new technologies
That's not necessarily the moral of the story though.
If we hadn't spent the last decade circle jerking about how we're all 10x devs and everyone else is a pleb there's a chance we could have built unions for ourselves.
That would mean we'd be able to actually wield political power collectively and establish standards for quality and technology adoption like all the other critical industries have.
We don't need to say "we're not adopting tech", we need to say "we're going to follow our process for adopting tech" which may be slightly slower, but the banks would still be online today. Further, it doesn't even necessarily need to be process, it just needs to be something we have a say in instead of a hand full of the craziest narcissists to ever walk the earth.
Unions that have "wielded power collectively to establish standards" are generally bad, and have mostly made the world worse. That's the last thing software needs.
Software wasn't exactly hard 12 years ago either, the barriers to entry were being lowered at a rapid pace. 2014 was the dawn of the software bootcamp where JS became more and more pronounced in web architecture.
I feel this piece is either more alarmist than it deserves to be, or it is late to the party.
It uses various incidents in 2026 to prove a point while dismissing those that did take place in the decade or so previous. For example, heartbleed, the log4j RCE, Rails' mass assignment DOS, Cloudflare updating a WAF rule and breaking the internet, Atlassian deleting almost 1000 of its customers' accounts and data, the Ashley Madison breach, Crowdstrike bricking their customers computers, Okta getting owned multiple times, etc.
My own point is that software became like this when we started becoming pragmatic. Nothing to do with AI or management really, we just realised, especially with web tech, that you can release something and then fix it. In the old age of getting CDs or floppies printed and distributed, that was not even remotely feasible. A similar incident in the 90s would involve a product recall or a way to get updated media from a retailer with the fix. Very expensive.
Standards have dropped and continue to do so, because we can also fix fast. It's not suddenly breaking down. If anything, it's just too centralised.
I don't remember the details but a decade ago there was another authority signing keys with 3 and 5 or something ridiculous close to that.
Software has always been broken at a systemic level. It’s a complicated continuously changing web of abstractions and abstraction breaking hacks attempting to survive prolonged contact with reality.
A decade ago there was a huge culture around having 100% uptime, nowadays it seems that not only is lower uptime acceptable, but services need to have some downtime and break when they are not profitable. Filtering out the undesirable users and upgrading them to plans with better economics is a new requirement in this age that wasn't present before during the growth stage and freemium VC backed startups
It was less broken before open source allowed an architectural explosion, because the dependencies were too expensive. It took a lot of arguing to get management to splurge on a single library or other tool. A lot of astronaut-architect wannabes were constantly grumpy about mgmt that wouldn't pay for the latest whiz-bang thingie.
The actual programming logic has certainly always been complex, but the integrations weren't so bad. You could learn your very small toolset quickly and focus on ideas. Now, you have an ultra-massive toolchain to come to terms with, made up of free software and "cloud" architecture that starts off cheap and gets expensive later on.
Dependencies have always been the problem, but that problem has expanded in every direction, at every layer, approaching our own limits of comprehension. AI has allowed us to catch up, temporarily, but given the collective penchant for upping the ante, I expect the real supernova is yet to come.
Is the problem open source? I haven't see these kinds of widespread problems coming from Java dependencies (25 years old) or Perl (maybe 30 years old)? Okay, maybe people aren't using Perl like they used to but Java is still a thing.
Part of it is the inclination of the developer to make the thing more robust. Part of it is lowering the barrier of entry, maybe to the point it's _too_ easy to submit a package. Part of it is bad decisions made by the package manager, like running arbitrary scripts.
I don’t doubt you would find a trove of issues in CPAN if you went looking. It’s not an interesting target, though, because it’s all but dead.
Java via maven central is similar. It’s not a popular language. We know those libraries are riddled with issues - log4j’a big vulnerability isn’t that old - and simply requiring a GPG signature on a package is a thin guard against compromised uploads.
The NPM ecosystem is uniquely suited to supply chain attacks though. It has a massive rate of churn, which makes it impossible to maintain dependencies safely. You either lock in and hope there’s no big issues or ride the wave and hope you don’t get hit by zero days.
I agree that CPAN is not as active as it once was. Maven is hardly dead, it's pretty active (as is Java the language). NPM is very busy but they also have a very real bot problem.
Yeah, I also question if this has anything to do with open source. Open source solutions have been around for decades, heck React has been open source for 13 years.
AI has allowed us to scale development effort massively - and shows the same weaknesses as scaling human teams too quickly - planning, verification/testing and code fragmentation become issues almost immediately.
I have seen mostly engineers complain that management wouldn't stop paying for the new whizz-bang thingy.
Well, yes, but the proportion of systemic level broken stuffs might have change a bit. Just like, ok, there was always rule34 lurking to make dirty everything innocent that remained of the untainted child memory one had.
Now it’s somehow guarantied that the volume of dirtied quickly prompted version of anything anyone ever cherished far exceed the scarce outcomes of the original.
I do embedded. I’ve always marveled at how ludicrous that “real software” is. I have truths. I have registers that will never be or act differently, I can make pretty damn good assumptions that no one will put this code somewhere else’s etc.
In the past few years though, it’s been creeping into my domain too.
>I have registers that will never be or act differently
Even this needs a qualifier. Radiation exposure can cause bit flips. Certain aerospace applications must address this, and even some earth-bound applications at high altitude.
At high altitude or high deployment numbers. Hyper scaler data centers measure these “one in a million” anomalies in thousands of times per hour.
Yeah, don't we have error correcting RAM for this reason, here on Earth?
Sure, but RAM isn’t the only place a bit can flip. Registers are a thing. IBM had error correction in Power architecture CPU registers, not sure where AMD, Intel, or Apple (lol) are on that issue. IIRC register bit flips first came up with Sun SPARC hardware, which Sun mitigated, but of course SPARC isn’t used too much these days…
A quick AI-assisted search informs me that neither AMD nor Intel document any on-CPU error correction, though perhaps there is some. Apple currently has zero systems that even use ECC, which seems insane to me given they’re shipping systems with up to 512 GB of RAM.
This seems to be an issue that’s falling through the cracks again. I guess nothing will be done until there’s a critical failure due to a bit flip, and verifying that was the cause for a given problem is difficult.
On ECC, Apple needs to Think Different (tm). Maybe it should introduce actual “Pro” machines…
Besides, chip bugs are famous since the Pentium, and unreliable circuits are the normal way to describe circuits, you have to add notice when you get a reliable one.
That's why we do requirement gathering and analysis. Engineering is always about tradeoffs and things that matters in certain context don't in others. The issue is that in a lot of orgs, while there's a record of decisions, we don't usually track assumptions and the overall system is full of contradictions. Also there's no overall keeper of knowledge (either one person or a team) that periodically ensure that contradictions are kept at minimum.
I don't get this thing where people are allergic to the idea that change can happen.
No matter what's trending worse there's always this thought terminating phantom law of conservation of everything. The fact that something is not an entirely new bespoke phenomenon always turns into denial that it could be happening more often or in a newly qualitatively worse way.
This feels a little hyperbolic, the grounding was a result of a cable swap and the ssh cert incident they cite was so human caused it’s not even funny. I think things are a little more shaky, but not massively so. I had the town I was living in during college lose its internet and phone lines due to a cable being cut on a mountain pass and it was fine. If your business is sensitive to an email outage, prioritize high reliability over integration advantages that outlook offers. The php2shell vuln is bad, but we also had log4j years ago.
I’d like to analogize the situation with the Kessler syndrome: the increasing density of satellites in low Earth orbit increases the probability that a cascading failure occurs, which would deny us access to LEO for centuries. The inciting incident could be intentional or could be accidental, but the result is the same.
The analogous situation is brewing. The depth and complexity of our computational infrastructure is increasing, and the impact of an incident is less and less likely to be contained solely to the affected system. Eg, a router replacement goes awry, and the entirety of the world’s aviation industry is broken for weeks.
Intentionality isn’t necessary: this is a systemic problem.
> the grounding was a result of a cable swap
When the official cited story (put forth by a known boot-licker) is contested by people on the front lines, I would defer my conclusions until more is known.
There are some decent points, but this is collating a lot of separate incidents with wildly different causes that just happened to occur in a close timeframe.
Let's see how many more incidents will just happen to occur in proximity to those events.
I don't even know if there are decent points. This flails around in a way that's just...not super useful to spend the time trying to digest.
Some things happened. AI is also happening, therefore AI is to blame (in the way that some people really believe a problem isn't a problem unless you test for it) and here's a lesson about organizational coherence or something.
These essays are the real AI slop of 2026. Not authored by AI, but always trying to get attention by tangentially being related to AI and getting the anxious upvotes of the "can I turn back the clock?" crowd. And there's a humorous commonality to all of these pieces that they all start with some sort of "don't think I'm not with it. I'm with it! I hand slung an LLM off the shoulder of Orion. I watched C-beams glitter in the dark near the Tannhäuser Gate." call to authority, then pivoting to the "everything has gone to shit" spiel.
We've always had a tenuously connected, barely operated network of systems. We've always had bugs and engineers doing shitty things. Every organization goes through periods of upheaval where all of the institutional knowledge is invalidated or discarded regardless.
None of this is new.
Most folks here didn't even know, but a Bitrise security outage hobbled our entire CI this weekend. Friday to Sunday night.
The security issue motivated all Bitrise customers rotating their secrets. this was a big deal for mobile deployments but didn't get much attention on HN..
Imho, the problem with AI is that management thinks that engineers can now do 10X (or do one item of work in 1/x)… and so there’s so much pressure to get things done. That’s how (tons of) bad software gets out in production
Imagine using AI but still producing the same amount of output/outcome. That would be delightful, great software could be possible… but it’s not “productive”
IMO control has been lost because we decided that moving fast is the most important thing and outsourcing everything that is not directly related to business (sometimes even business itself...) is a great idea.
The seed was there since at least a decade, AI just accelerated everything.
Correct
This is the usual reaction when people run into an uncomfortable truth. We're all bad at objectivity - we preferentially look for evidence that supports the outcome we want (or the one that's least uncomfortable). In this case it's pretty obvious that AI is better at coding than humans. Lower failure rate per task is better code. We're just shipping more of it. Nobody's claiming AI code is bug-free. Just that it's better and faster than how we used to do it.
> Just that it's better and faster than how we used to do it.
I think only one of those is true. It takes a lot of effort to get quality architecture and not-function-soup code (at least in Python) from LLMs. I can't imagine the pile of crap code bases these software factory type solutions are producing. Well, you can see a lot of them on GitHub I guess, so no need to imagine.
Further, one of the negative impacts I've seen from being able to go a lot of faster is that it translates to product teams being able to ship faster which is not always better. We've lost the "back pressure" we used to have in software development which forced product teams to consider, then consider again, then consider again as each new sprint requires reevaluating priorities. This resulted in a lot of requests stalling out and never being built that should have never been suggested in the first place. Or, to be more charitable, the delay gave the product team time to get over the initial excitement of a new idea and really consider if it was warranted.
Now we are building all-the-things and I think that's only better in some cases. I think it's proving worse in most other cases. Especially when you consider that every line of code that ships has liability attached to it.
That results in large bloated mess projects that no human can reasonably understand. That means we are dependent on LLMs to understand it and, the larger the code base, the more LLM tokens are required and the worse the LLMs are at understanding all the relevant parts and making wise architecture decisions. This creates a downward spiral that is harder and harder to manage and means features take longer and longer to ship (although, still maybe less time than pre-LLM dev...we'll see).
Do we want “faster” development? Like I couldn’t care less if my bank app or netflix didn’t ship a single new feature in the next 12 months. Users don’t need constant stream of features. But faster development is “better”. Progress and all they say. Like yeah, why I would like a car if my horse is just fine
I agree with the premise that things are breaking down, but I'm not sold yet that the problem is a systemic one.
At least some of these issues are ones that already existed, but are now being discovered by AI.
If that is the case, we would not see the exponential increase that the article mentions, but should instead see something like a temporary blip where discoveries in existing code spike, but eventually all get discovered and fixed.
I would love to see some more data that specifically looks into the number of issues caused by pre-ai code versus post-ai.
That being said, the comments about decoherence strongly resonate with me. Things are moving so fast in so many different directions right now. I don't think we know the right way to build things in this new future yet.
There have been parts of the software industry that have been mission critical from day one and have built processes, languages, and tools to make sure things are 100% reliable. Building software that way is slower and more expensive and in the past has been an overkill for most systems.
The problem software engineers are facing is that now that software is ubiquitous, almost everything needs to be 100% reliable (or close to it). This is coming into conflict with the move fast and break stuff everything needs to be done this quarter at minimal cost mentality.
Hence why we are at the breaking point.
I’ve had AI running to auto upgrade our infrastructure and it’s successful twice as much as humans. Failure rate of a human managed upgrade is 1/10, failure rate of an AI agent is 1/20 in my small set data.
Problem is I’ve had 5x more incidents. AI is operating at 10x the speed of a human so even though it’s failing less, it’s causing more incidents per month.
I wonder if people are ready for this kind of paradox where using AI may be safer and faster but less reliable for “human time”.
It feels like we are currently experiencing a soft version of the sewing machine riot.
Perhaps no one is intentionally destroying anything in this case, but malicious compliance and simple negligence can go a long way when it comes to disruption of complex systems. Destroying a sewing machine requires actual violence. An air traffic control system is an abstract concept. You don't need to touch it to break it.
Things are breaking down because they were weak to begin with, and demolition is needed in order to rebuild better, stronger. This is just a period of trial by fire: iron has to be heated and beaten numerous times (with carbon) before it becomes steel. All that's happening is AI is accelerating the evolution of industries into something which all eventually have the strength of steel.
My impression is it's always been a mixed bag. The level of bugginess has been relatively constant. In the meantime, scope and capabilities have gone way up. I hate Teams and Outlook and the virus scanner on my company's laptop though. Those are keeping me from doing my work. Windows + anti-virus makes file access and process spawning 10x to 100x slower compared to a Linux machine.
See also:
Preventing the Collapse of Civilization - Jonathan Blow, 2019
https://www.youtube.com/watch?v=pW-SOdj4Kkk
Which focuses on software, but gives many historic parallels of societies forgetting how to do things.
Previously discussed:
https://news.ycombinator.com/item?id=19945452
https://news.ycombinator.com/item?id=25788317
I think the article has a point that the if you have a very complicated legacy system, then if you just let an LLM run on this, there is a significant chance that some parts of the system will be become inconsistent with each other, and the whole thing will be more of a mess and more likely to break.
In my view, however, what is more likely to happen soon is usage of LLM to transform various parts of code into the form with much more clearly defined interfaces/rules/boundaries that can be essentially proven to work correctly. That in turn (in my view) will enable much more robust systems.
To talk abstractly about all software, you'd also need to analyze all the software that stays working. This article is just cherry picking a few recent incidents and try's to make some conclusions without really looking at causes and whether that can be generalized to "Software".
There were package managers long before npm. Maven, rubygems, etc...
Indeed I had to laugh at that.
"when tools like NPM hit the scene in 2014, it really felt like a gamechanger"
I can only conclude this was written by a young'un for whom the beginning of software history starts with the JS frameworks.
Apt and yum.
It was a very long time ago indeed that you needed to manually edit popular programs to build in your environment.
Thanks to the author. It's a good read and interesting HN comments. Nothing wrong with having concerns, and I'm sure many people share those concerns. It's scary sometimes. But, the view of the author is somewhat on the negative side. I feel that AI development is still in early phases and people or companies are discovering what works and what doesn't. AI empowers us in new ways with new problems, and new trade-offs. The old way of doing things had different hurdles. Interesting times. It's not going anyway any time soon I expect ;)
There's a lot of writing going around that we are all headed for some catastrophe, like everything has been building to this grand moment where society just crumbles and AI is the final piece.
I highly doubt this is going to happen, guys. The world has changed and software is never going to be the same. The same battle against complexity and hard things will continue to happen, but the front has moved.
Open source as we know it is probably dead. Tech companies that would create SaaS for X and get billion dollar valuations are also likely dead. DevOps is dead. Other stuff will take all these things places in time but it's going to take probably a decade. Get used to it and let's get on with it.
This reads like nostalgia way more than reality
I agree. In my life time there hasn't been a minute of software reliability. There's nothing special about this year or even this month.
I remember the first time I installed Linux in the early 2000s and was and owned within 5 minutes of putting it on the Internet. Desktop computers of any OS crashed regularly. Servers crashed and needed total rebuilds.
If anything software is way more reliable than it appears because the scale of computers and software is thousands of times larger than it was 20 years ago. More things can fail because there are just more things.
I've been using Linux since 1997, and I've never seen a Linux box being "owned within 5 minutes of putting it on the Internet". These things simply didn't happen in the early 2000s (it did happen with Windows 95/98/XP, though). The Linux kernel and Linux distributions had matured enough already at that time and surely didn't crash regularly, neither as desktop OS nor as server.
Things were less convenient and generally harder to set up back in the days (we had to fight with XFree86 configurations, desktop environments were immature, a lot of hardware didn't work due to lack of driver support, application software for many tasks were unavailable, there were barely any commercial games), but the kernel had excellent network support and scaled very well across multiple CPUs already (unlike Windows NT's SMP support, let alone the single-CPU-only DOS ancestors). During the 2000s, the Top 500 list of supercomputers was slowly being taken over by Linux clusters, and that wouldn't have happened if Linux would have been unstable. For developers, Linux was (and arguably still is) heaven on earth, and developers actually made sure that "their" system worked well for them. And it did.
There were many difficulties with software in the past, but Linux was rarely one of them.
> I've been using Linux since 1997, and I've never seen a Linux box being "owned within 5 minutes of putting it on the Internet".
Absolutely happened.
But then I went on to do Linux hosting and co-location for the next decade.
I don't see why you felt the need to respond. Linux is almost certainly more reliable and secure now than it has been historically. But software continues to grow and get new features, so there will always be new issues. Linux is not a special snowflake in either direction; it's software made by humans.
I write code with AI daily and it got much faster. If it brakes - it gets fixed then. We face new vulnerabilities, but we find them faster and have better instruments to investigate and fix. The current period is transitional, but for me it's obvious that it AI is a win for software development
The author seems to have glossed over the viruses of the early 2000s,or the Y2K panic, and other incidents from before. Building software has always been hard and it's always been abstractions on more abstractions.
The one compelling part is that now we might be forced to pay for some of the rickety towers that we've erected because AI has made exploits pretty cheap to try.
I think the author is pointing to something fundamentally different than software being hard.
Like before, individuals in an organization had various interests, motivations, connections, knowledge, context, that allowed our organizational brain to function properly. Now, everyone is (or will be in the future) a meat proxy, isolated, and disengaged which produces an organization with neurons but no neural connections: apparently healthy but, as the author put it, decoherent.
> You can find programmers who know how to manage the complexity, but they want to be paid lots of money.
This wasn't always the case, by the way. It started with the "dot-com" mania and VC investment in the late 1990s. Before that, a programming job paid decently well but it was on the level of other cost-center professional jobs such as accounting. A comfortable living, but nothing extravagant.
I mean all of the failures that happened have happened in the exact same way many times before, default passwords not changed keys signed with 5 and 7, we have seen all of this before, and the FAA issue, was it even software, I have not followed the coverage, but its systems setup 20ish years ago and forgotten.
Sure the MS thing might be vibe coded horri, but it's not like we have not seen outages like that in the past.
If anything AI is bringing all of the bad configs and legacy issues to the fore front, because "it" has the time to look for them.
Software isn't breaking down, people just haven't figured out how to use AI yet.
It reminds me when I first started work on a codebase with proper unit tests: I thought I could ship whatever and the tests would catch it. That was incorrect; while unit tests are a guardrail, I still had to properly test my code.
Similarly, people need to learn how to use AI and make sure they don't introduce bugs or slop into the codebase.
But this brings up a larger point for me: most people are grieving the loss of what their jobs used to be, and denial is a core phase of grief.
I went through it too. I tried to find ways to say AI was silly or wouldn't work. But now it's inescapably here and we need to work in the new paradigm. There are plenty of valid criticisms, but it's already very good, it's getting better, and you have to learn how to use it as best you can.
> About ten years ago, two small revolutions happened in our industry ... And then, we had the Open Source boom.
This seems a bit myopic. Or naive. Open Source "happened" a lot longer than 10 years ago. Microsoft didn't shit itself about Linux just 10 bare years ago. And it's not like prior to 10 years ago, the narrow focus of npm world wasn't cobbling web sites together with bits and pieces garnered from wherever.
That said...
> But AI is both a cause and a symptom
This I agree with 100%. The rise of AI and necessity of AI to continue to make progress is a massive indictment against an industry that prioritized more features and uptime over solid first principles for years, taking out promisory note after note in complexity, in hopes that the stuff papered over would quietly just fade away.
I don’t need to read these articles, it’s enough to read the headlines. Software isn’t breaking down, it’s accelerating, obviously.
Software was never engineering. It’s more like writing a novel, which is way harder because the success criteria is ambiguous.
Now some folks are finally being forced to understand this reality.
He obviously is preparing another doomsday conference talk. Nothing interesting to see here.
On the other hand software is improving dramatically at a systemic level, even without safe system languages arriving on the horizon.
> Most SWEs now are unable to tackle engineering problems without some form of AI assistance. That’s not because they’re incompetent - they just can’t keep on top of the complexity. Even if they could once, they now face a tsunami of AI generated code.
That's called outsourcing, which inevitably enables "vibe" effect of the coding: Your AI goes out of its constraints and does totally irrelevant shit you didn't ask for or does something barely passable you could approve, without knowing further consequences because you outsourced your thinking and control over the project.
To try to find a remedy for this shortcoming, they invented "skills", which totally negates the purpose of an AI (LLMs, genAI, whatever you call it to describe the current tech) IMHO.
It's also funny reading these comments. Most say "well, it was already broken" and they continue trusting AI more. Well, AI is not exempt from this "brokenness".
People keep believing that the big LLM players with somehow disprove Rice's Theorem. "Somehow we'll be able to get AI to check if a program does X or Y." "We'll be able to have the machines verify things automatically." But this is is of course as hopeless as solving the Halting Problem. Theoretically, we know this is impossible.
Empirically, we can observe the breaking effects of this broken thinking. Code quality goes down, and apps go from being 99% to 95% reliable. But that's not a big deal, they say, that's good enough... But when the quality of all the building blocks further down get degraded in the same way, you can't keep track of which parts are broken, and which parts are dependable, and the whole thing comes crumbling down.
People keep saying, "I can build an app with AI, therefore we can build everything with AI." This article talks about some potential anecdotes that could help to show that doesn't work.
As the sibling says, the problem is not really Rice's Theorem. We were doing some great progress into software reliability before the LLM-crazy destroyed the industry.
The problem for LLMs is less formal and way more wicked. We don't have any good theory of software reliability. At all. All the reliability we have is artisanal and designed ad-hock.
Rice's theorem is across arbitrary turing machines. We intentionally limit the kinds of programs we emit with type systems, knowing they will prevent us from writing some programs, but we'll be able to analyze the ones we can write much better.
We prove properties of programs all the time, that's what a type system is for.
Can we prove all properties about our program that we're interested in? Maybe, maybe not. But Rice's theorem is not the tool for that argument.
Are vibe-coded apps being being written in Rocq programs that perfectly match dependent types to every possible behavior? Is anyone anywhere writing software with type theory at the level where it perfectly limits things and saves our programs from semantic mistakes?
> every possible behavior > semantic mistakes
Of course the point is not whether we can now do this with AI, but whether AI can help us get the same quality we used to get, faster.
Also, consider that it should rarely be the goal to enumerate and judge every possible behavior of a program, with exceptions for life-saving or otherwise critical software.
Many of the recent maths solutions are just a lot of Lean code (types + guarantees).
Is defect rate increasing or is the amount of, and use of, software increasing? What is the defect rate relative to that?
Honestly there are 2 overlapping problems:
1. go fast and break things became the norm not the rare exception, mostly due to under-regulation when it comes to responsibility and other much more complex side effects of what you can describe as the US forgoing a fair free marked in favor of out of control monopoly markets they can use as soft power against other countries
2. software being really hard and most tries to fix that being stuck dead before maturing due to them being unfamiliar and often complex to get started with. There being a huge problem with opposite bad ends of "misguided simplicity" and "programmers loving 'clever' & overengineered" didn't help exactly either.
> It wasn’t always like this.
no it was, yes the 2010 felt like a high time. A lot of 90th nonsense has finally been left behind (but would still lead to security issues years later, e.g. log4shell in 2021); Tools for analysis had gotten better; Compute allowed expansive test suites and "go fast and break things" hadn't yet been spread to widely to critical infrastructure (at least kinda, _a lot of the issues with Azure today root back to decisions done in the 2010th_!!).
Most importantly while having well educated highly skilled Devs is needed, _it isn't sufficient_. They still are human. Blaming the huge flood of CVEs on AI generated code is simple, but wrong. Many of the CVE in recent years are human caused, some older then usable AI code gen and at least in case of the Linux kernel much of the code of the past had been written and reviewed by pretty professional, skilled, well educated developers and still AI found way way to many CVEs in it.
So yeah IMHO there is a long standing fundamental issue with how to handle software production, starting with the very first step of requirements engineering (which often isn't done, or at least not properly) and ending with how we fundamentally compose software components.
And the problem I see with AI it makes it worse. We already where approaching a breaking point with software dev and I hoped that this would create pressure to finally find a working way to reinvent it from scratch. But now AI allows us to apply the same failing approaches even more large scale, faster, cheaper and sure with more bugs, but also with AI making many bug fixes far faster and cheaper (if correctly used, it often isn't).
When I first started my career we used to ship like once a month and that was considered good, everything was extremely convoluted and worse than it is today, whole segments of the industry didn't even use Version Control, half the government websites you used stored passwords in the clear and emailed them to you for resets, e-commerce was a cgi script that sent an email for fulfilment and it took weeks to get your shipment, sites went down at night and weekends for maintenance regularly, servers fell over with traffic that would run on a raspberry pi now, but for some reason everyone keeps saying how bad things are now. They're delusional or don't actually remember the past.
Not everyone is using AI to make shittier software. Most big important software projects I know of are frantically fixing an avalanche of AI-found bugs. Important software is getting more robust and resistant to attacks, while still adding functionality.
There's a lot of slop, and it will almost certainly grow quicker than non-slop, but there is a big opportunity for those of us who still work at the bottom to make the pillars more solid, and take on re-engineering projects that would be infeasible otherwise.
Case in point: Before Claude a port of my compiler to a new architecture might take anywhere from months to a year. After AI, now that there are multiple backends as examples and an even larger--massive even--test suite, this can take days or less. It took Claude about 3 days to come up with an arm64 backend for Virgil (around April time frame). Then in August it took just 3 hours to build a complete RiscV backend. Both of them pass the entire test suite. And they shook out other bugs elsewhere. Overall, more targets, and more robust.
Things can have rocket boosters, you just gotta know where to place them.
On the flip side, once you hit rock bottom the only way is up.
> I believe that tech companies risk losing their “organisational cognition” - their ability to think and respond as a coherent unit.
LMAO. They can't lose something they never had in the first place.
This article has other pearls like this:
> Reddit and Bluesky speculated that even MSFT engineers had no real handle on the complexity of this outage.
Like, with all due respect, what the fuck do they know?
Also this:
>It’s the software that controls your flights, your banking, your petrol pumps, your TV, your payroll.
Dude, how are you gonna sneak TV software into the same sentence as flights and banking? :D
This is not a serious article, and I don't recommend reading it. But to finish on a positive note, since I've already (regretfully) read it, here's a sentence I like:
> This is the real threat: a scenario where nothing in the system is done with intent, and nobody in the system has control. We generate complexity faster than we can tame it.
I would just add that this was the case long before AI. And also, just as AI generates complexity at such a great pace, it can just as easily help you fix it and fight that complexity.
All software has bugs, and some software is critical
Which means all critical software has bugs!
There's also more of it! Even more once you finish reading this sentence. If security incidents scale sublinearly vs the growth of critical software, we should be proud.
Crappy work is not universal either. There are engineers out there, guiding agents diligently, doing a good job. Never letting an agent go wild unattended, writing the same code they'd have written themselves, but with more tests and more better assurances and more docs and better infra and and and.
The problem that leads to blog posts like this is that good, diligent work, like critical software that does its job, are invisible.
Because no one notices a building that's not on fire. They just walk right by.
Who could have forseen this except literally anyone with a braincell
"Decoherence: the accumulation of locally valid but mutually incompatible or redundant models within a system, such that the cost of change is dominated by translating among those models and preventing unintended side-effects."
This one is interesting. Not on its own merit which is more like a mother of facts, prime phenomenon, but that it was mentioned at AI. I had this feeling of decoherence with all the various techniques and approaches that existed out there, from programming languages through data management through composing instructions through user interactions, into the way orchestration of all of these, way before AI. Somehow none were matching the specific composition of products needs completely so had to tape together available parts at loose couplings, hammering things a bit if needed. Or choosing those that fit together perfectly on the cost of giving up on requirements/functionality for the product. Lots of good libraries not good enough for some details so a new one emerged or the existing one tried to be so many things that it reached digital schizophrenia stemming in complexity (allowing flame or even religious wars between developers about best/sacred approaches/techniques/mentality/...).
With AI I feel different: the promise of purpose made things fitting together well.
> Companies loved Cloud because it allowed them to remove sysadmins and ops people from the equation.
um what? who? where?
Most people here are dismissing this article just because it doesn't use variations of 'fuck' 50 times or throw in emojis to make it 'fun to read.' And as always, decent comments only appear at the end of the page.
If you want high quality results, you need skilled developers who know their business. It's that simple.
I could stop writing at this point because that's all you need to know to understand why "software is breaking down".
Anyway, the problem is just that there were never enough skilled developers while the demand for software was ever growing. That's the reason why the software industry has always been crowded with unskilled amateurs doing grunt work. Amateurs were tolerated simply because of the lack of highly skilled developers: one good developer can easily replace 5 untalented code monkeys, but it's better to have 5 untalented code monkeys than 0 good developers so that at least something gets done. Programming languages like PHP or JavaScript lowered the bar, attracting even more, even less talented people. It really started going downhill during the dotcom bubble, well before cloud computing was a thing, and it never stopped going downhill.
In such an environment, if you want quality results, the amateur work requires supervision and strict quality gateways so that good code can be distilled from the bad. That is, code monkeys must be managed and supervised by more disciplined, more skilled, more talented developers who are actually capable of understanding and managing the complexity of software.
These days, however, everybody is being dumbed down by AI. Everybody even seems to enjoy being dumbed down by AI, even those who actually are (or rather, used to be) good developers. Instead of doing proper code reviews by a professional, an AI is consulted to do the review. Instead of planning a software architecture, people are vibe-coding. Instead of doing a simple change that actually works, an AI is asked to generate spaghetti code (and you don't recognize the spaghetti because you don't understand your architecture anymore). Good practices that served as safety net against low software quality are ignored because "AI doesn't need them".
Of course software is breaking down, now much faster than ever before! It is because we (as an industry) are moving at neck-breaking speed and have stopped thinking, even those few of us whose job is to keep the quality at least at an acceptable level.
Spaghetti Claude is killing quality faster than most can realize, and it shows already. Your amateur code monkey crowd has been replaced by even less talented, but much faster AI with a knack for convincing language. Your code monkey crowd is still there, but it is now instructing the AI to produce results the monkeys don't understand and push to production. Quality gateways have been dumbed down or simply removed to keep up with the speed.
It's a bad development that needs to stop.
Good engineers have mentally checked out; some great engineers have already left the industry entirely.
Software was mostly always broken but it was hidden and invisible, now it’s visible because the cost of making it visible is so low
This is classic surveillance/detection bias: https://healthjournalism.org/glossary-terms/surveillance-bia...
It’s basically free to black/grey/whitehat penetration test anything and everything without having to even know what ghidra or metasploit so everything gets penetrated
There is one root cause for all major software system issues: business doesn't give a shit.
There isn't a software system in the world for which we can't predict the failure modes and make solutions for. You simply ask two questions: 1) "How can I tell for sure this thing is working?", 2) "If it doesn't work, how do I fix it so it does work?". You might not know the answers at first, and that's fine. The business's job is to ask the questions, do all the work to figure out the answers, and then test & implement the answers. If the business does this, they prevent most of the failures, and have a quick deterministic solution for when eventual failure happens. If the business doesn't do these things, they are choosing to let it fail in an unpredictable and likely very expensive way (they don't give a shit).
This process has a bunch of names. Failure Mode and Effects Analysis, Business Continuity Planning, Risk Management, etc. No matter how you slice it, it's the same basic work: find out what could go wrong (at a high level) and have a plan to fix it when it does. It's not always easy, but you can always figure out some kind of plan that works.
For all the security and reliability issues we've seen lately, there are known methods to either prevent those problems or fix them quickly. The incidents aren't happening because something new is going on. They're happening because businesses stopped giving a shit a long time ago, and the chickens are coming home to roost.
> business doesn't give a shit.
100% this. 30+ years in tech, this is always the source of slop. Faster, cheaper, and cheaper again are always bigger drivers than the engineering.
the 4 events (Php, emudhra , MS outage etc) were coincidental. Two of them were mostly theoretical . In the PHP case, despite the tremendous potential threat, the recorded damage has been relatively minimal. Everyone expected to see broad and severe compromises of PHP assets, akin to Y2K’s potential – and very little has happened.
A better narrative is that software quality has been very low for some time, and we are lucky that the threat actors have not compromised the assets as broadly as they could have been .
We have a great opportunity with AI to patch up the duct tape and popsicle sticks. Moreover, to realize that consolidating authority into central management with synchronous coordination is terribly brittle.
In other words, we can fix software by dismantling some of the dependencies.
Every chud on here thinks they're Howard Roark or Dagny Taggert but they're really just Peter Keating with a CS degree.
Yeah, yeah, it's all misunderstood and the timeline is wrong and it's always been this way and and nitpick this and nitpick that but: I believe this is the way the "vibe" is shifting. The average person is going to guess that AI caused all the weird outages in the last couple weeks, corporate and Trump admin denials notwithstanding, and I suspect something like that is what the history books will record. If you are all in on this technology and surprised at these attitudes (or at the "blue wave" in the US, or at scandals "suddenly" dropping on Jensen Huang) it is a good time to look around.
I’d argue this is a consequence of a lot that’s gone wrong in the industry. Maybe there was never a golden age, and I didn’t live through whatever there was, but there used to be real power in the craft of software engineering itself. Over time that power was eroded, co-opted, and sold off. There’s no single event to point to, just an accumulation that’s left us with top companies that operate like cults, people entering the field to chase status, and software that’s less the product of the craft and more a side-effect of dealmaking, politics, and greed. It isn’t true everywhere, but watching most of the talks given in the field now feels like touring the circles of hell. Everybody just seems blind now.
Ultimately, it was inevitable. What I'm saying is just the status quo for centuries and it was a matter of time before software was brought into the fold. There is nothing intrinsically moral about the field, just that respect used to be necessary before structures were created to launder that respect. The good stuff, like most things in life, are on the edges, but the edges don't stay edges forever i.e. see this forum.
Well there was this saying "Do One Thing and Do It Well", seems once again we are seeing the affects of ignoring that.
It's nice to say, "Oh we did it better in our day! You kids lack discipline!"
But look, the reality is we copy-pasted just as much slop we didn't really understand to get over the finish line back then as kids to today.
Deadlines were missed. There were bugs. Things broke. That's just life.
Now, do I wish we had TDD for everything? Yes. Do I wish there wasn't quite as much emphasis on AI? Also yes.
But there's a literal mountain of slop code that was written over the last 40+ years. Without AI, it sort of just sits there... people are afraid to touch it.
With AI, and I mean me using AI, I'm comfortable diving into a massive legacy code base and refactoring pretty much anything at will. And it's pretty cool when you stop looking at code as "I don't want to be the last one to touch it, because then I'll get blamed if it breaks."
And instead take the approach, "OK, without proper maintenance, this dies. How can I give it proper maintenance?" And look, if something breaks along the way, "Meh." I can fix it. Those old code bases were often just as buggy and duct-tapped as anything AI created... worse... years upon years of multiple developers doing patches. Convoluted AF.
I still like to think I have good discipline. I have a repo that has all my coding standards, and that's a good starting point for most of my projects. I weave in a bunch of rules as I go, things I learn, and that repo becomes the guide for the AI to do work on the project I'm working on. Then... when I'm not the one doing the work, it'll still be there to explain to the next person (or rather AI being managed by a person) why I did things, and what the goals were, and how shit works.
Most things that break... it's just because it wasn't documented in the first place. I find a lot of that... maybe they had discipline, and maybe the legacy systems that had the documentation were lost to time. Sure. Let's give them the benefit of the doubt. But... I'm guessing they did quite a few patches with just someone calling, "Hey Swanson, make the button blue, and make it say 2+2=7 to impress this client!" Right, so there's still a lot of that today too.
> At its worst, AI turns tech workers into “meat proxies” who spend their days massaging prompts and letting Claude figure out if the work is actually complete.
I am going to claim, AI is at its worst. Software companies are doing mass layoffs, they are decreasing the quality of software and reeking the profits, making sure only upper management (or worse, shareholders) are benefiting from software becoming so much cheaper and shittier.
And software workers are not loosing our jobs equally. Those that don‘t want to use AI are being fired much faster than those that do. And those that remain are de-skilling them selves, fast, letting AI do all the work for them.
I want to blame AI, and I want to blame capitalism and the profit motive for making sure that AI would inevitably cause as much damage as it possible could.
It's because amateurism has been hailed as the second coming of Christ.
Leftpad, mentioned in TFA, is a case in point: not because of leftpad itself but because of all those defending leftpadism.
"People can't write a proper leftpad".
Yeah, well... If a dev cannot write a proper leftpad, then it should never have been a dev in the first place.
Someone who cannot write leftpad is not qualified to use leftpad as a dependency (and, no, leftpad is neither the Linux kernel nor a cryptographic libray: it's fucking leftpad, which is the entire point).
We've had "devs" whose only qualification was writing glorified "punch the monkey" ads in JavaScript: the JavaScript surveillance/ad complex. Devs who only ever knew the mono-threaded JavaScript ecosystem and suffered from heavy leftpadism.
And those very same surveillance/ad complex bros are now generating countless crap with LLMs, in languages they don't understand.
They're just mediocre.
That's what happened: the veneration of mediocrity.
You're repeating points that were brought up in the discussions about leftpad a decade ago, but the root cause wasn't venerating amateurism (even though the everyone can code / software bootcamp trends of the time was a by-product of it); it was the veneration of velocity. Which, I suspect, is inevitable in software because it's so powerful, low-resource intensive, and lucrative compared to other engineering disciplines. This technology was always going to create an industry focused on speed above everything else.
I think the exact opposite has happened: needless gatekeeping and NIH-syndrome has created this idea that software development is a skill that measures intelligence.
Because of this, battle-scars from the rough edges of software are seen as part of software-competence, not as a failure of industry. Novices get met with 'you're holding it wrong', but at the same time veterans lose the plot and spend most their time doing technical onanism, or worse, fighting meaningless complexity.
Until the stack exceeds any individuals understanding, and the cracks start to show. Leaky abstraction upon leaky abstraction upon layers and layers of ducttape, under the guise of HA, distributed system mantras, robustness, you name it.
Every part needing babysitting because every part is half the solution to half of your problem, but admitting that is seen as a skill issue.
Yes my DB dies because it caches stuff and (under obscure circumstances) never frees it: 'thats just MySql, deal with it', yes of course my frontend framework needs to be kept up to date every week because everyone is evalling strings all the time, 'don't you sanitize your input?', multiplied by every layer of the stack.
Don't you let your static analysis automatically merge security PRs so you can stay on top of this giant heap of string evals? No? That's best practice man!