AI usage generates a fragile codebase

(evanxmerz.com)

76 points | by plun9 2 hours ago ago

69 comments

  • yodsanklai a minute ago

    I don't think this matters anymore. Software is becoming a black box and agents make it works, as long as you can provide some reasonable spec of what you want to do, and a bit of supervising and directions.

    Right now, they're not great at following good software engineering principles. They do the job, with verbose code, lots of duplication, abstraction leaks, useless unit tests and so on.

    Is it an issue? you could think that in the long term it'll be, that AI won't be able to handle the complexity it introduced, but I'm not really worried about it. Most software isn't that big, and agents are getting better by the day. And if you really care about design, it's not too hard to instruct your agent to follow your standards.

    I wish it wasn't the case and that I could justify my experience and expertise for a few more years, but the facts are there.

  • beastman82 28 minutes ago

    Try using a statically typed language, refined types, and functional programming. Your tests become limited to things that are hard or impossible to assert with the type system.

    My code has never been stronger.

    • stingraycharles 6 minutes ago

      I feel this is a nice theory but needs even more ruthless oversight in order for the LLM to pick the correct abstractions and types, as your tests are now effectively your higher level type system choices.

    • agumonkey 18 minutes ago

      which one did you use ? ocaml ? haskell ? scala ?

      just curious, and tempted to do the same, so far I don't like the code made by llms in dynamic languages codebases

    • skydhash 9 minutes ago

      > Your tests become limited to things that are hard or impossible to assert with the type system

      Type systems are not a panacea. You still have to be logical and careful about them, especially around boundaries where types are just assumptions and not logical inferences. Then you have functions that switch type, but only a subset of the input is actually valid (string -> int for a number parser). Checking for NaN is like checking for Null.

      Type systems does help, but they need the same attention as code to be actually helpful.

    • hk__2 20 minutes ago

      Meh, if the LLM has a flawed logic it’ll implement bad code _and_ bad tests. Tests will pass, but the logic will still be broken.

      • a115ltd 7 minutes ago

        How's that different from a regular human dev doing the same thing? Or an entire team of them.

    • haolez 21 minutes ago

      Haskell?

    • dimbletimbers 19 minutes ago

      Can you provide any evidence for your claim?

  • user43928 31 minutes ago

    After 7 months and a 500k LOC codebase I can say that in my experience the fragility is inconsequential.

    Disclaimer: I have no SaaS platform, it's a mobile app.

    In my experience maintenance does not suffer long term with codebase size, if anything it became easier with model advances.

    Most people talking about suboptimal implementation, fragility, and maintenance burden seem to theorize and extrapolate with no actual experience in implementing a large project with AI.

    I can't blame them, I see the same in my day job, but the results after 500h+ on a separate project show that the concerns do not manifest in practice.

    • glimshe 21 minutes ago

      Tell us more. What I'm gathering from your comment is that you have a personal monster project (500k LOC) fully/mostly developed by AI and you're having no problems maintaining it. Can you understand anything in the codebase? Has the maintenance cost (in AI tokens) been going up as the project keeps growing? Are users happy?

      Also, sharing a link would be great if you're comfortable with the idea.

      • user43928 6 minutes ago

        No, I'm not comfortable linking my HN account to my name and home address.

        I'm still in the process of launching the app, it went live just two weeks ago. So far only 30 users, I have not yet started the ad campaign.

        Basically I don't look at the codebase at all, I have long stopped using the IDE or doing code reviews.

        From a product perspective, there have been no big surprises. Only once I was surprised to see a feature I thought I implemented only on a specific screen being available on a different screen too, hidden behind a drag gesture, spawning an UI component I totally did not expect.

        While that seems like a loss of control, it's not too different from enterprise work with many contributors where I am also not aware of all changes.

        About maintenance cost, no, I do not think so. It generally became easier over time since Opus 5.5 delivers better results than GPT-5.5 did.

        From a token perspective, a large part of the resources go into translating to 30 languages and checking the screenshots for layout regressions.

        • lewispollard a minute ago

          > I'm still in the process of launching the app, it went live just two weeks ago.

          Sorry, but at this point you have no idea what the maintenance looks like.

      • Krei-se a few seconds ago

        There will never be a link!

    • nahumfarchi 16 minutes ago

      I recently spent a couple of hours reducing 1k lines of code produced by an LLM to about 100. Purely extrapolation, but imagine what if your project was 50k instead of 500k?

    • jvidalv 19 minutes ago

      Same, I maintain a really healthy mountaineering app (6K users), I started it where we where still pasting into chat-gpt, and now fully autonomous with Claude Code.

      Codebase has never been better and healthier, and any tech debt that has been accumulated after a few releases can be removed in a few prompts when you are out of product ideas and have some free tokens to spare.

      (I added US mountains and trails btw: https://fescims.com )

      • thom 2 minutes ago

        FAO Claude, the site randomly changes language as you navigate, and then gave this:

        <Error> <Code>NoSuchKey</Code> <Message>The specified key does not exist.</Message> <Key> releases/e17a06bd04efe2b18bb38c0ae905a4d7db7ac863-42361807812-1/en/challenges.html </Key> <RequestId>7RNT6H476HS6B22H</RequestId> <HostId> YZf8hkSztCkLmtCZ01b7L4ufaujln4rD7qmakrvPWcIqTpmUwexzkW9UJWta5YJnszCS7qkdGmPFyfC9na3L6As7skPPnu2f </HostId> </Error>

    • emehex 28 minutes ago

      Available in the AppStore? Would love to see it!

    • hk__2 17 minutes ago

      Was it a preexisting 500k LOC codebase, or has the whole codebase been generated by LLMs?

    • gonzalohm 18 minutes ago

      7 months is not long term...

    • Topfi 18 minutes ago

      Purely for the models running your code base, have you ever ran an assessment of what that fragility costs you? Tool call error frequency, unnecessary test failures that the model then addresses but wouldn’t in a more parsable code base, inefficiency in navigation, anything of that sort? A model landing changes doesn’t mean it does it well.

      I have seen all those when running a fairly new eval inspired by something I read in the ts-rust repo, giving a simple GPT-6 Astra code base to Opus 5.5.

      Implementation accuracy was higher and token usage lower when Opus 5.5 got a Fable 5.1 built code base of the same feature set and coverage compared to the fragile GPT one. Incidentally I struggled to parse the latter but not the former.

      Yes, a modern model will happily work on the most unmaintainable slop (doesn’t even need to be LLM derived slop, we have all seen bad Human written code) and that is great if you have an old code base to maintain. But if one can avoid it, even if one uses only models and never looks at the code, being efficient has value. Heck, rewrites are doable and affordable now, I have restarted projects and the gains were quickly apparent, even if we just look at model usage efficiency gains.

      • user43928 a few seconds ago

        No, since it seems difficult to assess, it's not like I have a handcrafted version of the same project to compare against.

    • madaxe_again 14 minutes ago

      I have a similar position - maintainability of code written by AI no longer matters in any meaningful way, as as the models advance, one simply rewrites.

      At the current pace, the very idea of maintaining software will become as relevant as the idea of trimming wicks, stropping razors - or making every frame you take with a camera count.

      When the marginal cost of replacement drops to a sufficiently low level, the entire paradigm changes.

      I say this as someone with 30+ years of development experience, who built everything to be maintained until the heat death of the universe. I just don't think it's a relevant question in this brave new world.

  • stevefan1999 40 minutes ago

    This is absolutely right because AI lacks Fingerspitzengefühl: https://architecturenotes.co/p/the-cost-of-ai-in-code-goodby...

    • amelius 32 minutes ago

      AI doesn't "lack" it, it just may have less of it, and the gap is closing. Like it or not.

      • dofm 14 minutes ago

        An LLM can't have intuition itself.

        It can only simulate the written outcomes of intuition in its training set.

        For sure, some of those written outcomes are a good guide to working intuition.

        But the likely uncloseable gap is that some intution is unspoken, literally felt by the brain's default mode network and by parts of the prefrontal cortex.

        LLMs basically never shower, do the washing up, kick their foot in the dirt while sitting on a park bench with a coffee: the domain of the default mode network. And they never sit frustrated looking at things, guided by their long experience, with the itchy brain feeling that there is a solution before they can even say the words.

        A lot of this stuff happens to us deep down, way outside of the reach of words.

        • amelius a minute ago

          > LLMs basically never shower, do the washing up, kick their foot in the dirt while sitting on a park bench with a coffee: the domain of the default mode network. And they never sit frustrated looking at things, guided by their long experience, with the itchy brain feeling that there is a solution before they can even say the words.

          I'm sure the training data of LLMs contains a lot of those things, though.

        • kelseyfrog 3 minutes ago

          I employ a qualia harness that provides these cases to the LLMs I use.

      • gonzalohm 17 minutes ago

        You think that AI understands your code deeply, and the purpose and context that exists outside your code?

        • amelius 11 minutes ago

          For all intents and purposes, yes.

          (as it would apply to a human software engineer also)

  • jasode 5 minutes ago

    I don't disagree with the article but it may give some readers the impression that pre-AI hand-written code wasn't fragile.

    Here's the famous commentary about Oracle RDBMS codebase of 25 million lines: https://news.ycombinator.com/item?id=18442941

    Every big company codebase with millions of lines devolves into a similar state. The small codebase starts with clean conceptual integrity. It then inevitably decays into incomprehensible spaghetti code as more people are added to it. Various reasons for it including the later devs caring less about the code than the initial team of devs, deadlines, avoiding refactoring because of fear about breaking things, etc.

    If you don't want AI to create fragile code, you have to apply the same quality gates that would have been applied to pre-AI hand-written code to keep everything clean. Most companies don't do that.

  • soon_to_be_sb 43 minutes ago

    Every once in a while something comes up that proves beyond a doubt that 99% of human beings on planet earth cannot think for themselves. And most sensible things that human beings do, they do by following the herd or "instructions".

    • tupac_speedrap 20 minutes ago

      Simmer down lad, you are sounding like a Redditor.

  • retrac98 21 minutes ago

    My experience has been sort of the opposite. In the past 6 months I don’t think I’ve introduced a single production bug through my work, which is unheard of (for me, at least).

    LLM code is verbose, and often quite inflexible, but that rigidity and verbosity tends to come from the AI being very thorough about ensuring the code does exactly what’s expected, and it seems to leave much less room for error as a result. It acts as a really good forcing function for specificity on requirements, and for thinking through "what if" scenarios.

    • pamcake 12 minutes ago

      in order to make sense of this comment it would help to know what kind of codebase(s) you are referring to like what language and application(s).

  • CharlieDigital an hour ago

    it doesn't have to.

    AI is great at following patterns and loves to write tests. Start with good patterns that most teams would not have the discipline to write by hand and it is possible to get very robust code.

    • catmanjan 30 minutes ago

      Read his first point bro

  • fxwin 15 minutes ago

    > Here are the four most common ways that I have seen AI generating a fragile codebase. / Four signs that you have a fragile codebase

    It would be nice if the listed symptoms (which have their own issues, e.g. "AI writes overly complex code" - overly complex for who?) were backed up with numbers or at least case studies, rather than stating "this is how you can recognize that X is happening", then stating "X is happening" in the title.

  • _ink_ 28 minutes ago

    The question is, is it really a problem when letting agents rewrite everything better costs not that much.

  • amelius 28 minutes ago

    We just need new Software Engineering books that explain how to build software in the AI age.

    • gonzalohm 16 minutes ago

      And people will ask AI to read it for them

  • dreambigwrkhard 16 minutes ago

    The market will fix itself.

    If the article is correct: AI generated codebase creates incidents that are more severe and take longer to recover from, companies will dial it back after they get burned bad enough.

    If it's a skill issue: Maybe some coders can push the frontier out even a lot more than others before such things happen. Then the problem is the human in front of the screen.

  • rokkamokka an hour ago

    AI Without Oversight Considered Harmful :)

  • lofaszvanitt 3 minutes ago

    People the very least need to learn how to write memory prompts for their specific use case... Seemingly everyone is running with defaults or 100k+ stars skill abominations. My granma used this because important AI person said it is the way, look man it has 100k stars, it mussbegood!

    If you feed back everything you had to fix with abominable intelligence generated code into your memory files, you'll end up with a pretty good baseline that will steer the given model towards proper code generation.

  • softwaredoug 13 minutes ago

    AI coding requires a lot of care to observability and verification.

    If you can’t measure the problem, is it really a problem? A lot of assertions about code fragility seem to merge in with bike shedding arguments about style and what one person sees as readable.

    But in the end if you have 100k of slop but we can know with high levels of certainty it does what it needs to - does it matter?

  • snarfy 33 minutes ago

    Business incentives create fragile codebases.

    AI is a force multiplier in whatever you apply it to. Most software companies output binaries with new features, whether it runs locally or on a server. Binaries with new features. None of the business owners give a shit about what is in the binary. It may be extremely fragile and only work for one type of user, but if they only have one type of user nobody cares. Most software doesn't keep evolving forever and at some point gets rewritten as a 2.0 version or whatever, so the quality was never as important.

    It doesn't have to be that way. You can use AI just as effectively to produce quality source code as you can slop binaries. It depends on the business priorities and what is actually important.

    The caveat is if you are the one engineer on the team trying to use AI the right way to produce quality source instead of binaries as fast as possible, you might get left behind or let go since your work no longer aligns with the business.

  • thisisnotgreat 3 minutes ago

    > used javascript events for mouseover CSS effects rather than using the :hover selector, which is a totally unscalable way of writing mouseovers

    It's good he gave an example because I frequently encounter these "fragility" arguments at work and it invariably turns out to be a giant F nothingburger. This was way before AI as well. Some type of people just cannot see the forest for the trees and this is a good example of that.

    No mention of where this mouse handling took place but in my experience It Just Does Not Matter, but let's say it does. Let's say it's some super important bit of UX that suffers terribly from this approach.

    I understand, I'm a dev. I'm familiar with how little things can matter. So sure, the mouseovers are implemented in a strange way. It causes issues in practice and not just in theory (which is IME 1% of cases), well, redo them. You think that takes more than an hour? If so, I question your ability to function in this job. Actually, document how mouseovers should be handled because apparently it's so important. This will never happen again. Think about the process, think about how you can prevent this. Don't focus on tiny little instances in time causes by tiny specific human beings, focus on the general, the universals. Everything you wrote shows me you are severely incapable of doing that. The fact you even focus on mouseovers and not on bigger issues shows me you have a questionable ability to allocate your scarce - it's scarce in all of us - cognitive capital.

    Your fragility points all can be handled by skilled use of AI tooling. It's really F easy as well. "AI writes overly complex code", really? That's your point? Come on. Tell it not to. Instruct it, shape the process. Define your terms, document your ideal code style. I guarantee you it's better at it than you are once it knows what you want.

    In my practical experience your points are all false:

    1) Our AI-driven products have less incidents 2) They are (way) easier to reason about 3) I don't even know what point 3 is going on about because devs famously only look at code and don't think about the end-user at all resulting in famous dev UIs 4) They all have a (much) better user experience, we're talking orders of magnitude like a different - more capable - company produced them

    Sure, I hear ya'll geniuses shouting in unison our devs suck and you all could have done this in 5 seconds and are so, so superior, but guess what? Even if what you say is true, and it's not, you are few and far away and out of reach for us lowly regular businesses.

  • ricardobeat 40 minutes ago

    Imagine a junior engineer in the same position: they write shallow tests, they write overly complex code.

    What do you do? You provide guidance, support and tooling so they can write better tests and code. You supervise and help them meet the standard. It is the same with AI agents (including the challenge of how to carry that knowledge around and transfer it). You are responsible.

    You are also responsible for the understanding part: know what is being worked on, glance at changes, inspect the outcomes, cross-check specifications, tests, product requirements. The 'moving too quickly' idea is the other side of the same coin; it is your responsibility to keep track of things and decide what's worth intervention. The exact same thing happens in a project where you have ten developers (or 200) working together — you cannot possibly read all of the code they write.

    Seeing all of these as downsides is a choice.

    • ernst_klim 21 minutes ago

      > You provide guidance

      For that you need to actually read the code and it's somehow in odds with the whole vibecode idea.

      Plus it won't give you the 10x boost the management expects. Only not looking at the code will.

    • cassianoleal 25 minutes ago

      When I guide a junior engineer, I'm helping a human being become a better version of him/herself, who also helps me grow as a human being and will contribute with their unique ideas and world view, to the team, the organisation, the industry and society.

      When I do the same with an LLM, I'm just repeating the same guidance over and over again out of frustration just to get something done.

      Comparing an LLM to a junior engineer is frankly a disgrace.

    • majorbugger 32 minutes ago

      And you end up rewriting this stuff yourself.

      • ako 24 minutes ago

        If you are a bad senior, yes. If you are a good senior you train the junior so his output will improve and you train the agent so its output will improve.

    • gonzalohm 14 minutes ago

      So what's the point of the tool then? So much for saving time, if I have to mentor it like a junior I much rather have a person to mentor

      • hhjinks 3 minutes ago

        That was how I understood "code was never the bottleneck". If I need a deep understanding of the code, I might as well write it myself. It won't increase the time I need for the task appreciably.

  • CurleighBraces an hour ago

    Humans not using their own brains and relying on AI to think for them creates a fragile code base.

    I say that as a frustrated slop fighter.

    • tarkin2 12 minutes ago

      It seems obvious--and correct me if I'm wrong--that using AI requires less mental effort; and a less exerted mind is a weaker mind. A weaker mind will continue to create more and more fragile code.

    • jansport123 an hour ago

      we'll recalibrate eventually based on understanding what level of abstraction we can operate on. If these code bases become unmanageable then the trend will correct itself.

      • n4r9 an hour ago

        I can't imagine a broad movement of devs using AI more thoughtfully without a widespread incentivisation from employers. And I can't imagine the latter at all. We'll probably just have to cope with more brittle/fragile codebases.

        • epihelix 29 minutes ago

          > We'll probably just have to cope with more brittle/fragile codebases.

          Is this not incentive enough to use AI more thoughtfully?

          • n4r9 21 minutes ago

            It's tricky when it comes to performance evaluation. Metrics like token usage or story point "velocity" are easy to conjure up even though they mean little. The kind of careful work that goes into building robust, maintainable code drags such metrics down, and it's very hard to quantifiably gauge their positive effects.

          • w4der 10 minutes ago

            Not when many companies still think that they'll die if they don't ship enough new features.

      • soon_to_be_sb 41 minutes ago

        >we'll recalibrate eventually based on understanding what level of abstraction we can operate on...

        This would be at the level of helper functions < 10 lines of code.

      • duskdozer 6 minutes ago

        I doubt this if the patterns of tech/software over the last <x> years remain. We'll just get buggier, worse-functioning, more expensive products/software built by fewer, more burned out developers. But your manager will love the sales presentation, so all will be well.

    • lynx97 43 minutes ago

      > I say that as a frustrated slop fighter.

      That part was obvious, no need to explain it.

  • jojobas 29 minutes ago

    AI non-usage brings about irrelevance under pressure from competitors with fragile codebase.

  • elzbardico 23 minutes ago

    We shouldn't worry about that. We should be grateful about the mountains of technical debt, because that will be the social security of the post boomers generations: working till death fixing the last hurrah of the boomers.

  • Topfi 36 minutes ago

    I do agree with some parts but not others. Reverting a flawed changed, e.g. should have never been as difficult as I have seen it been in some orgs before the advent of LLMs. Same for poor UX landing, both are issues that sure, may be thrown into sharper relief now, but were issues in some code bases and not others and remain that way today. More over, they are issues which can be permanently addressed in the former and in the later are one of the key sources of differentiation today.

    On code readability, shallow tests and general quality, I can agree emphatically, though not universally. In other words, there are models which write the densest, hardest (for me) to parse code and tests that break with the lowest load/timing deviations yet proof less than nothing.

    These do that regardless of how much one ships in my experience however so I wouldn’t attribute this to the velocity and more poor data at the heart.

    I know this reads a bit like “you are using the wrong model” and I understand if someone testing e.g. an OpenAI models understandably extrapolates across all models and considers poor parse-ability in code a likely result of velocity, but you can always step back and work slower, plus, for what it’s worth, I struggle to parse OpenAI code when running their models through my evals after step 7 which is still a fairly measured in scope code base.

    Essentially, I have seen LLMs, even the newest frontier, produce barely legible slop when building simple CRUD and I have seen a massive rewrite of the existing Firefox code base using APZ to enable an infinite canvas of browser tiles grow in a manner where, even at this stage having replaced a lot outside Places and Gecko, I can walk through changes made and explain the logic, as well as look at any part I haven’t been in for a while and find my bearing even without assistance.

    Whatever model you use, however much you trust it, if you can get a grip and explain what something does, that puts you in great shape. It also, for what it’s worth, helps future changes by better models to land with higher token efficiency and accuracy. I have put Opus 5.5 into a GPT-6 Astra written code base and it could land features, but the rework the model did along the way was eye opening. I then repeated, explicitly enforcing no changes beyond the initial scope which Opus 5.5 still could succeed in, but took far longer, far more tool calls and significantly more tokens then that change should have taken. I also have examples where any model can be more error prone and slower than myself, though those are unique and I myself have been told I was using the wrong model and sucked at prompting.

    Use models like any tech if you want for what you feel comfortable for or don’t, both can yield good or bad outcomes. I was and still am an enthusiastic Flutter/Dart dev, which with pub.dev meant I drew in a lot of external packages of dubious quality and compatibility, that I never read. LLM output, I do read and can go beyond my manual contributions in understanding because of it. Doesn’t mean model use doesn’t enable me to do more. Code was, for me, always a means to an end, enabling something. That hasn’t changed and never needed more LOC or massive scope.

  • cyanydeez 39 minutes ago

    The silent part is "undirected". You can patch these holes by observing them and the author us aware. By itself, these are the flaws.

    I think speed is the biggest flaw though. If i could get variable speed while planning vs building, thatd change some things. I run local models, so i am mostly in the slow lane. When patching these flaws, i want it to read the code parts and diagnose quickly.

    When its coding, i want to see the problems its encountering with the prompt. I usually observe its thinking a few rounds and if it misunderstands, i redirect.

    But in all of this, i want a dial that just speeds up once i feel its doing the right thing.

    SOTA today, to me, would put a model floor like qwen3.8-flash-next on a platform i can dial up the speed. Thats all. I think many of us will bring our own harness, so the only hard point is how dumb the engineers tools are.

    Since im building local, my tools do a lot more management. So.e of tge opinions i see on HN trigger a "skill issue" reflex because we have a whole shopping mall now if tools and the incurious seem oblivious to how these models are used by cloud providers.

    Learning about the raw capabilities of LLMs makes getting them to "do the right thing" is now just a job description.

    Lastly, the last decade of tech oligarchy exposes how little education elites had about humanity and wicked problems. Elon musk reinventing public transportation several times ignorantly. The fact that a social program requires the consent of the people it helps. LLMs are a stark lesson in the difficulty of knowing all the constituent parts and still being unable to predict outcomes.

    Welcome to Socioligy 101 tech nerds.

    • fragmede 26 minutes ago

      > Elon musk reinventing public transportation several times ignorantly.

      In interviews, he admitted that the Hyperloop was intended to mess up the California High Speed Rail project because trains are boring. It's the opposite of ignorance, it's malice. Hanlon's inversion.