I don't think I have a "burnout", but LLMs are really exhausting due to amount of pressure they generate. No one is really pushing me to increase my workload, but at every moment there is always something ready, done by my clankers or clankers of other people that I could be unblocking. In the past (before LLMs) it was already hard to keep up, but now it feels like there's 10x more things waiting at any given time, and there could be 10x more if everyone just "optimized" and streamlined processes fed the AI even more tasks in parallel faster. It just being a bottleneck of everything, all the time is tiring...
I am happy about all the little side-projects, and ideas it help my realize, and I enjoy exploring this new world, but I've noticed LLMs feed my unhealthy "don't want to take a break and waste time being idle" mindset, and I need to correct it.
W.r.t. article's main complain - I think the similar thing happened due to factory manufacturing automation. What used to be a varied skillful craft in a shop became standing in a single place of an assembly line doing the exact same thing whole day. LLM took away the more creative and variable part of the work, and left the repetitive QA rubber-stamping. Probably some of the mitigations used back then could be rediscovered today.
I am happy about all the little side-projects, and ideas it help my realize..
Same, but I really have to fight the urge to just add fun new features to things I work on any time inspiration strikes. I am an appalling 'feature factory' if I don't actively keep myself in check. The cost of just building everything is so low, but the value of those things is also incredibly low, so I'm often just bloating what I build.
There's been a lot of articles and posts about the increasing importance of 'taste' in software built with AI, and I'm finding I know need to look for strategies to find some.
> I don't think I have a "burnout", but LLMs are really exhausting due to amount of pressure they generate. No one is really pushing me to increase my workload, but at every moment there is always something ready, done by my clankers or clankers of other people that I could be unblocking.
I see a different type of pressure: I'm at a company that still is requiring everyone use LLMs with token leaderboards, time-spent measurements, and impacts to performance reviews, and all that. So I find myself having to carve out some percent of my time to stop doing productive work, and "go do AI to show token use." So my workload hasn't changed (or it's gone up), but I have N% less time to work on it because I have to spend time appeasing the AI gods...
Just have an agent chug on a side-project for you, or set up a CI script to review every pull request or some similarly “helpful” task. That should eat a lot of tokens!
Man. If I had this kind of mandate I could really burn some tokens. Review each new PR and extract 100 topics to debate related to it. Spin up 1000 sub agents, each with a different personality profile system prompt, to debate each point until consensus has been reached. Synthesize the learnings into a limerick. Build a Spotify playlist that pairs with the tone of the debates. Post the limerick and link to playlist on the PR and tag me to notify me that I have a PR to review.
Oh man, when MCP was still new and shiny I made an MCP that let the AI choose appropriate theme music for what it was doing and it was an absolute blast, I need to make a more modern one.
Peer Gynt Suite's "In the Hall of the Mountain King" made a prominent appearance, but so did Aqua's "Barbie Girl"
I echo this entirely, brother. I think a lot of us developers have a lot of ideas that were unrealized, and now we have this opportunity to do it. And any time an LLM is sitting idle, it feels like we're wasting our time. Why aren't we having it built something for us? Currently, I work on about three projects at work at the same time and about four personal projects at the same time. My day just zips by. I'll burn four hours without even thinking about it. It's exhausting but exhilarating. I do wonder if burnouts in the future though.
I'm getting so many requests to review LLM-generated documents - planning docs, docs intended for end-users, project docs, business plan docs. A team member sent me a zip file with about 30 LLM generated documents in it the other day and asked if I could review them right away. And a lot of it was just repetition and/or stuff that was just out of left field, made-up, hallucinated stuff. They're able to generate this stuff way faster than we can review. It used to be that it would take a significant part of a day for a project manager to come up with a planning doc - now they can generate one in a few minutes and send it out for review. It's just really tiring.
Wait, what? I thought everyone agrees that modern models post September 2025 (or whenever Opus or whatever 5.6789 was released) do not hallucinate, make things up, contradict themselves and can review their own output into perfection regardless of task, goal or context???? /s
In general I think from the coding side they're more robust now. However, people generating docs are maybe not as experienced with how to prompt in ways that avoid having the LLM tell you what you want to hear. I think this is still a pitfall that can easily be fallen into. Those of us who are doing LLM-assisted coding for the last couple of years are more aware of this now. Those who are planning/management folks are still kind of susceptible depending on how much experience they've had dealing with LLMs.
One of the reasons they exhaust me, is that it's always "one more prompt" to get a UI correct. It's often just slightly off, but it can take 5-10 mins sometimes to rework something. It has led to me working much longer hours.
I think this is in part because I am one of the software engineers that always liked building products more than writing complex software. So, I am driven by the feeling of creating something. And I want to get the feature perfect and complete. But getting from 95%->100% done can take a long time with UI work for me.
Maybe when they get better at making SVGs of pelicans riding bicycles, they'll also get better at making UIs that can be reworked into sensible form without too much effort.
LLMs drive the unit cost of cognition to zero. Therefore, you will exhaust yourself near-instantly trying to drive differentiated value out of cognitive work.
Non-arbitrable labor is one safe haven: bending steel, drilling wells, running cables, flying drones, etc. Physical agency gets you a premium the clankers can’t (yet?) trespass upon. That’s why guys building data centers are making bank & job-hopping while the SAs administering the computational guts of them are struggling.
A second vector is reputational: either by authority (you’re a regulator) or by taste (you’re a rare/reknown specialist) you make quality attestations about cheaply-produced cognitive artifacts.
The first vector is a big community; the second is not.
Get out of being in a knife fight with the clankers on their own turf, they’ll gut you.
Flying drones is an interesting one, I guess you do have to drive the car out to site and set up the drone. But a lot of drone ops are waypointed, automatic flight. I can see a future in which the only thing the operator does is drive the drone van to site, hit the deploy button as the drone pops out the roof, and wait for it to return. Mission set up already by an LLM prompt back at the office.
I don't have an employer. But most of the excuses I used to tell myself are simply not believable anymore and that causes pressure leading to overworking myself.
Individual gains from llm seem much larger than net productivity increases. I think a major source of this discrepency is people creating more work for their coworkers at the speed of slop. Especially the people with no idea.
"I did a Chat output, please fix and review it " is the kind of thing that empowers the people who used to have a minimal productivity, and now lets them to wreck things on an industrial scale.
This is valid in the other direction as well. Principle engineers, CTOs, with legitimately earned authority end up using that authority to 100x their output onto the team as if it was a Godsend unlock.
It's not. There is no one person that has universally good taste. Also, we're not in your head, no matter how much better of a coder or whatever. We're not in your head and it's all terribly painful to navigate.
>"I did a Chat output, please fix and review it " is the kind of thing that empowers the people who used to have a minimal productivity, and now lets them to wreck things on an industrial scale.
AI is not a productivity multiplier. There are diminishing results.
The ones that notice the highest increases of productivity are usually the ones that were unproductive at best and dangerously incompetent at worst.
> Individual gains from llm seem much larger than net productivity increases. I think a major source of this discrepency is people creating more work for their coworkers at the speed of slop. Especially the people with no idea.
Lots of companies (nearly all, I’d wager) of any size were leaving bare-minimum a 2x software development speed increase on the table before LLMs, having nothing whatsoever to do with how fast anyone was typing or thinking up code, and everything to do with how they organized and supported development work, and with your basic ordinary corporate dysfunction.
My company, I’d say it was more like 4x or 5x they could have achieved before LLMs, by fixing processes and reducing how often management steps on their own dicks.
All the people I’m seeing with crazy-high LLM productivity at my company? They’ve been given enormous autonomy to basically go do WTF ever they want, and people are jumping to get them anything they need (and most of what they’re doing is prototyping, for that matter). So right off the bat, if they’re competent, they should see a notable multiplier on productivity even if they weren’t using LLMs. Not that those aren’t helping, too, but if you don’t change processes they’re not all that effective, because the problem wasn’t speed of code-writing (and if you can change processes, you already could have sped up development a lot before LLMs…)
> No one is really pushing me to increase my workload, but at every moment there is always something ready, done by wankers
I confess that the above variant on the quotation is how I originally read it. And that's just about how I feel now with trying to sort through vibe-coded slop projects that are put forth by (well-meaning, probably good intentioned, not evil) people who represent them as if they're the handcrafted result of one dedicated developer.
The reason I'm getting LLM burnout is from dealing with the obvious neutering and opaque downgrading of all the top models.
Prior to the last 12mos AI companies were hell bent on squeezing out the best results from mediocre models.
But... now that the top models have progressed, those same AI companies have switched their efforts into reducing the computation (cost of a producing a result) as much as possible without being too obvious.
What was an exponential slope in the quality of results over the last 36 months has now nearly flat lined.
Addendum: IMHO results have 'flat lined' not because the models aren't much more capable than a year ago, but because conserving the enormous processing cost (of an over subscribed user base) supersedes the goal of following the user's explicit instructions (e.g. especially if that means more processing cost) to generate the best results.
I feel the same way about consumer AI tools now. Gemini and ChatGPT have been abysmal lately. They can no longer be relied on to do multi-turn searching and thinking.
Before, they could stay in thinking mode for more than 7 minutes. For example, "find a source for this claim" would search, analyze, and self-adjust the query. Nowadays, even if I push for it, I cannot make these tools work for more than 30 seconds before they give generic answers, even in "Pro" mode.
Hell, the Opus 4.5 moment was only last November, and that was when agentic coding and most coding CLI tools became truly first class options. That's a wild paradigm shift. Hell, GPT-5 wasn't even out (that's August of last year). Most people were using 4o. Their current offerings are wildly better for coding than 4o was.
I generally don't agree with the original commenter here. I think many of the complaints about model regressions are the result of increased usage and increased scrutiny revealing gaps there were there all the time. I've been more critical than most of the output quality since my initial "wow" moment was pretty early - GPT 3.5 API - and the results then were extremely obviously not production ready. But, keeping that level of scrutiny through my usage, I haven't seen the falloff that people who don't look at the output every time claim to see.
But that's also let me use "agent" stuff longer, I guess? The better you were at knowing what you wanted and how to ask for it, the less of an inflection point that you got from Opus 4.5 or GPT 5.
Some of the highest-time-saved-for-max-ROI agentic problems I've solved to date were in September and October of last year with Claude or Cursor.
I've started feeling slightly physically ill when I read Opus output for hours straight. This article rings very true for me. I've started complaining about it with my team; at least have a personal style guide in your agent rules that eliminates emdashes, the "it's not X, it's Y"s, the long lists of modifiers before the noun, using the word "land" to mean finish, etc. I hope this is just a phase of adolescent LLMs.
I was describing this exact feeling today. I haven't quite been able to put it into words but I do get slightly physically ill. Almost similar to mild trypophobia?
`arc land` is burnt into my brain by Phabricator, so I'm aware that the term predates LLMs, but it still drives me nuts.
It's impossible to undo some of these linguistic wobbles. Even if you could filter out 100% of LLM input, the humans themselves are learning to say "land" at a higher frequency now.
I had Claude make a world cloud from its responses because I was curious to see how big "honest" would be. It barely showed up, so I asked it to just give me the counts and it responded telling me it was trained not to use the word "honest" much because it makes people distrust responses (in addition to showing me the counts).
From my experience, there are mainly 3 burnout reasons.
1. Multi-tasking is the top one. I usually have to frequently switch between 3 to 5 agent windows which are on different things. It's extremely exhausting when each round takes a few minutes. Before coding agent era, I believe most developers had chance to spend 2+ hours focusing on one thing. Now coding agents have increased my spectrum on the tech stack, but the bandwidth to do deep work isn't increased.
2. Agents are good at getting things running without crash, but do not guarantee to produce correct code. This is quite different from human experts with fundamental knowledge.
3. I also get frustrated when reviewing piles of AI generated low quality PRs. My attention is a limited resource. I don't waste too much energy on other people's work, but if I don't spend more effort, the entire project is corrupted quickly by reckless AI generated code without human author's careful thoughts and designs. Working with people who have less due diligence in mind is painful, working with them in coding agent era is 10x painful because they produce 10x shit. It's a team culture challenge that cannot be easily enforced.
Agreed. I am working hard to restrict myself to only 1-2 agent workflows at a time. More is untenable, though it’s so easy to fall into the trap of deploying an agent “just for this minor fix.”
I do not have the burnout but I certainly operate similarly to the author. I continue to be unable to establish a workflow where allowing the LLM to generate code that I review is faster than writing the code myself. Literally the only two ways out of this dilemma is to blindly trust what was generated or to generate an uncharacteristically exhaustive suite of unit tests to validate every possible scenario. I just write the business logic myself and have the LLM do a lot of the rest. Boilerplate falls into the latter as well.
> generate an uncharacteristically exhaustive suite of unit tests to validate every possible scenario.
This is what you want. You want comprehensive tests at every level, far more than is reasonable for a human to build or maintain, from unit, functional, to full end to end and beyond. Adversarial testing (both TDD-style "write tests to demonstrate this bug", and posthoc "prove this patch wrong with a new test") is the best way to keep AI on track and make those diffs you have to read clean and easy.
An even better way is to use a more strongly typed language and really lock it down, but you can use testing in any language. I feel like my background in TDD and "TATFT" has been secret sauce when working with AI
I see this get mentioned a lot but I still am skeptical that AI can generate tests we can trust more than any other code we know we cannot trust.
Yes tests are conceptually isolated and that helps, but I've personally seen unit tests get generated that are semantically incorrect - that is, they test the structure of the code (e.g. they can check function output types and values), but they can't know _why_ the unit tests need to be there, so the really really helpful tests never get generated. Not to mention the obvious issues with generated tests only testing is x = x, or needless redundant tests for the same thing, or them essentially testing basic features of the language.
AI shouldn't write tests. At least not all of them. Definitely not e2e's. The tests should be guardrails to constrain agents. This way, the author of code matters less.
You have to iterate on the tests, review and validate them, just like any other code, and if you generate a whole project's tests all at once the quality is abysmal, of course. I've been using a lot of old school data-driven testing techniques, where the harness is just code I review, and the data itself is e.g. json files and drives the system.
I actually have a public (AGPL) example here: https://github.com/pgdogdev/pgdog/tree/main/integration/sql - pgdog is particularly testable since it is trying for complete transparency, so you have a perfect oracle in hand via base postgresql, but it demonstrates the concept at least.
Then this falls into the exact same pit the OP mentioned, either you need to blindly trust that the LLM is generating tests that actually work, or you need extensive test coverage for your tests to ensure that your tests are actually testing.
It turns out that you don't actually need tests for your tests, because the code provides a baseline truth for the tests. You do, at some point, have to be epistemically sound enough to actually look for correctness in either the code, behavior, or tests. We unfortunately haven't fully unlocked completely solipsistic value generation yet.
This is also part of why I like end to end tests that use actual UI flow, so I can watch it go by in slow mode before letting it loose fully automated.
Which is why test generation has to be carefully guided as well, and this is something at which I've incidentally been fast. Ultimately it's a constant battle between LLM handholding and doing things yourself.
I don't even care about tests being correct as you can still verify them even when tedious. What I care is that, more often than not, the shape of the solution is not fixed. Having unit tests for those can be extremely costly as when the changes happens, you have to change all the tests.
I've been burned by this in my honeymoon period with unit testing (pretty much the reason it ended). These days, I prefer broader scope of testing, especially user-facing part. The users may be other developers or end users. I only do unit testing for tricky algorithms or math formulae.
I want all the layers of the pyramid, eventually, but the top layers matter the most. I can't count the number of times my paranoid "make sure that customers can successfully pay us" end to end test suite has prevented the money faucet from being shut off. I install one perennially at any company I work at and they always pay for themselves surprisingly quickly.
I’ve been involved in B2B (so no payment flow). But it’s basically the same with an handful of integration tests for common workflows. They run fast and mostly serve a canary to ensure that we are not crippling some use cases. When a bug hits us, a test case is added/modified for it.
They’re mostly a reflection of the current requirement of the project.
It has good test coverage, mostly unit tests but also a number of end-to-end tests. I also made the LLM build a benchmark, which you can find at the bottom of the readme. It is obviously slow, but I thought that it is good enough to work. When I tried to write a 1 GiB file, I found that it broke down, and after writing half the file, the speed went to under one megabyte per second. Implementation is 10k+ LoC, and I have no idea what is going on there.
That's interesting because I would feed that benchmark back into the agent and loop over it, to see how much faster you could get it, and agents are really good at that kind of recursive optimization. And I would definitely add at least a simulated 1GiB write test, probably a real one honestly, if I was building something like that.
At least with agent-run tests I care about loop speed a lot, but I care about complete coverage more, so having the odd heavy weight full stack integration test is fine, I think.
You're right. This was just a performance issue, but what if next time it is a corruption bug or a security vulnerability or really anything that can cause real consequences if happened in production? I don't think that LLM systems are inherently bound to have this flaw, but I think that we are pretty far from harnesses and algorithms becoming advanced enough so that the LLM system can kind of continuously evaluate its output and ensure it is good in all aspects.
I don't know about that, Fable is, when properly guided, a better engineer for those things than I am. Narrow breadth, weird priorities, myopic and ivory tower as hell, but superhuman. Maybe that says more about me, or maybe not, but certainly it's caught bugs I would not have, and point it at things like a fuzzer, woo buddy, it has been many years since I broke out valgrind and nailed down a memory leak, but it sure can.
100% this is what I've done. I sucked it up and adapted myself to the tool (agents) by having as many implicit guardrails (static typing, functional, no nulls, great linting) and then layering on explicit guardrails (TDD) on top. I also want my workflow to be portable because I don't really trust the frontier model providers.
It is different though. Basically a lot of what I do has changed over the last 2 years. I totally get that a lot of people won't want to adapt though.
Maybe famous last words, but I'm not buying the hype that the "clankers" will take over. I suspect reality will catch up soon and we'll be left with a set of pretty powerful but still limited tools. I see no evidence to the contrary, just investment hype on one side and sky is falling on the other.
I've just been carefully reading the code. It is easy to slip into just accepting what comes out to speed things up, but reading the code is important.
I save myself by skimming things like tests, templates, some UI. Anything cosmetic. But I have to read the majority of code that ends up on my back end systems.
And for those that have similar-ish sentiments, what mental defect is had that prevents them from just drinking that sweet tasty kool-aid and just use the slop created. What demented trait is in them that causes everyone to just be a stick in the mud trying to ruin everyone else's good time?
In my personal experience, the ones most enthusiastic about LLM magic are those that can't code, but can now walk away with something functional if not quite the best code. Now that they can produce workable code, it will make everyone better. Yet, they have no idea how maintainable the slop is or if it's slop at all.
I actually dispute this, I read all the code, the core thing people have to give up is not "reading the code" per se, it's giving up on "that's not how I would have done it".
When you see a perfectly clear function or object that just isn't your style, you have to accept it and move on. Where there are concrete concerns, or it's unreadable, demand excellence, but treat it like a coworker, not an IDE.
This all reminds me of the differing experiences people had outsourcing coding in the 2010s when it was still called oDesk. You don’t need to read code, you just need to know that the code works. If something doesn’t show up as a problem it doesn’t need to be fixed, and reading code is the least efficient way to discover problems.
The only time I look at code is when something isn’t right and I ask for a root cause analysis. The LLM will show me some offending code or code for reference or evidence and then I quite often say “well that’s dumb you should do it like this instead” but I never need to actually go into the files. I do sometimes look at a git status or git diff.
Yeah this is how I feel about it. Does it look correct? is it doing something weird? Is it forgetting about some gotcha in our domain that it hasn't been taught about yet? Otherwise, ship it.
It sounds kind of like being stuck working with coworkers who--while not overtly hostile--need constant hand-holding and repeat the same kinds of mistakes every day and can't even be genuinely sorry about it.
Just because we work with computers doesn't mean we don't take, er, social-damage. Or perhaps parasocial damage, in this case.
This is legitimately the reason I'm looking to leave programming.
I got into programming because the problems of programming were interesting to me. But if the problems go from "figure out why this calculator is off by one in France" to "Get this LLM to stop spamming cutsey emojis", then maybe it's time for a career change.
Giving my "otherside", because the pressure to output more at work is real, but at the same time, out side of work, I love this. I'm able to do way more projects than ever before because a barrier to entry was always the amount of research+time required to start up a pet project.
My latest is, I'm really into fizzy/soda water and wanted my own continuous carbonator. My entire build from water source to tap with an ESP32 controlled pump, pressure, water level, cooling fans.
There were so many areas I made mistakes in my shopping cart and it found it - like Home Brewer likes 8mm lines but water filter systems like 9.5mm. Really optimized the versions from a simple on/off pump w/ float switch to effectively a full on PLC system. So many iterations gained by chatting with "someone more experienced". Once I get the parts I can build and have the software side running in less than an hour.
I got into programming to just build stuff, the coding is just a means to an end, try not to think too hard about the how and think more about the why and what
We get it, you don't have a passion for the act or the craft, just the end result, but I'm absolutely sick of hearing it all over this site as if it's a universal truth that some of us just don't recognize yet.
Sorry, some of us have a joy for programming where the how is just as important, if not more so, than the what and the why. No matter how much people proclaim that the how doesn't matter to them, it isn't going to suddenly make it true for others.
This isn't a response I expect from people who are here for a productive discussion. I'm sorry that you are sick of hearing this, but I'm not responsible for making sure you only read what you find worthy of your own personal brand of respect. Instead of attacking someone for simply offering their point of view, in what appears to be a quasi-gatekeeping effort, maybe you should look inward and discover what is making you this upset toward a complete stranger.
__I cannot take away the joy you have for programming simply by stating what drives me.__
I'm totally fine, just annoyed by how much this "try not to think too hard about the how and think more about the why and what"[0] is getting brought up every single time someone mentions why they prefer hand coding to vibe coding. At this point, it's being overstated, thus gives off less "this is why I use it" and more "come on, get onboard the hypetrain to dystopia or you're going to get left behind". It's hardly conducive to a "productive discussion" when it's the millionth time as a drive-by comment. What kind of response did you expect?
Look, I don't have a problem with your personal motivation. I just hate seeing it suggested that people should abandon their passion because someone else doesn't share it. There's absolutely nothing wrong with "I enjoy making something useful" just as there's nothing wrong with "I enjoy making something with my hands or figuring out how to make it". My problem is with "your enjoyment of that is invalid because I don't enjoy that, so learn to enjoy this".
0: Not really a statement of your drive, is it? More of a directive or suggestion.
If solving jigsaw puzzles with claude will enable the creation of tools that help the people I want to help (students with disabilities), then I would use it for that, without feeling any guilt at all for doing so.
Why should I regret that? Why should I care about your purity tests?
Part of the annoying thing is that if you're working on a product which uses LLMs, at some level you run out of levers to pull in terms of being able to fix things. At best you're stacking hacks on top of hacks to prevent unwanted output, but at the end of the day if the LLM really decides it simply doesn't want to follow your instructions, you can't do much other than resign to adding *IMPORTANT* and hoping the next model fixes it.
The experience is much closer to working with an external API that you don't have control over and which simply doesn't do what the documentation says. Those have always been the most frustrating parts of programming, but at least previously you could reverse engineer the actual implementation to work around bugs. You can't even do that now because the "boundary" randomly change every day.
(And I admit I'm salty that the "I don't give a shit about why the calculator doesn't work in France, I'm just here because they pay me to fix it" people were the ones vindicated by technological progress)
I'm no longer in corporate America, so maybe I'm out of touch a bit, but could you just...not...use an LLM? You can still solve interesting problems on your own if you choose to do so?
It’s not there yet but we’re clearly heading towards a world where the answer is “no, you have no choice”. AI is weaved into business processes. If Ai leaves a comment on your pr, you must resolve it before merging, you’re expected to “get things done” at a particular pace consistent with using ai, regardless of whether what you did is any good.
LLM skew the time estimate tho. Now everybody expect stuff based on LLM work instead of normal human work. I/we can choose to solve problem normally, but the expectations have changed.
Yeah at many places you still can. It’s just so easy to turn your brain off and let the robot do a maybe good enough job that even people who know better are merging slop.
We’ve had 3 production incidents this week that slipped past CI because there’s a whole team that is just shoving out PRs without understanding what’s going out.
A lot is said about context you can feed into the LLM but I do think there is still superior power in human context awareness. That kind of ambient collation and organisation of the whole business and its purpose, all the different work going on and how it all relates to eachother. It happens when you isolate business units a bit too much also.
It's not surprising that if you have a hundred separate, isolated contexts working on the same business, that don't cross-talk and have no ability to subconsciously receive and collate, prioritize the thousands of signals we get from our work environment, that you end up shipping lots of incomplete or incompatible work.
I get burnout from frustration when the LLM just can't follow instructions.
Like when I'm trying to get it to create an image, and the first pass is beautiful, but ten different request to modify it, with different phrasing and even example images, produce the same image ten times.
Or when you tell it not to use a cheap hack in AGENTS.md about six different ways and in your prompt, and it still does it again, and again.
It's like arguing with an idiot. And THAT gives me burnout.
Also: I've never once seen an emoji in LLM output. What are people talking about?
Excessive amounts of emojis in generated README.md and sometimes in printed/logging outputs. I don't whether this is still an issue because I have a "Never use emojis" instruction in the context.
I’m currently doing a project with someone who only uses LLMs, and it’s exhausting and mentally draining.
Whenever I give feedback on something, the answer is just “let me tell Claude”. The person has no understanding of how everything works, and most of the code reflects that.
The other day he hardcoded in a demo mode, simply because he didn’t even know how to set up a local environment and set environment variables. I’m confused as to why Claude didn’t even knew this, but it might just be the prompting.
I limit LLM usage myself, and if I do use it, I try to use it on extremely specific tasks. It’s the only way it works for me.
I honestly don’t understand how all these companies are getting away with generating AI code. Even in a small project I quickly fall behind on my understanding of the project.
I remember working with people like this before AI and it was annoying but they struggled with productivity because they didn't understand what they were working on enough to produce good code efficiently, so the problem usually took care of itself.
Now these people can thrive because LLM coding encourages the incurious and punishes the deep thinker.
is there any evidence that Alec Scollon, the first time blog author responsible for this post, even exists? look up the name. boo this post and the premise behind it.
It's burnt me out too. I'm generating 10x more features and multitasking across 4 disparate projects. My greatest concern is I don't really have a strong connection to the underlying fundamentals anymore. I need to see how the things works to internalize it. Now I just trust that the agent wrote this piece correctly.
The productivity drive and the sheer feature set you can generate in record time makes it easy to forget proper sdlc hygiene.
Knowing how things work, knowing what should be possible and where “there be dragons”, and having a pretty well-developed “sixth sense” for all kinds of things is proving just as valuable with LLM-heavy programming as it did before.
… but I am almost certain I’d never have developed those in the first place if I hadn’t spent 25ish years programming on a bunch of different platforms and setting up servers and networks and all that, without LLMs.
I dunno how you make another “me”, now, while before lots and lots of programmers naturally ended up as someone with skills and knowledge like mine, and those skills seem super useful when writing code with LLMs.
> My job has changed from designing and writing code to designing code, describing the design to an LLM, reviewing code the LLM produces
As a long-time engineering manager, PM and, eventually, product owner my response is, "Congrats! You've just been promoted to management." :-)
As a new manager, your first challenge will be successfully delivering commercial results using only a team of 'differently abled' new grad interns. Don't complain, new managers don't get to pick their first team! To be honest, these guys are more like alien brains raised in a vat with no direct senses. They've only ever experienced a data feed of the internet and, oh yeah, they get near-total amnesia a few times a day (but maybe you can teach them to write notes for themselves). They also have ADHD and are somewhere on the spectrum. But don't worry because what they lack in common sense, experience and intuition is offset by having a sort-of photographic memory and a willingness to grind on a problem 24/7. You should be fine. Good luck, we're all counting you...
I don’t understand what could possibly need to be made so fast that isn’t totally made up billable hours. Running at top speed long enough to be burned out is either ineffective, or valuable enough that someone else can take over while you sleep.
Even with Fabel and all that I constantly keep having to babysit it and correct it like it’s an adolescent and it gets really old and the amount of code. It produces not all of its great at all. I’m burnt out looking at. It’s poor coating that somehow magically works.
If an LLM was a person, it would be known as a "bullshit artist". I've worked with bullshit artists (people) before, and its exhausting trying to separate fact from fiction. It's a reason to avoid working with such people.
A good coworker will admit not knowing something, or if unsure give their best guess but discuss its limitations and why they might be wrong.
Question: Has anyone experimented with using voice to directly prompt an LLM, without doing speech-to-text? If an LLM can pick up on the skeptical nuances in a person's response, it might be prompted not to be overconfident in its subsequent output.
I've taken a bit longer than I wanted but it will be open sourced soon.
It's a durable orchestration engine that takes in specs/requirements and coordinates agents externally (meaning the engine drives the loop, not an agent) until the work is fully implemented/verified and reviewed.
It's meant to be used with any harness as basically the last step. You plan your work with whatever LLM you use and then hand off implementation to the engine (through an MCP server or other surfaces)
It can use your OpenAI/Anthropic subscriptions or any other provider and you can mix and match models across implementation and review in any way you want with fan out for parallel reviewers and more.
The goal is to produce high quality unsupervised code that matches your requirements and is reviewed throughout the implementation rather than at the end only, so that mistakes don't compound.
https://github.com/gastownhall/gascity is certainly a choice. I enjoyed playing with gas town but it was a little too nondeterministic for production code, I think.
Directionally if what you're doing is straightforward it's an amazing experience to be able to slap in an epic planning document and wake up the next day to it being "done", with a big asterisk that done-ness is directly proportional to how good of a spec and how good of a model you were using.
That being said, these days if you use Fable, slap in an epic planning document, and ask it to run a workflow (be sure to specify that subagents should use, say, Sonnet, or wave goodbye to your wallet), it's almost as good as gastown/gascity but far more predictable.
I think having style guidance in your context is valuable for avoiding this kind of thing. Having to read awful, cliched text all day is even worse than having to read reams of useless code. I have some simple humanizing content in there that specifically calls out the rhetorical devices that AI loves, and it drastically improves the diffs and comments. It also makes the coding performance generally slightly worse, but ergonomics uber alles.
I don't have much success with using the LLM to make changes to a big legacy codebase. Instead, I use the LLM to gripe about things I don't like in the code. Usually, it is a brilliant commiserator.
I don't mind interacting with LLMs myself and find they increase my productivity a decent amount. I just can't stand dealing with other people's slop.
Getting sent IM responses that are copy pasted LLM nonsense. Getting a massive PR to review that was generated overnight and the author didn't read it first.
So annoying, I'm dealing with a client that just constantly feeds my responses to their question into AI. Which of course just asks more questions and tells me how clever I am. I also know the client isn't reading because in many spots I put "[Client Name] you need to answer this directly as we need to know the actual real business detail" and its ignored or the AI provides a detail I know is made up.
I don't really understand how this isn't a self-inflicted problem? Perhaps it's because I'm not really mandated to use LLMs in a particular way, but I've had great success doing a combination of writing code myself and using smaller but faster models as a sort of "flood fill". The larger models can also be useful when you're implementing something which already exists in similar form in the codebase, because you can just put that code in the context and you'll get something very similar outputted. So the more code you write, the better the LLM can be later on. Codebases should get easier to add to the bigger they get, not harder.
Of course if you're supposed to achieve so much output that it's not possible to do anything but vibe it, fair enough.
I surprisingly had good results when I told the LLM to only communicate in ASCII memes. It did a fantastic job of summarizing the situation using relevant memes, and the humor was enough to keep things fresh. As silly as it sounds, it's worth trying when you're in that LLM burnout corner.
started to dread reading LLM output because I know what I’m going to find. False assumptions and hallucinations. Emphatic, staccato fragments. Excessive emojis .
I do not understand these complaints. Yes, those are the defaults and they're annoying, although the general public seems to like them. But you are not stuck with these. You can just tell the LLM how it should interact with you. If you're using any sort of harness beyond the chat window in a web browser, you can codify these instructions in a rules.md file or similar and have it automatically included in any new chat. It's not any harder than changing the default wallpaper or color scheme on your desktop operating system.
In reverse order, you can just tell the LLM to never use emojis. I don't like emphatic staccato fragments either, so I tell it to eschew the language of marketing and hype and stick to a factual and plain language, or to employ an academic tone. I explicitly instruct mine to ask clarifying questions whenever context is ambiguous and to push back on false assumptions or common misconceptions (by me). Hallucinationsa re the biggest problem of those you mention; it's not easy to totally eliminate them (for the same reason it's not easy to instruct people to not fall for scams or disinformation), but you can considerably reduce them by setting standards for citations.
I have ideas about reducing hallucinations over work material (ie a codebase) but am omitting them here as they are not fully thought out or tested.
What helped was a sleep and work system, oriented around being offline that was inspired by nature and from my earlier days in working in tech while car camping across the national parks.
Basically: the sun wins in terms of how all energy on the earth is structured, and expressed. All manners of cycles of organisms and living systems are in relation to its rise and fall, and even its particular color spectrum phases (whether thats night oriented or day). I call this our real circadian rhythm; it's used to being signaled by the light of the sun and maybe fire for millions of years and it isn't until recent centuries when we started tricking our biology with LEDs and lights. So the solution is simple. Orient yourself around the light of the sun and make sure it's the first and last major light source you see; blue limiting is the most important part BEFORE sunrise and AFTER CUT OFF ALL BLUE LIGHT. On my Mac I use a red light filter (using it now, it's 11:07pm ET and the sun went down about 2.5 hours ago). It's really hard to stay alert and chatting with an LLM when the only light sources are red and you keep them dim at that. Our ancestors would rest when the sun's at its peak (~1:05 pm today) and that's a good time to divide my own day productively as well. With intentional breaks diving the middle of the day with sunlight anchoring it, my nervous system is more relaxed, and by the evening time, it's also ready to transition out of anything blue-light assisted and most intellectual work and problem solving falls into this bucket. It's really hard to explain but it really works so simply. To enjoy the process a little more I made this fun sun clock, check it out at https://sunsignal.app
I once experimented with beeswax candles as my only after-dark light source. This meant no hyper-stimulating screen activities whatsoever, too. TV, phone, video games, browsing the web? Nope, nope, nope, and nope. Just dim, warm light from actual flames.
Cured my lifelong “night owl” “trait” in a couple days. Shockingly effective.
Turned out to be hard to keep up and still, like, exist with other people, and you’d probably need to relax it a little in Winter unless your job lets you work reduced hours to kinda “hibernate” (otherwise when would you do anything that’s not work but requires light or electronics?) but it sure worked.
LLMs poison your mind. The more AI slop you read, the more your mind turns into something like slop. This isn't very different from the idea that the food you eat is what your body is made of.
It sounds far fetched at first, but I think it could be true. There was a time in my life when I read literature constantly and started using those patterns in my writing. The habit fell off quickly after I stopped reading for a while, so there might be hope.
Ask random questions until you get enough sense of things to ask "the right" questions. Never stop asking questions. Use everything you can to get the answers and never settle for any answer no matter how good it sounds, doubt it and go back to step 1.
I don't think I have a "burnout", but LLMs are really exhausting due to amount of pressure they generate. No one is really pushing me to increase my workload, but at every moment there is always something ready, done by my clankers or clankers of other people that I could be unblocking. In the past (before LLMs) it was already hard to keep up, but now it feels like there's 10x more things waiting at any given time, and there could be 10x more if everyone just "optimized" and streamlined processes fed the AI even more tasks in parallel faster. It just being a bottleneck of everything, all the time is tiring...
I am happy about all the little side-projects, and ideas it help my realize, and I enjoy exploring this new world, but I've noticed LLMs feed my unhealthy "don't want to take a break and waste time being idle" mindset, and I need to correct it.
W.r.t. article's main complain - I think the similar thing happened due to factory manufacturing automation. What used to be a varied skillful craft in a shop became standing in a single place of an assembly line doing the exact same thing whole day. LLM took away the more creative and variable part of the work, and left the repetitive QA rubber-stamping. Probably some of the mitigations used back then could be rediscovered today.
I am happy about all the little side-projects, and ideas it help my realize..
Same, but I really have to fight the urge to just add fun new features to things I work on any time inspiration strikes. I am an appalling 'feature factory' if I don't actively keep myself in check. The cost of just building everything is so low, but the value of those things is also incredibly low, so I'm often just bloating what I build.
There's been a lot of articles and posts about the increasing importance of 'taste' in software built with AI, and I'm finding I know need to look for strategies to find some.
> I don't think I have a "burnout", but LLMs are really exhausting due to amount of pressure they generate. No one is really pushing me to increase my workload, but at every moment there is always something ready, done by my clankers or clankers of other people that I could be unblocking.
I see a different type of pressure: I'm at a company that still is requiring everyone use LLMs with token leaderboards, time-spent measurements, and impacts to performance reviews, and all that. So I find myself having to carve out some percent of my time to stop doing productive work, and "go do AI to show token use." So my workload hasn't changed (or it's gone up), but I have N% less time to work on it because I have to spend time appeasing the AI gods...
Just have an agent chug on a side-project for you, or set up a CI script to review every pull request or some similarly “helpful” task. That should eat a lot of tokens!
Man. If I had this kind of mandate I could really burn some tokens. Review each new PR and extract 100 topics to debate related to it. Spin up 1000 sub agents, each with a different personality profile system prompt, to debate each point until consensus has been reached. Synthesize the learnings into a limerick. Build a Spotify playlist that pairs with the tone of the debates. Post the limerick and link to playlist on the PR and tag me to notify me that I have a PR to review.
Oh man, when MCP was still new and shiny I made an MCP that let the AI choose appropriate theme music for what it was doing and it was an absolute blast, I need to make a more modern one.
Peer Gynt Suite's "In the Hall of the Mountain King" made a prominent appearance, but so did Aqua's "Barbie Girl"
Also, post the limerick to HN.
Ideally, you should ask the LLM to write that CI script.
What work are you doing where an LLM can’t meaningfully contribute to your everyday work
I echo this entirely, brother. I think a lot of us developers have a lot of ideas that were unrealized, and now we have this opportunity to do it. And any time an LLM is sitting idle, it feels like we're wasting our time. Why aren't we having it built something for us? Currently, I work on about three projects at work at the same time and about four personal projects at the same time. My day just zips by. I'll burn four hours without even thinking about it. It's exhausting but exhilarating. I do wonder if burnouts in the future though.
> done by my clankers or clankers of other people
I'm getting so many requests to review LLM-generated documents - planning docs, docs intended for end-users, project docs, business plan docs. A team member sent me a zip file with about 30 LLM generated documents in it the other day and asked if I could review them right away. And a lot of it was just repetition and/or stuff that was just out of left field, made-up, hallucinated stuff. They're able to generate this stuff way faster than we can review. It used to be that it would take a significant part of a day for a project manager to come up with a planning doc - now they can generate one in a few minutes and send it out for review. It's just really tiring.
> now they can generate one in a few minutes and send it out for review.
I think we will very soon move to a prove to me you've read it protocol and/or introduce speed bumps to slow things down.
Feed it into an AI and ask it to adversarially criticize it, doc for doc, send back 30 responses in a zip folder, wipe hands on pants, return to HN.
Thats gonna be a no from me dog. I don’t expect anyone to read something I didn’t read myself
Wait, what? I thought everyone agrees that modern models post September 2025 (or whenever Opus or whatever 5.6789 was released) do not hallucinate, make things up, contradict themselves and can review their own output into perfection regardless of task, goal or context???? /s
In general I think from the coding side they're more robust now. However, people generating docs are maybe not as experienced with how to prompt in ways that avoid having the LLM tell you what you want to hear. I think this is still a pitfall that can easily be fallen into. Those of us who are doing LLM-assisted coding for the last couple of years are more aware of this now. Those who are planning/management folks are still kind of susceptible depending on how much experience they've had dealing with LLMs.
One of the reasons they exhaust me, is that it's always "one more prompt" to get a UI correct. It's often just slightly off, but it can take 5-10 mins sometimes to rework something. It has led to me working much longer hours.
I think this is in part because I am one of the software engineers that always liked building products more than writing complex software. So, I am driven by the feeling of creating something. And I want to get the feature perfect and complete. But getting from 95%->100% done can take a long time with UI work for me.
So I work much longer hours now, unfortunately.
Maybe when they get better at making SVGs of pelicans riding bicycles, they'll also get better at making UIs that can be reworked into sensible form without too much effort.
Perhaps do the last 5% yourself?
This is what I do as I have learnt after much frustration.
LLMs drive the unit cost of cognition to zero. Therefore, you will exhaust yourself near-instantly trying to drive differentiated value out of cognitive work. Non-arbitrable labor is one safe haven: bending steel, drilling wells, running cables, flying drones, etc. Physical agency gets you a premium the clankers can’t (yet?) trespass upon. That’s why guys building data centers are making bank & job-hopping while the SAs administering the computational guts of them are struggling. A second vector is reputational: either by authority (you’re a regulator) or by taste (you’re a rare/reknown specialist) you make quality attestations about cheaply-produced cognitive artifacts. The first vector is a big community; the second is not. Get out of being in a knife fight with the clankers on their own turf, they’ll gut you.
Flying drones is an interesting one, I guess you do have to drive the car out to site and set up the drone. But a lot of drone ops are waypointed, automatic flight. I can see a future in which the only thing the operator does is drive the drone van to site, hit the deploy button as the drone pops out the roof, and wait for it to return. Mission set up already by an LLM prompt back at the office.
> No one is really pushing me to increase my workload
Spoken like someone who is not at an org/team that has undergone layoffs and reduced hiring in the last 3 years.
You might be in the minority there - especially when it comes to those who are facing burnout.
I don't have an employer. But most of the excuses I used to tell myself are simply not believable anymore and that causes pressure leading to overworking myself.
Individual gains from llm seem much larger than net productivity increases. I think a major source of this discrepency is people creating more work for their coworkers at the speed of slop. Especially the people with no idea.
"I did a Chat output, please fix and review it " is the kind of thing that empowers the people who used to have a minimal productivity, and now lets them to wreck things on an industrial scale.
This is valid in the other direction as well. Principle engineers, CTOs, with legitimately earned authority end up using that authority to 100x their output onto the team as if it was a Godsend unlock.
It's not. There is no one person that has universally good taste. Also, we're not in your head, no matter how much better of a coder or whatever. We're not in your head and it's all terribly painful to navigate.
I'm just waiting for Bezos to order all work with human agents to use the same API and guardrails as an LLM.
>"I did a Chat output, please fix and review it " is the kind of thing that empowers the people who used to have a minimal productivity, and now lets them to wreck things on an industrial scale.
AI is not a productivity multiplier. There are diminishing results.
The ones that notice the highest increases of productivity are usually the ones that were unproductive at best and dangerously incompetent at worst.
Could you describe your usual workflows and usage patterns with AI?
> Individual gains from llm seem much larger than net productivity increases. I think a major source of this discrepency is people creating more work for their coworkers at the speed of slop. Especially the people with no idea.
Lots of companies (nearly all, I’d wager) of any size were leaving bare-minimum a 2x software development speed increase on the table before LLMs, having nothing whatsoever to do with how fast anyone was typing or thinking up code, and everything to do with how they organized and supported development work, and with your basic ordinary corporate dysfunction.
My company, I’d say it was more like 4x or 5x they could have achieved before LLMs, by fixing processes and reducing how often management steps on their own dicks.
All the people I’m seeing with crazy-high LLM productivity at my company? They’ve been given enormous autonomy to basically go do WTF ever they want, and people are jumping to get them anything they need (and most of what they’re doing is prototyping, for that matter). So right off the bat, if they’re competent, they should see a notable multiplier on productivity even if they weren’t using LLMs. Not that those aren’t helping, too, but if you don’t change processes they’re not all that effective, because the problem wasn’t speed of code-writing (and if you can change processes, you already could have sped up development a lot before LLMs…)
That ALL sounds horrible, is that really life in tech these days?
> No one is really pushing me to increase my workload, but at every moment there is always something ready, done by wankers
I confess that the above variant on the quotation is how I originally read it. And that's just about how I feel now with trying to sort through vibe-coded slop projects that are put forth by (well-meaning, probably good intentioned, not evil) people who represent them as if they're the handcrafted result of one dedicated developer.
Yeah, but it's that KIND OF AWESOME?
The reason I'm getting LLM burnout is from dealing with the obvious neutering and opaque downgrading of all the top models.
Prior to the last 12mos AI companies were hell bent on squeezing out the best results from mediocre models.
But... now that the top models have progressed, those same AI companies have switched their efforts into reducing the computation (cost of a producing a result) as much as possible without being too obvious.
What was an exponential slope in the quality of results over the last 36 months has now nearly flat lined.
Addendum: IMHO results have 'flat lined' not because the models aren't much more capable than a year ago, but because conserving the enormous processing cost (of an over subscribed user base) supersedes the goal of following the user's explicit instructions (e.g. especially if that means more processing cost) to generate the best results.
I feel the same way about consumer AI tools now. Gemini and ChatGPT have been abysmal lately. They can no longer be relied on to do multi-turn searching and thinking.
Before, they could stay in thinking mode for more than 7 minutes. For example, "find a source for this claim" would search, analyze, and self-adjust the query. Nowadays, even if I push for it, I cannot make these tools work for more than 30 seconds before they give generic answers, even in "Pro" mode.
You sure about that? Maybe it is reality hitting expectations after the initial “holy shit” wears off
I'd bet more on this personally.
This seems hilariously, extremely revisionist.
Hell, the Opus 4.5 moment was only last November, and that was when agentic coding and most coding CLI tools became truly first class options. That's a wild paradigm shift. Hell, GPT-5 wasn't even out (that's August of last year). Most people were using 4o. Their current offerings are wildly better for coding than 4o was.
I generally don't agree with the original commenter here. I think many of the complaints about model regressions are the result of increased usage and increased scrutiny revealing gaps there were there all the time. I've been more critical than most of the output quality since my initial "wow" moment was pretty early - GPT 3.5 API - and the results then were extremely obviously not production ready. But, keeping that level of scrutiny through my usage, I haven't seen the falloff that people who don't look at the output every time claim to see.
But that's also let me use "agent" stuff longer, I guess? The better you were at knowing what you wanted and how to ask for it, the less of an inflection point that you got from Opus 4.5 or GPT 5.
Some of the highest-time-saved-for-max-ROI agentic problems I've solved to date were in September and October of last year with Claude or Cursor.
Cannot relate, my expectations might just gone up, because when I compare what I was producing with agents a year ago vs now, it's night and day.
I've started feeling slightly physically ill when I read Opus output for hours straight. This article rings very true for me. I've started complaining about it with my team; at least have a personal style guide in your agent rules that eliminates emdashes, the "it's not X, it's Y"s, the long lists of modifiers before the noun, using the word "land" to mean finish, etc. I hope this is just a phase of adolescent LLMs.
I was describing this exact feeling today. I haven't quite been able to put it into words but I do get slightly physically ill. Almost similar to mild trypophobia?
`arc land` is burnt into my brain by Phabricator, so I'm aware that the term predates LLMs, but it still drives me nuts.
It's impossible to undo some of these linguistic wobbles. Even if you could filter out 100% of LLM input, the humans themselves are learning to say "land" at a higher frequency now.
Voice really matters in writing. If everyone uses Opus to write without editing, then it all sounds the same regardless of who it came from.
It's kind of offputting how much Anthropic models these days keep repeating "real", "genuine" and "honest". They've RL'd that way over the top.
"You're right for pointing this out. Honestly, your comment raises a real concern — they genuinely RL'd that over the top." - Claude
I had Claude make a world cloud from its responses because I was curious to see how big "honest" would be. It barely showed up, so I asked it to just give me the counts and it responded telling me it was trained not to use the word "honest" much because it makes people distrust responses (in addition to showing me the counts).
From my experience, there are mainly 3 burnout reasons. 1. Multi-tasking is the top one. I usually have to frequently switch between 3 to 5 agent windows which are on different things. It's extremely exhausting when each round takes a few minutes. Before coding agent era, I believe most developers had chance to spend 2+ hours focusing on one thing. Now coding agents have increased my spectrum on the tech stack, but the bandwidth to do deep work isn't increased. 2. Agents are good at getting things running without crash, but do not guarantee to produce correct code. This is quite different from human experts with fundamental knowledge. 3. I also get frustrated when reviewing piles of AI generated low quality PRs. My attention is a limited resource. I don't waste too much energy on other people's work, but if I don't spend more effort, the entire project is corrupted quickly by reckless AI generated code without human author's careful thoughts and designs. Working with people who have less due diligence in mind is painful, working with them in coding agent era is 10x painful because they produce 10x shit. It's a team culture challenge that cannot be easily enforced.
Agreed. I am working hard to restrict myself to only 1-2 agent workflows at a time. More is untenable, though it’s so easy to fall into the trap of deploying an agent “just for this minor fix.”
I do not have the burnout but I certainly operate similarly to the author. I continue to be unable to establish a workflow where allowing the LLM to generate code that I review is faster than writing the code myself. Literally the only two ways out of this dilemma is to blindly trust what was generated or to generate an uncharacteristically exhaustive suite of unit tests to validate every possible scenario. I just write the business logic myself and have the LLM do a lot of the rest. Boilerplate falls into the latter as well.
> generate an uncharacteristically exhaustive suite of unit tests to validate every possible scenario.
This is what you want. You want comprehensive tests at every level, far more than is reasonable for a human to build or maintain, from unit, functional, to full end to end and beyond. Adversarial testing (both TDD-style "write tests to demonstrate this bug", and posthoc "prove this patch wrong with a new test") is the best way to keep AI on track and make those diffs you have to read clean and easy.
An even better way is to use a more strongly typed language and really lock it down, but you can use testing in any language. I feel like my background in TDD and "TATFT" has been secret sauce when working with AI
I see this get mentioned a lot but I still am skeptical that AI can generate tests we can trust more than any other code we know we cannot trust.
Yes tests are conceptually isolated and that helps, but I've personally seen unit tests get generated that are semantically incorrect - that is, they test the structure of the code (e.g. they can check function output types and values), but they can't know _why_ the unit tests need to be there, so the really really helpful tests never get generated. Not to mention the obvious issues with generated tests only testing is x = x, or needless redundant tests for the same thing, or them essentially testing basic features of the language.
AI shouldn't write tests. At least not all of them. Definitely not e2e's. The tests should be guardrails to constrain agents. This way, the author of code matters less.
You have to iterate on the tests, review and validate them, just like any other code, and if you generate a whole project's tests all at once the quality is abysmal, of course. I've been using a lot of old school data-driven testing techniques, where the harness is just code I review, and the data itself is e.g. json files and drives the system.
I actually have a public (AGPL) example here: https://github.com/pgdogdev/pgdog/tree/main/integration/sql - pgdog is particularly testable since it is trying for complete transparency, so you have a perfect oracle in hand via base postgresql, but it demonstrates the concept at least.
Then this falls into the exact same pit the OP mentioned, either you need to blindly trust that the LLM is generating tests that actually work, or you need extensive test coverage for your tests to ensure that your tests are actually testing.
It turns out that you don't actually need tests for your tests, because the code provides a baseline truth for the tests. You do, at some point, have to be epistemically sound enough to actually look for correctness in either the code, behavior, or tests. We unfortunately haven't fully unlocked completely solipsistic value generation yet.
This is also part of why I like end to end tests that use actual UI flow, so I can watch it go by in slow mode before letting it loose fully automated.
Which is why test generation has to be carefully guided as well, and this is something at which I've incidentally been fast. Ultimately it's a constant battle between LLM handholding and doing things yourself.
I don't even care about tests being correct as you can still verify them even when tedious. What I care is that, more often than not, the shape of the solution is not fixed. Having unit tests for those can be extremely costly as when the changes happens, you have to change all the tests.
I've been burned by this in my honeymoon period with unit testing (pretty much the reason it ended). These days, I prefer broader scope of testing, especially user-facing part. The users may be other developers or end users. I only do unit testing for tricky algorithms or math formulae.
I want all the layers of the pyramid, eventually, but the top layers matter the most. I can't count the number of times my paranoid "make sure that customers can successfully pay us" end to end test suite has prevented the money faucet from being shut off. I install one perennially at any company I work at and they always pay for themselves surprisingly quickly.
I’ve been involved in B2B (so no payment flow). But it’s basically the same with an handful of integration tests for common workflows. They run fast and mostly serve a canary to ensure that we are not crippling some use cases. When a bug hits us, a test case is added/modified for it.
They’re mostly a reflection of the current requirement of the project.
I used an LLM to build this
https://github.com/dprkh/eventfs
It has good test coverage, mostly unit tests but also a number of end-to-end tests. I also made the LLM build a benchmark, which you can find at the bottom of the readme. It is obviously slow, but I thought that it is good enough to work. When I tried to write a 1 GiB file, I found that it broke down, and after writing half the file, the speed went to under one megabyte per second. Implementation is 10k+ LoC, and I have no idea what is going on there.
That's interesting because I would feed that benchmark back into the agent and loop over it, to see how much faster you could get it, and agents are really good at that kind of recursive optimization. And I would definitely add at least a simulated 1GiB write test, probably a real one honestly, if I was building something like that.
At least with agent-run tests I care about loop speed a lot, but I care about complete coverage more, so having the odd heavy weight full stack integration test is fine, I think.
You're right. This was just a performance issue, but what if next time it is a corruption bug or a security vulnerability or really anything that can cause real consequences if happened in production? I don't think that LLM systems are inherently bound to have this flaw, but I think that we are pretty far from harnesses and algorithms becoming advanced enough so that the LLM system can kind of continuously evaluate its output and ensure it is good in all aspects.
I don't know about that, Fable is, when properly guided, a better engineer for those things than I am. Narrow breadth, weird priorities, myopic and ivory tower as hell, but superhuman. Maybe that says more about me, or maybe not, but certainly it's caught bugs I would not have, and point it at things like a fuzzer, woo buddy, it has been many years since I broke out valgrind and nailed down a memory leak, but it sure can.
Yep.
100% this is what I've done. I sucked it up and adapted myself to the tool (agents) by having as many implicit guardrails (static typing, functional, no nulls, great linting) and then layering on explicit guardrails (TDD) on top. I also want my workflow to be portable because I don't really trust the frontier model providers.
It is different though. Basically a lot of what I do has changed over the last 2 years. I totally get that a lot of people won't want to adapt though.
> I totally get that a lot of people won't want to adapt though.
Or people don't want to be reverse centaur keeping the clankers happily running. Instead of helping to solve users/consumers problem.
Maybe famous last words, but I'm not buying the hype that the "clankers" will take over. I suspect reality will catch up soon and we'll be left with a set of pretty powerful but still limited tools. I see no evidence to the contrary, just investment hype on one side and sky is falling on the other.
Yeap. Hundreds of tests for small tools are completely trivial now.
I've just been carefully reading the code. It is easy to slip into just accepting what comes out to speed things up, but reading the code is important.
I save myself by skimming things like tests, templates, some UI. Anything cosmetic. But I have to read the majority of code that ends up on my back end systems.
And for those that have similar-ish sentiments, what mental defect is had that prevents them from just drinking that sweet tasty kool-aid and just use the slop created. What demented trait is in them that causes everyone to just be a stick in the mud trying to ruin everyone else's good time?
In my personal experience, the ones most enthusiastic about LLM magic are those that can't code, but can now walk away with something functional if not quite the best code. Now that they can produce workable code, it will make everyone better. Yet, they have no idea how maintainable the slop is or if it's slop at all.
Don’t read the code!!
I actually dispute this, I read all the code, the core thing people have to give up is not "reading the code" per se, it's giving up on "that's not how I would have done it".
When you see a perfectly clear function or object that just isn't your style, you have to accept it and move on. Where there are concrete concerns, or it's unreadable, demand excellence, but treat it like a coworker, not an IDE.
This all reminds me of the differing experiences people had outsourcing coding in the 2010s when it was still called oDesk. You don’t need to read code, you just need to know that the code works. If something doesn’t show up as a problem it doesn’t need to be fixed, and reading code is the least efficient way to discover problems.
The only time I look at code is when something isn’t right and I ask for a root cause analysis. The LLM will show me some offending code or code for reference or evidence and then I quite often say “well that’s dumb you should do it like this instead” but I never need to actually go into the files. I do sometimes look at a git status or git diff.
Yeah this is how I feel about it. Does it look correct? is it doing something weird? Is it forgetting about some gotcha in our domain that it hasn't been taught about yet? Otherwise, ship it.
It sounds kind of like being stuck working with coworkers who--while not overtly hostile--need constant hand-holding and repeat the same kinds of mistakes every day and can't even be genuinely sorry about it.
Just because we work with computers doesn't mean we don't take, er, social-damage. Or perhaps parasocial damage, in this case.
This is legitimately the reason I'm looking to leave programming.
I got into programming because the problems of programming were interesting to me. But if the problems go from "figure out why this calculator is off by one in France" to "Get this LLM to stop spamming cutsey emojis", then maybe it's time for a career change.
Giving my "otherside", because the pressure to output more at work is real, but at the same time, out side of work, I love this. I'm able to do way more projects than ever before because a barrier to entry was always the amount of research+time required to start up a pet project.
My latest is, I'm really into fizzy/soda water and wanted my own continuous carbonator. My entire build from water source to tap with an ESP32 controlled pump, pressure, water level, cooling fans.
There were so many areas I made mistakes in my shopping cart and it found it - like Home Brewer likes 8mm lines but water filter systems like 9.5mm. Really optimized the versions from a simple on/off pump w/ float switch to effectively a full on PLC system. So many iterations gained by chatting with "someone more experienced". Once I get the parts I can build and have the software side running in less than an hour.
It doesn't make money, but man I really enjoy it.
I got into programming to just build stuff, the coding is just a means to an end, try not to think too hard about the how and think more about the why and what
I got into programming because I like programming, and computers. Not whatever the hell this is.
> Not whatever the hell this is.
do you mean my enjoyment from building things? I'm genuinely confused by this response.
We get it, you don't have a passion for the act or the craft, just the end result, but I'm absolutely sick of hearing it all over this site as if it's a universal truth that some of us just don't recognize yet.
Sorry, some of us have a joy for programming where the how is just as important, if not more so, than the what and the why. No matter how much people proclaim that the how doesn't matter to them, it isn't going to suddenly make it true for others.
You okay?
This isn't a response I expect from people who are here for a productive discussion. I'm sorry that you are sick of hearing this, but I'm not responsible for making sure you only read what you find worthy of your own personal brand of respect. Instead of attacking someone for simply offering their point of view, in what appears to be a quasi-gatekeeping effort, maybe you should look inward and discover what is making you this upset toward a complete stranger.
__I cannot take away the joy you have for programming simply by stating what drives me.__
I'm totally fine, just annoyed by how much this "try not to think too hard about the how and think more about the why and what"[0] is getting brought up every single time someone mentions why they prefer hand coding to vibe coding. At this point, it's being overstated, thus gives off less "this is why I use it" and more "come on, get onboard the hypetrain to dystopia or you're going to get left behind". It's hardly conducive to a "productive discussion" when it's the millionth time as a drive-by comment. What kind of response did you expect?
Look, I don't have a problem with your personal motivation. I just hate seeing it suggested that people should abandon their passion because someone else doesn't share it. There's absolutely nothing wrong with "I enjoy making something useful" just as there's nothing wrong with "I enjoy making something with my hands or figuring out how to make it". My problem is with "your enjoyment of that is invalid because I don't enjoy that, so learn to enjoy this".
0: Not really a statement of your drive, is it? More of a directive or suggestion.
Claude: Solve this jigsaw puzzle for me...
If solving jigsaw puzzles with claude will enable the creation of tools that help the people I want to help (students with disabilities), then I would use it for that, without feeling any guilt at all for doing so.
Why should I regret that? Why should I care about your purity tests?
Part of the annoying thing is that if you're working on a product which uses LLMs, at some level you run out of levers to pull in terms of being able to fix things. At best you're stacking hacks on top of hacks to prevent unwanted output, but at the end of the day if the LLM really decides it simply doesn't want to follow your instructions, you can't do much other than resign to adding *IMPORTANT* and hoping the next model fixes it.
The experience is much closer to working with an external API that you don't have control over and which simply doesn't do what the documentation says. Those have always been the most frustrating parts of programming, but at least previously you could reverse engineer the actual implementation to work around bugs. You can't even do that now because the "boundary" randomly change every day.
(And I admit I'm salty that the "I don't give a shit about why the calculator doesn't work in France, I'm just here because they pay me to fix it" people were the ones vindicated by technological progress)
Accounting is desperate for accountants because they’re necessary for legal and compliance reasons. Join up today!
I'm no longer in corporate America, so maybe I'm out of touch a bit, but could you just...not...use an LLM? You can still solve interesting problems on your own if you choose to do so?
It’s not there yet but we’re clearly heading towards a world where the answer is “no, you have no choice”. AI is weaved into business processes. If Ai leaves a comment on your pr, you must resolve it before merging, you’re expected to “get things done” at a particular pace consistent with using ai, regardless of whether what you did is any good.
LLM skew the time estimate tho. Now everybody expect stuff based on LLM work instead of normal human work. I/we can choose to solve problem normally, but the expectations have changed.
Yeah at many places you still can. It’s just so easy to turn your brain off and let the robot do a maybe good enough job that even people who know better are merging slop.
We’ve had 3 production incidents this week that slipped past CI because there’s a whole team that is just shoving out PRs without understanding what’s going out.
A lot is said about context you can feed into the LLM but I do think there is still superior power in human context awareness. That kind of ambient collation and organisation of the whole business and its purpose, all the different work going on and how it all relates to eachother. It happens when you isolate business units a bit too much also.
It's not surprising that if you have a hundred separate, isolated contexts working on the same business, that don't cross-talk and have no ability to subconsciously receive and collate, prioritize the thousands of signals we get from our work environment, that you end up shipping lots of incomplete or incompatible work.
Leave programing for what though?
That is the conundrum, and I genuinely enjoy building software.
iwould advise that if you love it, stay and coast.
this AI bubble will pop. when it does you'll be hot stuff all over again.
I get burnout from frustration when the LLM just can't follow instructions.
Like when I'm trying to get it to create an image, and the first pass is beautiful, but ten different request to modify it, with different phrasing and even example images, produce the same image ten times. Or when you tell it not to use a cheap hack in AGENTS.md about six different ways and in your prompt, and it still does it again, and again.
It's like arguing with an idiot. And THAT gives me burnout.
Also: I've never once seen an emoji in LLM output. What are people talking about?
Excessive amounts of emojis in generated README.md and sometimes in printed/logging outputs. I don't whether this is still an issue because I have a "Never use emojis" instruction in the context.
I’m currently doing a project with someone who only uses LLMs, and it’s exhausting and mentally draining.
Whenever I give feedback on something, the answer is just “let me tell Claude”. The person has no understanding of how everything works, and most of the code reflects that.
The other day he hardcoded in a demo mode, simply because he didn’t even know how to set up a local environment and set environment variables. I’m confused as to why Claude didn’t even knew this, but it might just be the prompting.
I limit LLM usage myself, and if I do use it, I try to use it on extremely specific tasks. It’s the only way it works for me.
I honestly don’t understand how all these companies are getting away with generating AI code. Even in a small project I quickly fall behind on my understanding of the project.
I remember working with people like this before AI and it was annoying but they struggled with productivity because they didn't understand what they were working on enough to produce good code efficiently, so the problem usually took care of itself.
Now these people can thrive because LLM coding encourages the incurious and punishes the deep thinker.
At least to me, before LLMs, those people could somewhat be trained. You could give them feedback, and they would improve and learn the code.
Now your feedback is just another prompt for them, the code might be slightly better, but the person learned nothing from it.
LLM generated code is obtuse and dense, unnecessarily so
LLM burnout is when you catch yourself asking ChatGPT 'how are you' and genuinely waiting for an answer that isn't 'I'm an AI language model'
It's amazing how many do not have experience with the full pre and post SDLC.
is there any evidence that Alec Scollon, the first time blog author responsible for this post, even exists? look up the name. boo this post and the premise behind it.
The domain "alecscollon.com" was also registered today, according to WHOIS
It's burnt me out too. I'm generating 10x more features and multitasking across 4 disparate projects. My greatest concern is I don't really have a strong connection to the underlying fundamentals anymore. I need to see how the things works to internalize it. Now I just trust that the agent wrote this piece correctly.
The productivity drive and the sheer feature set you can generate in record time makes it easy to forget proper sdlc hygiene.
Knowing how things work, knowing what should be possible and where “there be dragons”, and having a pretty well-developed “sixth sense” for all kinds of things is proving just as valuable with LLM-heavy programming as it did before.
… but I am almost certain I’d never have developed those in the first place if I hadn’t spent 25ish years programming on a bunch of different platforms and setting up servers and networks and all that, without LLMs.
I dunno how you make another “me”, now, while before lots and lots of programmers naturally ended up as someone with skills and knowledge like mine, and those skills seem super useful when writing code with LLMs.
Burn out? This guy is weakling with zero experience in full QA and validation.
Come on, Claude is not that bad.
> My job has changed from designing and writing code to designing code, describing the design to an LLM, reviewing code the LLM produces
As a long-time engineering manager, PM and, eventually, product owner my response is, "Congrats! You've just been promoted to management." :-)
As a new manager, your first challenge will be successfully delivering commercial results using only a team of 'differently abled' new grad interns. Don't complain, new managers don't get to pick their first team! To be honest, these guys are more like alien brains raised in a vat with no direct senses. They've only ever experienced a data feed of the internet and, oh yeah, they get near-total amnesia a few times a day (but maybe you can teach them to write notes for themselves). They also have ADHD and are somewhere on the spectrum. But don't worry because what they lack in common sense, experience and intuition is offset by having a sort-of photographic memory and a willingness to grind on a problem 24/7. You should be fine. Good luck, we're all counting you...
I don’t understand what could possibly need to be made so fast that isn’t totally made up billable hours. Running at top speed long enough to be burned out is either ineffective, or valuable enough that someone else can take over while you sleep.
Even with Fabel and all that I constantly keep having to babysit it and correct it like it’s an adolescent and it gets really old and the amount of code. It produces not all of its great at all. I’m burnt out looking at. It’s poor coating that somehow magically works.
What to do to achieve this kind of burn out, I feel like I am a stubborn old developer, I'm coding since 2013 and I am 25 years old.
My mind still can't function well without having knowledge about everything.
split jobs and managing people is hard.
Do does managnig agents.
If an LLM was a person, it would be known as a "bullshit artist". I've worked with bullshit artists (people) before, and its exhausting trying to separate fact from fiction. It's a reason to avoid working with such people.
A good coworker will admit not knowing something, or if unsure give their best guess but discuss its limitations and why they might be wrong.
Question: Has anyone experimented with using voice to directly prompt an LLM, without doing speech-to-text? If an LLM can pick up on the skeptical nuances in a person's response, it might be prompted not to be overconfident in its subsequent output.
> My main project right now is to establish a framework for large-scale, unsupervised code generation in our codebase
Anyone else working on something like this or know of any projects attempting it?
I'm building something for that.
I've taken a bit longer than I wanted but it will be open sourced soon.
It's a durable orchestration engine that takes in specs/requirements and coordinates agents externally (meaning the engine drives the loop, not an agent) until the work is fully implemented/verified and reviewed.
It's meant to be used with any harness as basically the last step. You plan your work with whatever LLM you use and then hand off implementation to the engine (through an MCP server or other surfaces)
It can use your OpenAI/Anthropic subscriptions or any other provider and you can mix and match models across implementation and review in any way you want with fan out for parallel reviewers and more.
The goal is to produce high quality unsupervised code that matches your requirements and is reviewed throughout the implementation rather than at the end only, so that mistakes don't compound.
https://engine.build if you want to get notified when it releases.
https://github.com/gastownhall/gascity is certainly a choice. I enjoyed playing with gas town but it was a little too nondeterministic for production code, I think.
Directionally if what you're doing is straightforward it's an amazing experience to be able to slap in an epic planning document and wake up the next day to it being "done", with a big asterisk that done-ness is directly proportional to how good of a spec and how good of a model you were using.
That being said, these days if you use Fable, slap in an epic planning document, and ask it to run a workflow (be sure to specify that subagents should use, say, Sonnet, or wave goodbye to your wallet), it's almost as good as gastown/gascity but far more predictable.
An almost infinite supply of such modern-day alchemists.
Current AI is like the film company producing TV series or movies
Review AI code line by line is like watch movies frame by frame, and is impossible, very difficult, terribly boring, or abandoned sooner or later.
I think having style guidance in your context is valuable for avoiding this kind of thing. Having to read awful, cliched text all day is even worse than having to read reams of useless code. I have some simple humanizing content in there that specifically calls out the rhetorical devices that AI loves, and it drastically improves the diffs and comments. It also makes the coding performance generally slightly worse, but ergonomics uber alles.
I feel you. 'But we need to adapt or quickly become invaluable'. It is a tough reality to swallow - me included.
I don't have much success with using the LLM to make changes to a big legacy codebase. Instead, I use the LLM to gripe about things I don't like in the code. Usually, it is a brilliant commiserator.
If you can afford it, I highly recommend quitting and going independent at least for a while. It’s so much fun building right now.
Can you elaborate on what you're doing to make a living?
Avoid Gemini and the lesser ChatGPT models and your emoji problem goes away.
I don't mind interacting with LLMs myself and find they increase my productivity a decent amount. I just can't stand dealing with other people's slop.
Getting sent IM responses that are copy pasted LLM nonsense. Getting a massive PR to review that was generated overnight and the author didn't read it first.
So annoying, I'm dealing with a client that just constantly feeds my responses to their question into AI. Which of course just asks more questions and tells me how clever I am. I also know the client isn't reading because in many spots I put "[Client Name] you need to answer this directly as we need to know the actual real business detail" and its ignored or the AI provides a detail I know is made up.
Also known as clanker fatigue
i think we're interacting with a character not a person.
I don't really understand how this isn't a self-inflicted problem? Perhaps it's because I'm not really mandated to use LLMs in a particular way, but I've had great success doing a combination of writing code myself and using smaller but faster models as a sort of "flood fill". The larger models can also be useful when you're implementing something which already exists in similar form in the codebase, because you can just put that code in the context and you'll get something very similar outputted. So the more code you write, the better the LLM can be later on. Codebases should get easier to add to the bigger they get, not harder.
Of course if you're supposed to achieve so much output that it's not possible to do anything but vibe it, fair enough.
Tell the llm to answer like a cavemen, if llm talk like cavemen, the answers become shorter and more compressed.
https://github.com/JuliusBrussee/caveman
It's for getting it to output shorter answers, but also could help with your burnout.
I surprisingly had good results when I told the LLM to only communicate in ASCII memes. It did a fantastic job of summarizing the situation using relevant memes, and the humor was enough to keep things fresh. As silly as it sounds, it's worth trying when you're in that LLM burnout corner.
started to dread reading LLM output because I know what I’m going to find. False assumptions and hallucinations. Emphatic, staccato fragments. Excessive emojis .
I do not understand these complaints. Yes, those are the defaults and they're annoying, although the general public seems to like them. But you are not stuck with these. You can just tell the LLM how it should interact with you. If you're using any sort of harness beyond the chat window in a web browser, you can codify these instructions in a rules.md file or similar and have it automatically included in any new chat. It's not any harder than changing the default wallpaper or color scheme on your desktop operating system.
In reverse order, you can just tell the LLM to never use emojis. I don't like emphatic staccato fragments either, so I tell it to eschew the language of marketing and hype and stick to a factual and plain language, or to employ an academic tone. I explicitly instruct mine to ask clarifying questions whenever context is ambiguous and to push back on false assumptions or common misconceptions (by me). Hallucinationsa re the biggest problem of those you mention; it's not easy to totally eliminate them (for the same reason it's not easy to instruct people to not fall for scams or disinformation), but you can considerably reduce them by setting standards for citations.
I have ideas about reducing hallucinations over work material (ie a codebase) but am omitting them here as they are not fully thought out or tested.
I had this too. Started after lay off in October.
What helped was a sleep and work system, oriented around being offline that was inspired by nature and from my earlier days in working in tech while car camping across the national parks.
Basically: the sun wins in terms of how all energy on the earth is structured, and expressed. All manners of cycles of organisms and living systems are in relation to its rise and fall, and even its particular color spectrum phases (whether thats night oriented or day). I call this our real circadian rhythm; it's used to being signaled by the light of the sun and maybe fire for millions of years and it isn't until recent centuries when we started tricking our biology with LEDs and lights. So the solution is simple. Orient yourself around the light of the sun and make sure it's the first and last major light source you see; blue limiting is the most important part BEFORE sunrise and AFTER CUT OFF ALL BLUE LIGHT. On my Mac I use a red light filter (using it now, it's 11:07pm ET and the sun went down about 2.5 hours ago). It's really hard to stay alert and chatting with an LLM when the only light sources are red and you keep them dim at that. Our ancestors would rest when the sun's at its peak (~1:05 pm today) and that's a good time to divide my own day productively as well. With intentional breaks diving the middle of the day with sunlight anchoring it, my nervous system is more relaxed, and by the evening time, it's also ready to transition out of anything blue-light assisted and most intellectual work and problem solving falls into this bucket. It's really hard to explain but it really works so simply. To enjoy the process a little more I made this fun sun clock, check it out at https://sunsignal.app
I once experimented with beeswax candles as my only after-dark light source. This meant no hyper-stimulating screen activities whatsoever, too. TV, phone, video games, browsing the web? Nope, nope, nope, and nope. Just dim, warm light from actual flames.
Cured my lifelong “night owl” “trait” in a couple days. Shockingly effective.
Turned out to be hard to keep up and still, like, exist with other people, and you’d probably need to relax it a little in Winter unless your job lets you work reduced hours to kinda “hibernate” (otherwise when would you do anything that’s not work but requires light or electronics?) but it sure worked.
LLMs poison your mind. The more AI slop you read, the more your mind turns into something like slop. This isn't very different from the idea that the food you eat is what your body is made of.
It sounds far fetched at first, but I think it could be true. There was a time in my life when I read literature constantly and started using those patterns in my writing. The habit fell off quickly after I stopped reading for a while, so there might be hope.
Learn to code
How?
Ask random questions until you get enough sense of things to ask "the right" questions. Never stop asking questions. Use everything you can to get the answers and never settle for any answer no matter how good it sounds, doubt it and go back to step 1.