What I'm Hearing About Cognitive Debt (So Far)

(margaretstorey.com)

90 points | by raphaelcosta 2 hours ago ago

36 comments

  • Ozzie_osman 8 minutes ago

    I think the antidote is ownership. Every part of a company or product needs some person or small group that owns it, understands it, and feels ownership over its long-term health. They review or decide what level of review changes to it can take.

    So much of what makes high-functioning teams work is a sense of ownership and stewardship, and what makes low-functioning teams break is a lack thereof. Someone with pride, drive, and a high standard feeling responsible for a particular area or thing.

    In the past, that ownership could be individual or collective, but with AI and a lot more lane-crossing, ownership should tend toward smaller groups (or individuals).

    A developer can design, but a designer needs to review it. A designer can code, but the owner of the code must review it.

    This might feel like gatekeeping, but it's the only way.

  • hogehoge51 an hour ago

    Unfortunately I think Cognitive Debt is the cry of the software craftsperson who thought they were an Engineer. Upon working with the agent subcontractor, the agent factory, the agent part vendor, they approached it as a craft; they found themselves wanting to walk through the offices of the subcontractor reviewing screens, inspect pieces at the factory, and get the internal design for the parts they ordered. It's natural to get overwhelmed: this is why Engineers have contracts, specifications, design drawings, datasheets, and characterization data, handed over at clearly defined boundaries of abstraction, accepting the other side may be a black box.

    Of course, we have had compilers and tooling, but those are the pencil and drafting board of the draftsperson. An ecosystem of packages, dependencies and APIs has evolved, but those are often just spells the software magician invokes after reading the spellbook^H^H^H^H^H^H^H^H^H stackoverflow^H^H^H^H^H^H^H^H^H^H^H^H^H API documentation.

    We are going to need to build a new set of boundaries and abstractions with new handover protocols to manage this mess.

    • platzhirsch 7 minutes ago

      Because waterfall software engineering has been so successful, right? ;-)

      • saulpw a minute ago

        Because agile has been so successful, right? ;)

  • pizzly 42 minutes ago

    Cognitive Debt has existed much earlier before LLMs became mainstream. Technical people got good at their jobs and then was promoted to management. After time they lost their technical abilities but if they are a good manager they kept up to date with the technological landscape and used their engineering thinking to ensure that the people below them worked to their optimum efficiency to achieve the companies goals.

    Now we all know horrible mangers who didn't keep up to date nor used their thinking. This will happen with AI useage too. What is more we are expecting people who are engineers to have a manager's mindset (by managing AI agents, products requirements, etc). Many engineers are horrible at this and have no desires or ability to become a manager. This is why they went to engineering in the first place.

    • yuye a few seconds ago

      >Many engineers are horrible at this and have no desires or ability to become a manager. This is why they went to engineering in the first place.

      Bingo. If I wanted to spend my life managing incompetent sycophants, I would've studied for an MBA to try to rise the ranks at McKinsey.

    • casualscience 34 minutes ago

      While this isn't a unique perspective, I think it's wild more people don't understand this. What happened is everyone is being "promoted" to staff+ level engineer and they're realizing the realities of that situation.

      The funny part is that these are the same people who are upset that these folks up the food chain "do nothing".

    • yuedongze 15 minutes ago

      This, and I would even say we are promoting people to be kings and queens. I'm afraid AI will amplify our worst parts because they are ultimately sycophants. I've heard so many things about AI enabling a single person to run a billion dollar business. But I believe without the right mindset/discipline, a person cannot go too far with any technology.

  • afro88 2 hours ago

    I find it disconcerting that an article about cognitive debt contains many "tells" of being written by AI.

    • unignorant an hour ago

      I had the same reaction, but the article is not AI-generated according to pangram, which I've generally found reliable. I wonder if LLM turns of phrase and even thought patterns are creeping into normal human thought.

      • Zetaphor 23 minutes ago

        It's worth mentioning pangram is more confident in it's positive detections than it's negative ones, as stated by the founder in an interview on the most recent ThursdAI episode

      • pizzly 37 minutes ago

        I think its bidirectional. We change our writing based on what we see (AI generated content on the internet) and AI will learn based on what we write.

      • erikerikson 41 minutes ago

        Or, stay with me here, the LLMs were trained on how we, statistically, write.

        • shiveenp 11 minutes ago

          Anytime I see “this is not just x, it’s y” i can almost guarantee with high degree of confidence that slop was used.

        • unignorant 21 minutes ago

          There are typical LLM voices and styles, just like human writers have differentiated voices and styles. And some common elements of the typical LLM style are distinct from humans I've previously read.

          • erikerikson 10 minutes ago

            I recognize this. It's also the case that I suspect that I've read more about how annoying suspected LLM output is to read than I have read LLM output. The slop is, to me, an incredibly unwelcome contribution of humans that don't enjoy the craft but complaining about it is equally stuck in and further exacerbating the froth rather than distilling down to the substance. That is it keeps the focus on the surface rather than on what the core content is and whether it has value.

          • donaldjbiden 10 minutes ago

            I heard the typical LLM style closely reflects the writing habits of low-class English-speaking Kenyans. You already know why.

    • chromacity an hour ago

      Independent of that, the article is just a summary of a HN thread...

  • 01100011 an hour ago

    These sorts of articles just seem silly to me. Use AI where it helps you and avoid where it doesn't. That dividing line may change week to week.

    I think it's great for writing tests and sanity checking changes but wouldn't let it write core driver code(I'm a systems programmer so YMMV). Maybe in a month I'll think differently.

    • Eufrat 34 minutes ago

      Using a tool as a tool is hard when the market is telling you to use it in everything as if it’s the new sliced bread.

  • darth_avocado an hour ago

    > the question becomes how teams will manage cognitive debt

    That’s the neat trick kiddo, they won’t. Across the industry, the messaging is clear: use AI and be more productive. Management is salivating at the idea of getting rid of people and keeping a higher share of profits for themselves. Most ICs I talk to are increasingly expressing the feeling of burnout, fear of losing jobs and resentment that AI is being pushed the way it is being pushed. I have more than a few conversations where people have clearly expressed that they are mostly focused on keeping their jobs. They don’t care about cognitive debt and some are looking forward to the time when the debt comes due.

    It is depressing, but it is the reality.

  • skybrian an hour ago

    Sometimes people make an assumption that every codebase has a team (or at least a single person) devoted to maintaining it. Companies with large codebases may not be able to afford that, or don't think it's worthwhile. You could have dozens or hundreds of libraries and only a few maintainers. The libraries are effectively "done" until something comes up. Work on them is interrupt-driven.

    In that situation, coming in cold to a library that you haven't worked on before to make a change is the normal case, not "cognitive debt."

    If you have common coding standards that all your libraries abide by, then it's much easier to dive into a new one.

    Also, being able to ask an AI questions about an unfamiliar library might actually help?

  • protocolture an hour ago

    Its more and more clear to me that AI is a force multiplier for small teams and hobby workflows, but seems to have diminishing returns for larger teams.

    • melvinroest an hour ago

      How so? Could you give some specific examples?

      • hogehoge51 41 minutes ago

        My experience moving between startup/SME/corp:

        Smaller teams have more agency to move and usually team members with broader responsibility and understanding of the systems. Also possibly closer to stakeholders, so are already involved in specification creation and know where automation can add value. Add an AI agent and they can pick and choose where they can be most effective at a system level.

        Bigger teams have clear boundaries that stop agency - blockers due to cross team dependencies, potentially no idea what stakeholders want, just piecemeal incremental change of a bigger system specified by someone else. If all they can do is automate that limited scope it's really just like faster typing.

        • melvinroest 7 minutes ago

          Ah, I feel what you're saying. Yea that makes total sense.

  • BLKNSLVR an hour ago

    Just reading the first paragraph and I've already started to experience it when attempting to apply AI to Acceptance Criteria that testers have to test against.

    The software is necessarily complex due to legislative requirements, and the corpus of documentation the AI has access to just doesn't seem to capture the complexities and subtleties of the system and its related platforms.

    I can churn out ACs quicker, but if I just move on to the next thing as if they're 'done' then quality is going to decline sharply. I'm currently entirely re-writing the first set of ACs it generated because the base premise was off.

    This is both a prompt engineering and an availability-of-enough-context documentation problem, but both of those have fairly long learning curve work. Not many places do knowledge management very well, and so the requisite base information just may not be complete enough, and one missing 'patch' can very much change a lot of contexts.

    • nevdka an hour ago

      I work with Australian tax - lots of regulatory complexity, add the documentation often assumes the reader is a CPA. I've got decent results by telling the chat bot to ask questions instead of making assumptions, and then grilling it to find edge cases.

      I did a live demo in front of the CPAs, using their documentation, and Claude asked clarification questions they hadn't thought of and exposed gaps in the old manual processes.

  • gdulli 2 hours ago

    > High-performing teams have always managed technical debt intentionally.

    The ability to generate code has seemingly transposed what people think of as a "high-performing team" from one that produces quality to one that produces quantity. With the short term gains obviously increasing long term technical debt.

  • nik282000 34 minutes ago

    > the accumulated gap between a system’s evolving structure and a team’s shared understanding of how and why that system works and can be changed over time

    That just sounds like everyone is going to be management. Blindly setting goals and demanding features of a black box, formerly the development team, soon to be 'AI' agents.

    • anilgulecha 25 minutes ago

      I mean that's already happened. Everyone is expected to be a manager of agents. Anyone not doing this is programming for hobby.

  • 0xbadcafebee 16 minutes ago

    For my entire career, most of the people working on the systems I've worked on have not known how the systems work. Sometimes a few people learn a great deal about the system. They are spoken of in hushed tones, as one would a powerful mage, or witch. However, never in my entire career did anyone complain about the fact that nobody knew how the system worked. Most people never noticed it, much less cared once they found out. "This is just how things are. Business as usual."

    AI is making people afraid as they run into these things. It's a little sad that they don't have the historical context or perspective to realize these are old problems. I imagine this is what Samurai felt like as flintlock guns came in and completely upended hundreds of years of martial tradition. How will they be able to defend themselves if they don't learn Kenjutsu? What will happen to our Bushido?

    And I do think the fear is warranted. But I don't think people are going to act differently once they realize this unfortunate status quo hasn't yet led to the collapse of civilization, or their paychecks. Once the fear has passed, we will move on into the new normal, willfully ignorant and mildly disappointed.

  • melvinroest an hour ago

    Edit: yep, I really do type this much. I'm a bit of a "thinking out loud" person.

    > Cognitive Debt, Like Technical Debt, Must Be Repaid

    In quite a few circumstances, cognitive debt doesn't entirely need to be repaid. I personally found with multiple projects that certain directions aren't the one I want to go in. But I only found it out after fully fleshing it out with Claude Code and then by using my own app realizing that certain things that I thought would work, they don't.

    For example, I created library.aliceindataland.com (a narrative driven SQL course). After a while, I noticed that the grading scheme was off and it needed to be rewritten. The same goes for how I wanted to implement the cheatsheet, or lessons not following the standard format. Of course, I need to understand the new code but I don't need to understand the old code.

    With other small forms of code, I just don't really need to know how things work because it's that simple. For example, every 5 minutes I track to which wifi network I'm connected with. It's mostly useful to simply know whether I went to the office that day or not. A python script retrieves the data and when I look at it, I can recognize that it's correct. But doing it this way is sure a lot faster than active recall.

    At work, I've had similar things. At my previous job I created SEO and SEA tools for marketing experts. So I remember creating this whole app that gave experts insights into SEO things that Ahrefs and similar sites don't, as it was tailored to the data of the company I worked at. The feedback I basically got was: the data is great, the insights are necessary, but the way the app works is unusuable for us. I was a bit perplexed as I personally didn't find it that complicated. But I also know that I'm not the one using it. Then I created a second version and that was way more usable. The second version assumed a completely different front-end app and front-end app architecture though. All the cognitive debt of V1? No payback needed.

    The reason that this is the case, as it seems to me, fall under a few categories:

    1. Experimenting with technologies. If you have certain assumptions about how a technology works but it turns out you're wrong, or you learn through the process that an adjacent technology works way better, then you need to redo it. Back when coding by hand was such a thing, I had this with a collaborative drawing project called Doodledocs (2019). I didn't know if browsers supported pressure sensitivity and to what extent it was easy to implement. It required a few programming experiments.

    2. It's a small and simple script, not much more to it.

    3. Experimenting with usability. A lot of the time, we don't know how usable our app is. In my experience, this seems to be either because (1) it's a hobby project or (2) the UX people have been fired years ago. In these cases, more often than not, UX becomes an afterthought. But with LLMs, delivering a 95% fully working version is usually done within a week for a greenfield project. This 95% fully working version is an amazing high fidelity interaction prototype (95% no less). Once you do that for a few iterations, you then understand what you really need. Once you understand what you really need, then you can start repaying the cognitive debt.

    I've found it's usually category 3, sometimes 2 and rarely 1.

  • saltyoldman 26 minutes ago

    It is a bit surprising sometimes when you vibe code an AI tool and it ends up doing a bunch of regular expressions to "detect the user's intention". Instead of the code using an LLM to see which tool to run, or if they want to see the SQL or the code you end up seeing .*SQL or \i^build (or some crazy regex). It really likes to use a lot of regex when it's building AI tools.

  • bugbuddy an hour ago

    The most important lesson from Gen AI is that it does not matter how much money you have, make, lose, or spend because in the long run everyone is…

    So the logical next step is to focus on Biological Immortality and short of that Digital Immortality. God speed everyone.

  • jdw64 an hour ago

    Personally, my observation is that “cognitive debt” feels closer to a tool for selling essays than a precise engineering concept.

    Lack of documentation, failed onboarding, poor architectural understanding, missing tests, review fatigue — if all of these are simply grouped together as “cognitive debt,” isn’t that just a failure to build a proper workflow?

    The scope is too broad. It reminds me of Stepanov, the creator of the STL, saying that if everything is an object, then nothing is.

    When an abstraction tries to cover too many things, that abstraction inevitably fails.

    The way AI specifically amplifies this problem is through the difference between direct work and indirect work. The core issue is that “it works” can easily create the illusion that “I understand it.”

    Another thing I felt while reading this essay is that it almost seems to go against the direction of modern software engineering. Once software grows beyond a certain size, it is already impossible for anyone except perhaps the original designer to understand the entire system. The goal is not for everyone to understand everything.

    The real goal is to make local changes safely, and to ensure that the system keeps running without major disruption when one replaceable part — including a person — leaves.

    At this point, many things being described in the industry as “cognitive debt” look to me like rhetorical tools for selling essays.

    Reading this, I even wondered: if I write about trendy terms like cognitive debt or spec-driven development on my own blog, will people pay more attention?

    To be honest, spec driven development has a similar issue. When you go from a specification down into implementation, information loss is inevitable. LLMs cannot fully solve that. In the end, a human supervisor still has to iterate several times and tune the result precisely. The real question should be: how far down should the specification go? In other words, at what local scope does it become faster for a human programmer to modify the code directly than to keep steering the AI-generated code?

    But that discussion is often missing.

    As people sometimes say, “when you start talking about Agile, it stops being agile.” In the same way, I think the “cognitive debt” frame may be a flawed abstraction of the current phenomenon.

    The moment a living practice is nominalized, packaged, and turned into a consulting product, it loses its original dynamism and context-dependence, becoming a dead template.

    It puts various discomforts that emerged after AI adoption — review burden, lack of understanding, fatigue — into a single box.

    Then it attaches the economic metaphor of “debt” to emphasize the seriousness of the problem, and subtly injects the normative idea that “this must eventually be repaid.”

    Thinking back to Parnas’s 1972 work on information hiding, software engineering was built on the principle that local understanding should be sufficient, and global understanding is not the goal.

    The cognitive debt framing seems to implicitly reverse that principle by treating “shared understanding” as something that must be preserved as a global unit. I do not understand why the discussion keeps moving toward the idea that everything must be understood.

    It reminds me of Bjarne’s onion metaphor for abstraction: if an abstraction works, you do not necessarily need to peel it apart without reason.

    My main issue with the current cognitive debt framing is that the layer it tries to cover is too broad.