This was a fantastic article and beautifully captured something I had been circling but hadn't quite put into words:
> The notetaking people—and I say this with all the love in the world—are never, like, a researcher at the cutting edge of their field, building this vast cathedral of knowledge, note-by-note, so they can derive new insights. Never a historian who has to read tens of millions of words across thousands of sources to synthesize the life of some historical person. It’s never someone doing something hard. It’s always some blogger. Their “digital garden” is about how to keep a digital garden. It’s very solipsistic: there’s no output, no deliverables. The deliverable is you take a screenshot of your Obsidian graph and tweet about it to show off how much it looks like an incomprehensible ball of twine.
> Sometimes, tools don’t move the needle because there’s no needle to move.
It reminds me of something my old CS mentor, now elderly, had said about LLMs a few months ago: "it's a force multiplier, but there has to be some force to multiply."
This likewise is a basic fact I encounter over and over:
> Knowledge is another limiting factor. I find that even very educated people tend to underrate the importance of knowledge. A lot of people have this attitude that you can just Google everything just-in-time as it comes up. Like Babbage, I can’t rightly apprehend the confusion of ideas that would lead someone to think this.
> It’s always some blogger. Their “digital garden” is about how to keep a digital garden. (...) never a researcher (...) never a historian
It's quite possible that those researchers and historians do exist, but are working instead of posting about notetaking. It's like saying the people who write Python tutorials never seem to ship anything impressive, so Python must be useless.
> Could you imagine if you found everything interesting? You’d spend years living in a basement curating a wiki of late Soviet military hardware or something.
This was a good read but this felt like a personal attack :)
As a product manager I have found people's understanding of what they want to be tenuous at best. They know they want it to be easier, less frustrating, and do the darn thing they want. But collecting the input of all the people with the power to make choices, boiling that all down, and actually figuring out what will work is a total bottle neck. Technology and programming have hardly ever actually been a blocker. Legacy systems, conflicting requirements, and ever shifting choices represent a much harder problem.
This is all very right, but I'd like to add this: As your capacity to deal with abstraction (which is a function of intelligence and executive function and, to some extent, knowledge) grows, you become more and more constrained by the extent to which you can manipulate symbols. AI-based solutions for that are potentially really powerful, and that's the mechanism through which, as TFA says, "intelligent, educated people with working reward circuitry stand to gain more from AI."
And I'd also add that AI strongly disaggregates the returns to different levels of the capability to deal with abstraction -- higher levels get more, lower levels get less -- rather than uniformly boosting returns across the board (unfortunately). Of course, this has been the trend of information technology since at least the '80s, but now the slice at the top is really small and the returns very high.
Heh, joke's on the author. I've built both a flashcard app and a notes app using AI, and I've learned more Polish in 6 months than other languages I've studied for years. Jury still out on the notes app.
But of course he's not really wrong. I've been a heavy user of Anki and a heavy reader of certain schools of academic literature on second language acquisition and knew exactly what I wanted and why and how it differed from existing tools.
The lesson I take is that you need a specific problem that you truly understand, whether it's your own problem or not.
I've been trying to solve a lot of the issues brought up here, like personal automation, note taking, article summarizing/indexing, etc. What I've learned so far is that no level of AI-enabled automation will help me with my inherent ADD. All it means is that I have more started projects, but just as many finished ones. It has been a big enabler for things I'm already knowledgeable about, but the cognitive debt issue is real: A machine that thinks for me can't help me think better by itself.
"Most people are not autodidacts because most people have no material reason to learn a specific topic (i.e. their job does not require it) and the problem with learning for the sake of learning is opportunity cost: there is no a priori reason to learn one thing over another, so better to do nothing and wait for something to appear which actually grabs your interest. Again, this is likely rational! Could you imagine if you found everything interesting? You’d spend years living in a basement curating a wiki of late Soviet military hardware or something."
Great read, I feel many of the topics discussed firsthand.
It gives me some relief to know there are others out there who struggle with some similar issues, but I was hoping the piece may offer some guidance, but sadly I do not feel it has.
As an aside, the term 'human bottleneck' resonated with me because I've been trying to describe situations that look like bottlenecks but are actually just "the process everything else is working towards".
I think using 'bottleneck' to describe a process that isn't amenable to automation frames the situation incorrectly in my head. 'positive bottleneck' isn't any help.
> Could you imagine if you found everything interesting? You’d spend years living in a basement curating a wiki of late Soviet military hardware or something.
Yes... imagine...
Tell me you've got little empathy for the autist mind without telling me...
> Rather: if you don’t have the knowledge, you don’t understand the question, or why it matters, or how to judge the answers, and you won’t ever think to ask. You’re in a completely different continent from “writing the prompt”.
Reminds me of a time someone asked an influencer how to write better blog posts with LLMs. They responded "oh, it's easy", and then crafted a very specific and niche prompt about comparing and contrasting very specific things with technical details they clearly had deep knowledge in.
I completely agree with the author. As I've been saying repeatedly: AI is not magic. It's just another tool. An amazing search engine that simplifies a lot of mundane manual work to find what you're looking for. But of course, you have to have some idea of what you're looking for.
Excellent read - thanks! Always felt there was a ‘self-licking ice cream cone’ at the heart of the present moment. If only AI had better context than the messy reality that is human knowledge…oh wait, we have AI to help with that problem
Eventually AI will be good enough that whatever is inside a human brain isn't going to matter much. It's like how the most precise craftsmanship of swiss watches didn't matter because quartz watches were far more accurate. Once a new paradigm jumps the curve, the bottleneck of the old paradigm can just be circumnavigated entirely. It wouldn't make any sense to wait for technology to improve such that gear-based watched could get 10x better, the other tech is just better by nature.
AI is just better in a fundamental way vs human intelligence. It can be reproduced infinitely, has perfect logging, no mental illnesses/hangups over certain things, far faster, able to ingest more kinds of data, etc. The only limitation now is a lack of intuitive in-context learning during test time, but once that final bottleneck falls then humans will have nothing valuable comparatively.
This was a fantastic article and beautifully captured something I had been circling but hadn't quite put into words:
> The notetaking people—and I say this with all the love in the world—are never, like, a researcher at the cutting edge of their field, building this vast cathedral of knowledge, note-by-note, so they can derive new insights. Never a historian who has to read tens of millions of words across thousands of sources to synthesize the life of some historical person. It’s never someone doing something hard. It’s always some blogger. Their “digital garden” is about how to keep a digital garden. It’s very solipsistic: there’s no output, no deliverables. The deliverable is you take a screenshot of your Obsidian graph and tweet about it to show off how much it looks like an incomprehensible ball of twine.
> Sometimes, tools don’t move the needle because there’s no needle to move.
It reminds me of something my old CS mentor, now elderly, had said about LLMs a few months ago: "it's a force multiplier, but there has to be some force to multiply."
This likewise is a basic fact I encounter over and over:
> Knowledge is another limiting factor. I find that even very educated people tend to underrate the importance of knowledge. A lot of people have this attitude that you can just Google everything just-in-time as it comes up. Like Babbage, I can’t rightly apprehend the confusion of ideas that would lead someone to think this.
> It’s always some blogger. Their “digital garden” is about how to keep a digital garden. (...) never a researcher (...) never a historian
It's quite possible that those researchers and historians do exist, but are working instead of posting about notetaking. It's like saying the people who write Python tutorials never seem to ship anything impressive, so Python must be useless.
> it's a force multiplier, but there has to be some force to multiply
The assumption here is that the multiplier x is a number >1 and not a fraction <1
> Could you imagine if you found everything interesting? You’d spend years living in a basement curating a wiki of late Soviet military hardware or something.
This was a good read but this felt like a personal attack :)
This may be some saltiness on my part. I wish I could curate the wiki of late Soviet military hardware!
no saltiness detected, I laughed at it, because I have pet niche projects like that and find everything interesting, I consider it a gift at times.
Im not sure if I feel seen or accused. :)
As a product manager I have found people's understanding of what they want to be tenuous at best. They know they want it to be easier, less frustrating, and do the darn thing they want. But collecting the input of all the people with the power to make choices, boiling that all down, and actually figuring out what will work is a total bottle neck. Technology and programming have hardly ever actually been a blocker. Legacy systems, conflicting requirements, and ever shifting choices represent a much harder problem.
This is all very right, but I'd like to add this: As your capacity to deal with abstraction (which is a function of intelligence and executive function and, to some extent, knowledge) grows, you become more and more constrained by the extent to which you can manipulate symbols. AI-based solutions for that are potentially really powerful, and that's the mechanism through which, as TFA says, "intelligent, educated people with working reward circuitry stand to gain more from AI."
And I'd also add that AI strongly disaggregates the returns to different levels of the capability to deal with abstraction -- higher levels get more, lower levels get less -- rather than uniformly boosting returns across the board (unfortunately). Of course, this has been the trend of information technology since at least the '80s, but now the slice at the top is really small and the returns very high.
Heh, joke's on the author. I've built both a flashcard app and a notes app using AI, and I've learned more Polish in 6 months than other languages I've studied for years. Jury still out on the notes app.
But of course he's not really wrong. I've been a heavy user of Anki and a heavy reader of certain schools of academic literature on second language acquisition and knew exactly what I wanted and why and how it differed from existing tools.
The lesson I take is that you need a specific problem that you truly understand, whether it's your own problem or not.
W Szczebrzeszynie chrząszcz brzmi w trzcinie, a Szczebrzeszyn z tego słynie.
very impressive
I've been trying to solve a lot of the issues brought up here, like personal automation, note taking, article summarizing/indexing, etc. What I've learned so far is that no level of AI-enabled automation will help me with my inherent ADD. All it means is that I have more started projects, but just as many finished ones. It has been a big enabler for things I'm already knowledgeable about, but the cognitive debt issue is real: A machine that thinks for me can't help me think better by itself.
> no level of AI-enabled automation will help me with my inherent ADD
I have another post you might find useful :)
https://borretti.me/article/notes-on-managing-adhd
"Most people are not autodidacts because most people have no material reason to learn a specific topic (i.e. their job does not require it) and the problem with learning for the sake of learning is opportunity cost: there is no a priori reason to learn one thing over another, so better to do nothing and wait for something to appear which actually grabs your interest. Again, this is likely rational! Could you imagine if you found everything interesting? You’d spend years living in a basement curating a wiki of late Soviet military hardware or something."
uhoh
i've been spotted
We've all been there! Find something looking fun, spend a week on it then never touch again. It's normal human psychology.
Great read, I feel many of the topics discussed firsthand.
It gives me some relief to know there are others out there who struggle with some similar issues, but I was hoping the piece may offer some guidance, but sadly I do not feel it has.
As an aside, the term 'human bottleneck' resonated with me because I've been trying to describe situations that look like bottlenecks but are actually just "the process everything else is working towards".
I think using 'bottleneck' to describe a process that isn't amenable to automation frames the situation incorrectly in my head. 'positive bottleneck' isn't any help.
The missing part is usually the real workflow. If there is no output and no cost for being wrong, the AI tool just becomes another toy to maintain.
> Could you imagine if you found everything interesting? You’d spend years living in a basement curating a wiki of late Soviet military hardware or something.
Yes... imagine...
Tell me you've got little empathy for the autist mind without telling me...
> Rather: if you don’t have the knowledge, you don’t understand the question, or why it matters, or how to judge the answers, and you won’t ever think to ask. You’re in a completely different continent from “writing the prompt”.
Reminds me of a time someone asked an influencer how to write better blog posts with LLMs. They responded "oh, it's easy", and then crafted a very specific and niche prompt about comparing and contrasting very specific things with technical details they clearly had deep knowledge in.
I completely agree with the author. As I've been saying repeatedly: AI is not magic. It's just another tool. An amazing search engine that simplifies a lot of mundane manual work to find what you're looking for. But of course, you have to have some idea of what you're looking for.
Excellent read - thanks! Always felt there was a ‘self-licking ice cream cone’ at the heart of the present moment. If only AI had better context than the messy reality that is human knowledge…oh wait, we have AI to help with that problem
Eventually AI will be good enough that whatever is inside a human brain isn't going to matter much. It's like how the most precise craftsmanship of swiss watches didn't matter because quartz watches were far more accurate. Once a new paradigm jumps the curve, the bottleneck of the old paradigm can just be circumnavigated entirely. It wouldn't make any sense to wait for technology to improve such that gear-based watched could get 10x better, the other tech is just better by nature.
AI is just better in a fundamental way vs human intelligence. It can be reproduced infinitely, has perfect logging, no mental illnesses/hangups over certain things, far faster, able to ingest more kinds of data, etc. The only limitation now is a lack of intuitive in-context learning during test time, but once that final bottleneck falls then humans will have nothing valuable comparatively.
I would laugh so hard if we finally were able to build real AI (not LLMs) and discovered they had all of our own flaws.