I was going to write that this comparison makes me think of largely-wrong “surprising truths!” airport non-fic books, but I see on further reading that the author is way ahead of me and this analysis includes those in its scope.
To the formulaic-ness of it, there’s the recurring ha-ha-only-serious quip from the If Books Could Kill podcasters that the more bad self-help, political hack, and trendy “science” books they read, the more sure they are that they’re all actually the same book.
I used to be a sucker for this type of books: non-fiction, "science-based", feel-good, insight-porn pamphlets inflated to 300 pages with shoehorned anecdotes and a very distinct flavour of extrapolative slop.
They are usually written by journalists, TED-addicted researchers and, more recently, Youtube and Twitter influencers.
The depressing thing is: For me, the realization that 99% of them are shit only came after reading hundreds of them. My guess is that my brain needed enough samples in order to start accurately classifying them.
It's like one develops taste only after being exposed to a lot of terrible art, since then the actual great art stands out like a beacon.
In Adler’s (How to Read a Book) terms, I think lots of folks don’t naturally learn to do his “third reading”[1] simultaneously with the first two readings (or maybe at all?), and tend to read a book only once, or to read it more than once but doing the “same” reading each time. That plus some assumptions that the social proof of a book getting recommended by, say, Bill Gates and being a bestseller means it must not just be a whole bunch of lazy crap that sometimes contradicts its own citations, can explain a lot of things slipping by a person that, viewed with hindsight, were pretty sketchy.
Certain social science or liberal arts programs are pretty good at, if nothing else, pushing students to get better at that “third reading” and developing a heuristic, automatic sense for when a piece is behaving in a maybe-dodgy way, but I think a lot of other programs aren’t, so even educated people, at least those who don’t just naturally pick the skill up young, may take a rougher road to it, if they ever pick it up. (Yes, I’m aware that my calling out some social science training as a good way to hone that skill is funny, given how much junk science, pop and otherwise, centers around the social sciences)
[1] Adler’s three readings are: 1) coming to terms—that is, seeking an understanding of the basic assumptions, definitions, context, et c, the author is operating under, and the basic thrust or topic or goal of the work; 2) Understanding the author’s argument, message, points and so forth; 3) critical reading—is the message any good? Does it hold together? Are the definitions and premises well-considered, useful, and applied consistently? Does that passage there look a bit iffy, is perhaps worded like the author’s trying to get away with something, and maybe needs a closer look? If such-and-such part crumbles on closer examination or consultation of other sources, what of the rest is salvageable, if any?
I tend to avoid stuff by journalists and prefer stuff by researchers writing about their own field for this reason. Journalists often seek narrarive.
Researchers can still be wrong, but usually less belligerently so. The academic publication process and the science it prodices, for all its flaws, generally forces researchers towards more nuanced and continent positions.
I believe that we process facts in a narrative. If you look at a bare fact, your mind automatically creates a narrative.
Murder rate 1%
22 Angstrom
Purple Burglar Alarm
That narrative takes place in a framing story.
'Science', among other things, is a very detailed framing story that modern society uses to evaluate facts.
An individual's framing story includes all of their history and biases and motivated reasoning and incentives and all the rest of the fallacies.
So yes, bad science is fiction, but even good science is narrative. If this feels wrong, go re-read a paper. Look at a figure or a sentence in the abstract. Your mind constructs a narrative.
Regarding storytelling: Thomas Basbøll and I have argued that stories are important not just for communicating science to others but also in doing good science.
My perspective here is:
1. Fictional stories are an informal sort of predictive model checking, in which the process of working out a story is a way of understanding the implications of a scenario.
2. Nonfiction stories are a sort of prior predictive check, where the unexpected twist that makes the story so satisfying represents a revelation of some aspect of reality that confounds our expectations, that is, does not fit with our implicit or explicit model of the world.
I was going to write that this comparison makes me think of largely-wrong “surprising truths!” airport non-fic books, but I see on further reading that the author is way ahead of me and this analysis includes those in its scope.
To the formulaic-ness of it, there’s the recurring ha-ha-only-serious quip from the If Books Could Kill podcasters that the more bad self-help, political hack, and trendy “science” books they read, the more sure they are that they’re all actually the same book.
100%
I used to be a sucker for this type of books: non-fiction, "science-based", feel-good, insight-porn pamphlets inflated to 300 pages with shoehorned anecdotes and a very distinct flavour of extrapolative slop.
They are usually written by journalists, TED-addicted researchers and, more recently, Youtube and Twitter influencers.
The depressing thing is: For me, the realization that 99% of them are shit only came after reading hundreds of them. My guess is that my brain needed enough samples in order to start accurately classifying them.
It's like one develops taste only after being exposed to a lot of terrible art, since then the actual great art stands out like a beacon.
In Adler’s (How to Read a Book) terms, I think lots of folks don’t naturally learn to do his “third reading”[1] simultaneously with the first two readings (or maybe at all?), and tend to read a book only once, or to read it more than once but doing the “same” reading each time. That plus some assumptions that the social proof of a book getting recommended by, say, Bill Gates and being a bestseller means it must not just be a whole bunch of lazy crap that sometimes contradicts its own citations, can explain a lot of things slipping by a person that, viewed with hindsight, were pretty sketchy.
Certain social science or liberal arts programs are pretty good at, if nothing else, pushing students to get better at that “third reading” and developing a heuristic, automatic sense for when a piece is behaving in a maybe-dodgy way, but I think a lot of other programs aren’t, so even educated people, at least those who don’t just naturally pick the skill up young, may take a rougher road to it, if they ever pick it up. (Yes, I’m aware that my calling out some social science training as a good way to hone that skill is funny, given how much junk science, pop and otherwise, centers around the social sciences)
[1] Adler’s three readings are: 1) coming to terms—that is, seeking an understanding of the basic assumptions, definitions, context, et c, the author is operating under, and the basic thrust or topic or goal of the work; 2) Understanding the author’s argument, message, points and so forth; 3) critical reading—is the message any good? Does it hold together? Are the definitions and premises well-considered, useful, and applied consistently? Does that passage there look a bit iffy, is perhaps worded like the author’s trying to get away with something, and maybe needs a closer look? If such-and-such part crumbles on closer examination or consultation of other sources, what of the rest is salvageable, if any?
I tend to avoid stuff by journalists and prefer stuff by researchers writing about their own field for this reason. Journalists often seek narrarive.
Researchers can still be wrong, but usually less belligerently so. The academic publication process and the science it prodices, for all its flaws, generally forces researchers towards more nuanced and continent positions.
Yeah, lately I've begun asking myself "How much work would you actually need to do to know this or prove this, and what would that work look like"
I believe that we process facts in a narrative. If you look at a bare fact, your mind automatically creates a narrative.
Murder rate 1%
22 Angstrom
Purple Burglar Alarm
That narrative takes place in a framing story.
'Science', among other things, is a very detailed framing story that modern society uses to evaluate facts.
An individual's framing story includes all of their history and biases and motivated reasoning and incentives and all the rest of the fallacies.
So yes, bad science is fiction, but even good science is narrative. If this feels wrong, go re-read a paper. Look at a figure or a sentence in the abstract. Your mind constructs a narrative.
Andrew addressed this in the comments:
That second point is a bit murky but this is a comment on a blog :) Here is the much more detailed paper he co-wrote with Basbøll: http://stat.columbia.edu/~gelman/research/published/storytel...