Coordinated with Fredrik

The Speed of Being Wrong


Listen Later

Let me start somewhere most people never go. Below the waterline of a warship, in the engine room, where I spent a good part of my twenties. I was a marine engineer in the navy: fast attack craft, corvettes, submarines, the machine side. Diesel engines, gas turbines, electric propulsion, battery banks.

The thing about a gas turbine is that it is not an abstraction. It has sound, heat, vibration, fuel, routines, and consequences. You are responsible for the whole system, not one component. Four-hour sleep shifts, weeks away at sea, conditions that do not negotiate with you. And if something fails out there, you cannot call support. You cannot google it. You fix it, or you have a very serious problem.

I think about that environment a lot, because of one feature it has that almost nothing else in modern life has. Out there, being wrong is fast. It is physical. There is no dashboard sitting between you and the consequence, softening it, delaying it. If your understanding of the system is wrong, the system tells you, in seconds, in heat and noise and the wrong number on a gauge. The sea does not care how confident you were.

Years later I left the sea and did something that felt completely different. I spent five years on a PhD. And the single most valuable thing I got from those five years was not the title, and it was not the result. It was discovering that the central promise of my own thesis was wrong.

I read an essay this week, by a researcher who writes as Vivek, called how to be good at research. It put words to something I had felt for a long time without saying it cleanly. There is a gap between looking like a researcher and being one, and most people, without meaning to, train the first thing. The real skill is a stack of smaller habits, almost every one of which can be deliberately trained. And when I went through them, one by one, I noticed they all bottom out in the same place: how fast you can find out you are wrong.

I did not learn that from the essay. I learned it on a ship, and then I printed it. At the top of the methods chapter of my thesis, I put a borrowed line. You should take the approach that you are wrong. Your goal is to be less wrong.

Pick your own problems

The essay opens with a habit that made the mathematician Richard Hamming unpopular at lunch. At Bell Labs he would sit with the chemists or the physicists, and after a while he would ask them what the important problems in their field were. And then, once they answered, the question that emptied the table: then why aren’t you working on them? He tells it himself, with comic timing: “I wasn’t welcomed after that. I had to find somebody else to eat with.” One chemist later admitted the question had gotten under his skin and stuck with him all summer, and went on to do important work. Hamming’s quiet, devastating coda is that he never heard the names of any of the other men at that table mentioned in science again.

The sting is that most of us have no good answer. We don’t choose our problems; we absorb them, from an advisor, from whatever a big lab announced last quarter, from the paper everyone is quote-tweeting this week. And the trouble with an absorbed problem is that you hold the conclusion without the reasoning. When the famous lab pivots, you find out a year too late, racing a thousand people who started earlier and have more compute than you.

I started my PhD with an absorbed problem. The shipping industry hands you one the moment you walk in: burn less fuel. So I spent my first years deep in thermodynamics, waste-heat recovery, exergy analysis, squeezing a few percent out of an engine that throws most of its energy away as heat. I genuinely loved the thermodynamics, the calculating on a real process and watching the energy balance out. But the problem was not mine. It was the industry’s. And it nearly made me quit.

What kept me in was a person, not a result. A mentor of mine, Magnus Genrup, talked me into staying at the exact moment I was wrong about quitting. And in that second phase I found my own problem. It started small, almost annoying: a ship is covered in sensors, producing an enormous amount of data every minute, so why do we still not know how much fuel each part of it is burning? Could the ship just tell you, from the data it already produces, without bolting a fuel meter onto every engine? A virtual fuel meter.

That question was mine. The researcher John Schulman splits the work into two modes: in one you read the literature and hunt for things to improve; in the other you choose an outcome you genuinely want to exist and reason backwards to the experiments. The second mode manufactures originality almost as a side effect, because a goal you actually care about drags you into territory no survey paper covers. The fuel question dragged me out of marine engineering and into machine learning, and I never really came back. I did not switch fields. The problem moved, and I followed it.

Upgrade your inputs

Shared reading lists produce shared ideas. If your information diet is the trending page of arxiv plus whatever survives the group chat, you’ll reliably reach the same conclusions as everyone else, at the same time, which makes those conclusions worth approximately nothing.

So where is the edge? One place is almost embarrassingly underpriced: old material. This field reruns its own past on a delay. There’s a thousand-word essay by Rich Sutton from 2019, the bitter lesson, that predicts the shape of the whole field better than surveys ten times its length. It’s called bitter because it’s so uncomfortable: looking back at seventy years of AI, the same pattern repeats. Researchers lovingly build their human knowledge into a system by hand; it helps for a while; and then it’s beaten, decisively, by a simpler method that just throws computation at search and learning. When Deep Blue beat Kasparov in 1997, it didn’t understand chess like a grandmaster; it searched staggeringly fast. Sutton notes, with an edge, that the chess researchers who’d encoded human knowledge for years were “not good losers.” The same story repeated in Go, in speech, in vision. The bitter part is that a lot of your hard-won specialist cleverness is a dead end, and you only learn it by reading the quiet old essay, not the loud new thread.

Which is the second habit: read the paper itself, not the thread summarizing it. The appendix is where the bodies are buried, and the limitations section is usually the most honest paragraph in the document. The interesting part of anyone’s work is almost never the headline. It’s the quiet place where they tell you what they couldn’t make work.

There’s an even older voice in the essay, and it’s the one I’d tattoo on a wall. Claude Shannon, the father of information theory, gave an informal talk to his Bell Labs colleagues in 1952 on how creative people actually crack hard problems. His first and most powerful move was radical simplification: cut it down to size. Strip everything from the problem except the essentials, until it barely resembles what you started with. And then, he says, very often if you can solve that simple problem you can add refinements, one at a time, until you get back to the one you started with. Shrink it, solve the toy, grow the difficulty back. That single trick has carried me through more walls than any productivity advice ever written.

But the place the edge hides that I know in my hands is different. Sometimes upgrading your inputs doesn’t mean finding new data at all. It means reading the data everyone already had and nobody bothered to look at. On the cruise ship I studied, almost a fifth of the electricity on board could have been replaced by heat that was already there, pouring out of the exhaust, doing nothing. In Swedish we have an expression for wasting heat like that: elda för kråkorna, heating the crows. The whole ship was heating the crows, and the signal that proved it had been sitting in the logging system the entire time. Nobody was wrong about it. Nobody had looked.

I notice that kind of thing because of how I’m wired, and it goes back a long way. When my parents got me a 286 computer, something locked into place out of pure curiosity. I built my own machines instead of buying finished ones. I ran a bulletin board system. I was online in the very early nineties, before most people knew the networked world existed. I wrote little programs, optimized DOS to claw back a few kilobytes, ran Linux early, when running Linux meant understanding the machine all the way down. None of it was a plan. It was just the joy of taking a system apart in my head and putting it back together better.

And I get to watch it happen again, one generation later. My fifteen-year-old son has exactly the same fire, except his machine isn’t a 286. He builds his own AI agents. He trains his own language models. He runs Arch Linux, because the harder, barer version of the system is more interesting to him than the easy one. Same instinct. Same curiosity. Same refusal to use the polished surface and look away. The habit underneath all of this isn’t technical at all. It’s wanting, badly, to know how the thing actually works.

Tighten the loop, and stare at the outputs

If you remember one idea from this, make it this one. Research speed is mostly the speed at which you discover you are wrong.

The stories are about volume, not genius. Alec Radford, behind some of the most important work in modern machine learning, joined a leading lab in his early twenties straight out of undergrad, no PhD. One of his most famous early results is a perfect little parable. He didn’t design a clever system to detect emotion in text. He trained a model to do something almost mindless: predict the next character, one at a time, across more than 82 million Amazon reviews. A month of training on four graphics cards, just guessing the next letter. And when he looked inside, a single neuron had, on its own, learned to track whether a review was positive or negative. Nobody built that. It emerged out of sheer scale and iteration.

That’s the game. Not being right on the first try, but being wrong quickly, over and over, until what’s left standing is true. Which is why the essay treats tooling as a first-class research activity. Launching an experiment should be one command; plotting it should be one more. If comparing two runs takes an afternoon of digital archaeology, you’ll run fewer of them and discover you’re wrong more slowly. And there’s a daily exercise I’ve come to love: before you run an experiment, predict the result. Write the number down. Cover a paper’s results section and guess it from the method alone. Taste isn’t a gift you’re born with. It’s a forecast plus a correction, repeated a few hundred times, until your instinct gets trained the same way a model does.

Here’s what tightening the loop actually looked like for me. To get my data I went aboard the ship, the M/S Birka Stockholm, four trips across 2014 and 2015, pulling the data out of the engine system onto USB sticks. A year of operation, thousands of signals, logged every minute. And then the romance ended and the research began. Having the data was almost nothing. There’s a wonderful recipe by Andrej Karpathy whose very first step, before you write any model code, is to “become one with the data”: spend time, measured in hours, scrolling through thousands of raw examples by hand, looking for the corrupted ones, the duplicates, the quiet biases. I didn’t have a choice about that. The data forced it on me, because the hard part, the part that took months, was figuring out what each signal actually was. The exact physical location of thousands of sensors wasn’t documented. So I sat with the machinery schematics and the electrical drawings, and I talked to the engine crew, and slowly mapped each number back to a real pipe, a real engine, a real place on the ship. The bottleneck was never the algorithm. It was system understanding. And the thing that let me do it was that I’d stood in engine rooms like that one; the crew trusted me, because I knew what they were talking about.

Karpathy has a second step that sounds silly and is one of the best debugging ideas I know: before scaling anything, deliberately overfit a single tiny batch, as few as two examples, and confirm the model can drive the error to zero on just those two. It proves the plumbing works before you waste a month discovering it doesn’t. And then the most underrated habit in research, taught by Andrew Ng for over a decade: when your system makes mistakes, don’t theorize, pull about a hundred of the examples it got wrong and read them by hand. Sort them into piles. In his cat-detector example you discover some failures are dogs, some are blurry, and a surprising number are wrecked by Instagram filters, so you add a column for Instagram mid-review and attack the biggest pile first. An hour or two of reading failures can save you a month chasing the wrong fix. A descending loss curve is not analysis. It’s reassurance, and the two are terribly easy to confuse.

When the system understanding was finally right, the payoff was almost embarrassing. The model could predict a full year of fuel consumption, from signals the ship was already producing, with an accuracy of over ninety-eight percent. The ship had been telling us the answer the whole time. The work was learning how to listen. I have a line I keep coming back to: a dashboard can see something I can’t see directly, and that makes it valuable, but it can also make me forget how much it cannot see. The dashboard does not know the smell in the engine room. It gives you a trace of the system. It is not the system.

Write everything down

Now something that doesn’t flatter me. The essay’s next habit starts with Paul Graham, who has the cleanest statement of it I’ve read. We think of writing as a way to record a thought we already have; Graham says it’s really a test of whether you had the thought at all. “Putting ideas into words is a severe test.” You feel like you understand something, then you sit down to write it and the sentence refuses to finish itself, because your understanding was vaguer than it felt. And his conclusion is the unsettling part: if you never subject your ideas to that test, you’ll not only never have fully formed ideas, you’ll never even realize it.

The essay then reaches for two scientists who turned this into a discipline. Feynman, in his 1974 cargo-cult-science talk, described Pacific islanders who, after the war, built runways out of bamboo and lit fires and sat a man in a hut with wooden discs for headphones, mimicking every visible form of an airbase, and no planes ever came. A lot of work looks like science and is missing the one invisible thing that makes it real: ruthless honesty with yourself. “The first principle is that you must not fool yourself, and you are the easiest person to fool.” And Darwin, who built an actual machine against self-deception. Late in life he described a golden rule: whenever he met a fact or a thought that went against his theories, he forced himself to write it down “without fail and at once,” because he’d found by experience that such facts escaped the memory far more easily than the favourable ones. He’d noticed his own mind quietly deleting the evidence that threatened his life’s work, and built a habit to catch the deletions.

Here’s my confession. I’ve written by hand for years. I keep journals, every morning and lately every evening. I love a fountain pen gliding across paper, the way the world disappears and I can concentrate like a Roman. Writing is how I catch myself thinking. But if I’m honest about why I write, I write to consolidate what I already believe, to make my own thoughts more solid. I do not write, the way Darwin wrote, to hunt down the evidence that would break me. I had this pointed out to me recently, going back through years of my own pages, and the pattern was uncomfortable: I was much better at writing conviction than at writing what would falsify the conviction. I was building cathedrals to my own ideas, not stress-testing them.

That’s a serious problem for someone who fell in love with Karl Popper as a doctoral student, because the whole point of Popper is that your job is never to prove your hypothesis but to try to falsify it. If I only use writing to strengthen the thesis, writing becomes propaganda. If I use it to ask what would break the thesis, writing becomes judgment. Same pen, opposite direction. Though there’s a failure mode on the other side too: if you’re too eager to call every bad result a falsification, you quit before reality has had time to answer. The skill is holding a belief firmly enough to test it properly and loosely enough to drop it when the test comes back. The page is where I keep score on that, when I’m brave enough to. I can go back to something I wrote three years ago and see, in my own handwriting, that I was far more certain than the evidence allowed.

Wander, find your people, and play the long game

Your first subfield is an accident of timing, so treat it like one. I was, for years, an odd bird at a maritime academy: a former naval officer doing machine learning in a research environment that had basically just been founded. There was no map, and that was the gift, because when there’s no map the problems are still yours to pick. The essay pairs that wandering with a ruthlessness I learned the hard way: run the disposable version of every idea first and let most of them die young; tune your baselines until it hurts, because the graveyard of this field is full of gains that evaporated against a properly tuned baseline; and ablate, until you know which single component is actually carrying the result, because it’s usually one component and usually not the one in the title.

There’s a feeling that comes with all this wandering, and the essay is honest enough to name it. I once wrote a newspaper column titled Am I a fraud? I was writing my licentiate thesis on the sofa, kids climbing over me, headphones on, and underneath it a gnawing sense that I didn’t really master my own work. For a long time I thought that feeling was a problem to solve. Now I think it’s just what standing at the edge of what you know feels like. If you never feel it, you’re probably working too far inside the map.

Then the part about people. Hamming noticed that colleagues who kept their office doors closed got more done in any given day, but ten years later didn’t quite know which problems were worth working on. The ones who kept their doors open got constant interruptions and less done today, but those interruptions carried information about what the world actually needed, and they did the work that mattered. Your open door, the essay says, is probably your inbox. I think about it in terms of a submarine, where everyone has a specialty but the vessel can’t survive on each person knowing only their private fragment. Everyone needs enough of the whole to catch when something outside their specialty is becoming dangerous. Down there, shared context isn’t a nice cultural feature. It’s survival infrastructure. It’s the same in research.

So I try to keep the door open in two directions. One is generosity outward. When I finished my thesis I gave it away to anyone who asked, a couple of hundred people, because doing the work in the open meant the whole industry could use it. I once stood on a stage and explained my entire research to a room of non-experts in four minutes, using nothing but that homely idiom, heating the crows, and won the competition. Two researchers, Chris Olah and Shan Carter, wrote that fields accumulate research debt, “the accumulation of missing interpretive labor,” where every newcomer has to re-climb the same undocumented mountain. Explaining something hard in plain language isn’t a chore beneath real research. It pays down that debt for everyone behind you. The other direction is letting people push back on me: having people around whose actual job is to tell me an idea is bad before I sink three months into it. That person is worth more than compute, and the relationship can’t be bought, only earned.

Which brings me back to my own thesis, and the promise that turned out to be wrong. I’d built my whole research life on a simple moral idea: make things more efficient, use less, and you help the world. And somewhere in those five years I ran into the uncomfortable truth that efficiency, on its own, doesn’t behave the way that idea promised. Making a ship a little more efficient mattered far less than I’d believed. Not morally wrong. Mechanically wrong. (I went at that properly back in Episode 30, The Efficiency Trap, so I won’t relitigate it here.) The central promise of my doctorate didn’t survive contact with reality, and I’m glad it didn’t. Because that’s what a PhD actually is underneath the formality: a long, structured exercise in adapting, in being wrong slowly enough to learn from it. I sometimes say I’ve failed my way forward my entire life, and I mean it as a compliment to the method. The person who’s afraid to be wrong doesn’t dare to do anything at all.

The cheap part

So, back to the engine room, where being wrong is fast and physical and there’s no dashboard to hide behind. Every habit in that essay — picking your own problems, upgrading your inputs, tightening the loop, staring at the real outputs, writing the thing that could break you, keeping the door open — reduces to one discipline. How fast you let reality tell you that you’re wrong, and whether you thank it when it does.

There’s a line everyone half-remembers, from Louis Pasteur, and the popular version cuts off the most important words. People say chance favors the prepared mind. What Pasteur actually said, to young people heading into factories in 1854, was: in the fields of observation, chance favors only the prepared mind. The compass needle twitches near a wire for everyone. Only the prepared mind notices that the world just changed.

The daily edges look trivial in isolation. What you read. What you write down. How quickly your loop runs. Who you let argue with you. But they compound, like interest, and given a few years they produce something that looks, from the outside, like luck. So start compounding earlier than feels necessary. The future version of you already knows this was the cheap part.

I still keep that borrowed line at the top of my methods chapter, and I think it’s the truest thing I ever wrote down, even though I didn’t write it first. You should take the approach that you are wrong. Your goal is to be less wrong. And the whole game is just how fast you’re willing to get there.

Be less wrong. Faster.



This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit frahlg.substack.com
...more
View all episodesView all episodes
Download on the App Store

Coordinated with FredrikBy Fredrik Ahlgren