Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Touch reality as soon as possible (when doing machine learning research), published by Lawrence Chan on January 3, 2023 on The AI Alignment Forum.

Related to: Making Beliefs Pay Rent, The Feeling of Idea Scarcity, Micro-Feedback Loops and Learning, The Three Stages of Rigor, Research as a Stochastic Decision Process, Chapter 22 of HPMOR.

TL;DR: I think new machine learning researchers often make one of two kinds of mistakes: not making enough contact with reality, and being too reluctant to form gears-level models of ML phenomena. Stereotypically, LW/AF researchers tend to make the former mistake, while academic and industry researchers tend to make the latter kind. In this post, I discuss what I mean by “touching reality” and why it’s important, speculate a bit on why people don’t do this, and then give concrete suggestions.

Epistemic status: I'm pretty frustrated with how slow I write, so this is an experiment in writing fast as opposed to carefully. That being said, this is ~the prevailing wisdom amongst many ML practitioners and academics, and similar ideas have been previously discussed in the LessWrong/Alignment Forum communities, so I'm pretty confident that it's directionally correct. I also believe (less confidently) that this is good advice for most kinds of research or maybe even for life in general.

Acknowledgments: Thanks to Adrià Garriga-Alonso for feedback on a draft of this post and Justis Mills for copyediting help.

Introduction:

Broadly speaking, I think new researchers in machine learning tend to make two kinds of mistakes:

Not making contact with reality. This is the failure mode where a new researcher reads a few papers that their friends are excited about, forms an ambitious hypothesis about how to solve a big problem in machine learning, and then spends months drafting a detailed plan. Unfortunately, after months of effort, our new researcher realizes that the components they were planning to use do not work nearly as well as expected, and as a result they’ve wasted months of effort on a project that wasn’t going to succeed.

Not being willing to make gears-level models. This is the failure mode where a new researcher decides to become agnostic to why anything happens, and believes empirical results and only empirical results even when said results don’t “make sense” on reflection. The issue here is that they tend to be stuck implementing an inefficient variant of grad student descent, only able to make small amounts of incremental progress via approximate blind search, and end up doing whatever is popular at the moment.

That’s not to say that these mistakes are mutually exclusive: embarrassingly, I think I’ve managed to fail in both ways in the past.

That being said, this post is about the first failure mode, which I think is far more common in our community than the second. (Though I might write about the second if there's enough interest!)

Here, by “touching reality”, I mean running experiments where you check that your beliefs are right, either via writing code and running empirical ML experiments, or (less commonly) grounding your ideas either in a detailed formalism (to the level where you can write proofs for new, non-trivial theorems about said ideas). I don’t think writing code or inventing a formalism qualify by themselves (though they are helpful); touching reality requires receiving actual concrete feedback on your ideas.

Why touch reality?

I think there’s four main reasons why you should do this:

Your ideas may be bad

When you’re new to a field, it’s probably the case that you don’t fully understand all of the key results and concepts in the field. As a result, it’s very likely the case that the ideas you come up with are bad. This is especially true for fields like machine learning that have significant amounts of tacit knowledge. ...