Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Call for Papers: The Economics of Animal Welfare, Stanford Institute for Theoretical Economics 2022, published by zdgroff on April 4, 2022 on The Effective Altruism Forum.

Stanford Economics is proud to host its annual Stanford Institute for Theoretical Economics (SITE) Conference this summer 2022. Despite its name, SITE sponsors sessions that encompass both economic theory and empirical work and cover a broad range of topics. It brings together established and emerging scholars to present leading-edge economic research, to educate, and to collaborate.

On July 12 2022, SITE will be hosting its first-ever session on The Economics of Animal Welfare. The session will bring together theoretical work on animal wellbeing and empirical work on supply and demand forces on animal welfare. For instance, we welcome submissions on how to bring animal wellbeing into welfare economics and on the effects of policies intended to improve humane treatment of animals.

We encourage submissions of abstracts and papers taking a range of approaches to the topic.

The meeting will be hybrid (in-person and via Zoom). In-person attendees will meet in a beautiful conference room at the Stanford Institute for Economic Policy Research (SIEPR) and enjoy travel support as well as meals during the day and group dinners.

Please submit by April 30, 2022 at this link:. Notifications will be emailed approximately 2 weeks after the submission deadline.

The conference website is:

ORGANIZED BY:

Marc Fleurbaey, Paris School of Economics

Lisa Kramer, University of Toronto

Nicolas Treich, Toulouse School of Economics

Caroline Hoxby, Stanford University

with the assistance of Zach Freitas-Groff, Stanford University

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: The EA Training Board is now live, published by andreferretti on April 5, 2022 on The Effective Altruism Forum.

We're excited to announce Training For Good's new EA Training Board.

Today, 80,000 Hours and Animal Advocacy Careers have great EA job boards, and the EA Internship Board covers internships. But at Training For Good, our board focuses on—you guessed it—training.

What you can find

Reading groups (e.g., The Precipice Reading Group)

Grants (e.g., FTX Future Fund's call for proposals)

Contests (e.g., FLI's Worldbuilding Contest)

Events (e.g., EAGx Prague)

Volunteering (e.g., local EA Group)

Fellowships (e.g., Atlas Fellowship)

Coaching (e.g., 80,000 Hours 1:1 advice)

Courses (e.g., Good Food Institute's Alternative Proteins Course)

Online tools (e.g., Open Philanthropy's Calibrate Your Judgement)

What you can do

Share the EA Training Board with a friend.

Suggest a new training programme with this form.

Subscribe to our newsletter for monthly updates on new training programmes.

Give us feedback in the comments, or email us.

After checking our board, which programme will you apply to?

What other programmes would you like to see on the board?

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Case for emergency response teams, published by Jan Kulveit on April 5, 2022 on The Effective Altruism Forum.

This project is seeking a paid coordinator. Please apply here.

Motivation

See Hinges and crises for expanded explanation.

The hinge of history

So far, long-termist efforts to change the trajectory of the world focus on far-off events. This is on the assumption that we foresee some important problem and influence its outcome by working on the problem for longer. We thus start working on it sooner than others, we lay the groundwork for future research, we raise awareness, and so on.

Many longtermists propose that we now live at the “hinge of history”, usually understood on the timescale of critical centuries, or critical decades. But ”hinginess” is likely not constant: some short periods will be significantly more eventful than others. It is also possible that these periods will present even more leveraged opportunities for changing the world’s trajectory.

These “maximally hingey” moments might be best influenced by sustained efforts long before them (as described above). But it seems plausible that in many cases, the best realistic chance to influence them is “while they are happening”, via a concentrated effort at that moment. Some reasons for this: there are decreased problems with cluelessness; an increase in the resources spent on the problem; actual decisions being made by powerful actors, and so on.

Crises as times of opportunity

Even if a specific crisis is not a “hinge of history”, crises often bring opportunities to change the established order. For example, policies well outside the Overton window can suddenly become live options, and intellectual development in disciplines and technologies (think of the sudden intense focus on reproducibility in epidemiology). These effects often persist for decades after the initial event (think of taking off your shoes at the airport; think of face masks in post-SARS countries) - and so shaping the response is of interest to longtermists.

How to impact hinges or crises

Acting effectively during hinges or crises may depend on factors such as "do we have a relevant policy proposal in the drawer?", "do we have a team of experts able to advise?" or “do we have a relevant network?” It is possible to prepare these!

This document proposes the creation of an “emergency response team” as one sensible preparation.

We tested the above principles during the COVID pandemic, by launching a good-sized research and policy effort, Epidemic Forecasting (EpiFor). We had some success, with associated members advising legislators and international bodies and the associated research getting into top journals and so reaching millions of people.

Some takeaways:

There are various paths to impact. COVID illustrated an opportunity for small teams: there are worlds where what's needed is being able to think clearly and do research really fast.

Often our main bottleneck was project managers, particularly people with both research skills and PM skills. Also EpiFor was only effective because core members already knew each other from FHI RSP or CZEA. And we could have been much more effective, if the team had "trained" together before COVID, to sort out differences in management styles, communication, commitment, etc.

How to improve longtermists’ emergency response capabilities

Is it possible to fund, train, and sustain a longtermist response team in the absence of a current emergency? Perhaps - but for people we most want on board, the opportunity costs of being thus “benched” might be too high.

A more viable alternative is a reserve, or standing army: a team of researchers and managers who are “on call” for a future emergency, undergoing annual wargaming or similar refreshers to maintain readiness.

What the team ideally should have:

existing e...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: What an actually pessimistic containment strategy looks like, published by lc on April 5, 2022 on LessWrong.

Israel as a nation state has an ongoing problem. For the last twenty years or so, Iran has been covertly developing nuclear weapons. Iran is a country with a very low opinion of Israel and is generally diplomatically opposed to its existence. Their supreme leader has a habit of saying things like Israel is a "cancerous tumor of a state" that "should be removed from the region". Because of these and other reasons, Israel has assessed, however accurately, that if Iran successfully develops nuclear weapons, it stands a not-insignificant chance of using them against Israel.

Israel's response to this problem has been multi-pronged. Making defense systems that could potentially defeat Iranian nuclear weapons is an important component of their strategy. The country has developed a sophisticated array of missile interception systems like the Iron Dome. Some people even suggest that these systems would be effective against much of the incoming rain of hellfire from an Iranian nuclear state.

But Israel's current evaluation of the "nuclear defense problem" is pretty pessimistic. Defense isn't all it has done. Given the size of Israel as a landmass, it would be safe to say that it's probably not the most important component of Israel's strategy. It has also tried to delay, or pressure Iran into delaying, its nuclear efforts through other means. For example, it gets its allies to sanction Iran, sabotages its facilities, and tries to convince its nuclear researchers to defect.

In my model, an argument like "well, what's the point of all this effort, Iran is going to develop nuclear weapons eventually anyways" would not be very satisfying to Israeli military strategists. Firstly, that the Iranians will "eventually" get nuclear weapons is not guaranteed. Secondly, conditional on them doing it, it's not guaranteed it'll happen the expected lifetime of the people currently living in Israel, which is a personal win for the people in charge. Thirdly, even if it's going to happen tomorrow, every day that Iran does not possess nuclear weapons under this paradigm is a gift. Delaying a hypothetical nuclear holocaust means increasing the life expectancy of every living Israeli.

An argument like "well, what if you actually radicalize the Iranians into hardening their stance on developing nuclear weapons through all of this discouragement" would also be unsatisfying. Disincentivizing, dissuading, and sabotaging people's progress toward things generally does what it says on the tin, and Iran is already doing nuclear weapons development. Any "intervention" you can come up with towards Iranian nuclear researchers is probably liable to make things better and not worse.

There are many people on this website that believe the development of AGI, by anyone in the world, would be much worse in expectation than Iran developing nuclear weapons, even from the perspective of a fiercely anti-Iranian nationalist. There are also some people on this website who additionally believe there is little to no hope for existing AI safety efforts to result in success. Since so far it doesn't seem like there are any good reasons to believe that it's harder and more genius-intense to develop nuclear weapons than it is to develop AGI, one might naively assume that these people would be open to a strategy like "get existing top AGI researchers to stop". After all, that method has had some degree of success with regard to nuclear nonproliferation, and every hour that the catastrophic AGI extinction event doesn't happen is an hour that billions of people get to continue to live.

One would think that this opens up the possibility, and even suggests the strategy, of finding a way to reach and convince the people actual...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Greyed Out Options, published by ozymandias on April 4, 2022 on LessWrong.

Imagine that life is a choose-your-own adventure game.

In any moment, you have literally millions of options. At the moment I’m typing this, I could change the tab to innumerable websites, I could read any of the hundreds of books in my house, I could make myself a snack of olives, I could stand up and see how far I could jump, I could pet a cat, I could walk into my best friend’s bedroom and call them an idiot, and so on and so forth.

But, most of the time, we only think of a menu of a few dozen options—sometimes much fewer. The rest are sort of grayed out.

To a large extent, this is a good thing. Most of the options theoretically available at any given moment are very stupid. (Just ask anyone with intrusive thoughts—yes, brain, I understand I could put the lightbulb in my mouth, stop bringing it up!) But I think it’s important to think about the ways that grayed out options limit our behavior.

You can go outside in pajamas. It isn’t illegal. No one will stop you. Most of the time, no one will even comment. Sure, you might run into someone you know, but in many cities that’s not going to happen, and anyway they’re likely to assume you have a stomach flu or otherwise have some perfectly good reason for running around in pajamas. You’re unlikely to face any negative consequences whatsoever.

But when I’ve suggested this to people, they tend to object not because they have no particular reason to go places in pajamas (pajamas are very comfortable) but because people don’t do that. It’s just not on the list of available options. If you did, you’d probably feel anxious and maybe even ashamed, because it’s genuinely hard to do something that people don’t do.

To be clear, I’m not suggesting that you should go places wearing pajamas! I don’t. I’m suggesting that you consider thoughtfully which of your options are grayed out and why.

Here are some other grayed-out options I’ve observed among people I’ve met:

Starting a conversation with a stranger.

Asking someone out.

Eating at a restaurant alone.

Walking alone at night (especially if you’re female or were raised female).

Writing a novel or a blog post.

Drawing a picture.

Submitting your writing to a publisher.

Emailing a professor to ask a question about their discipline.

Making a pull request on Github.

Editing Wikipedia.

Writing a computer program to fix a problem you have or automate a piece of work you have to do often.

Starting a recurring event like a bookclub or a meetup.

Throwing a party.

Complaining to customer service.

Opening up a broken machine and poking around in there to see if something obvious is wrong.

Googling your problem.

Negotiating your salary.

Researching a topic of interest on Google Scholar.

Doing parkour on walls etc that you find on your walk.

Planting potato eyes etc and getting food from them.

Talking to a famous person.

Singing in public.

Transitioning.

Dating people of the same gender.

Talking openly with your partner about your relationship needs.

Traveling the world and crashing on various friends’ couches instead of having a house.

Cutting off your family.

Being out about something stigmatized (your disability, your sexual orientation, your religion, your hobbies.).

Asking for your unusual preferences to be accommodated (by service workers or by people you know).

Different people have different grayed-out options, and I think this is actually a really common reason that people behave differently from each other. The reason that I write blog posts and other people don’t is not that I’m good at writing and they’re not; it’s that writing a blog post about something I’m thinking about is on my action menu and it’s not on theirs.

Some grayed-out options just don’t occur to you (like eating a lightbulb, if you are more ble...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Project Intro: Selection Theorems for Modularity, published by TheMcDouglas on April 4, 2022 on LessWrong.

Introduction - what is modularity, and why should we care?

It’s a well-established meme that evolution is a blind idiotic process, that has often resulted in design choices that no sane systems designer would endorse. However, if you are studying simulated evolution, one thing that jumps out at you immediately is that biological systems are highly modular, whereas neural networks produced by genetic algorithms are not. As a result, the outputs of evolution often look more like something that a human might design than do the learned weights of those neural networks.

Humans have distinct organs, like hearts and livers, instead of a single heartliver. They have distinct, modular sections of their brains that seem to do different things. They consist of parts, and the elementary neurons, cells and other building blocks that make up each part interact and interconnect more amongst each other than with the other parts.

Neural networks evolved with genetic algorithms, in contrast, are pretty much uniformly interconnected messes. A big blob that sort of does everything that needs doing all at once.

Again in contrast, networks in the modern deep learning paradigm apparently do exhibit some modular structure.

Top: Yeast Transcriptional Regulatory Modules - clearly modular Bottom: Circuit diagram evolved with genetic algorithms - non-modular mess

Why should we care about this? Well, one reason is that modularity and interpretability seem like they might be very closely connected. Humans seem to mentally subpartition cognitive tasks into abstractions, which work together to form the whole in what seems like a modular way. Suppose you wanted to figure out how a neural network was learning some particular task, like classifying an image as either a cat or a dog. If you were explaining to a human how to do this, you might speak in terms of discrete high-level concepts, such as face shape, whiskers, or mouth.

How and when does that come about, exactly? It clearly doesn’t always, since our early networks built by genetic algorithms work just fine, despite being an uninterpretable non-modular mess. And when networks are modular, do the modules correspond to human understandable abstractions and subtasks?

Ultimately, if we want to understand and control the goals of an agent, we need to answer questions like “what does it actually mean to explicitly represent a goal”, “what is the type structure of a goal”, or “how are goals connected to world models, and world models to abstractions?”

It sure feels like we humans somewhat disentangle our goals from our world models and strategies when we perform abstract reasoning, even as they point to latent variables of these world models. Does that mean that goals inside agents are often submodules? If not, could we understand what properties real evolution, and some current ML training exhibit that select for modularity and use those to make agents evolve their goals as submodules, making them easier to modify and control?

These questions are what we want to focus on in this project, started as part of the 2022 AI Safety Camp.

The current dominant theory in the biological literature for what the genetic algorithms are missing is called modularly varying goals (MVG). The idea is that modular changes in the environment apply selection pressure for modular systems. For example, features of the environment like temperature, topology etc might not be correlated across different environments (or might change independently within the same environment), and so modular parts such as thermoregulatory systems and motion systems might form in correspondence with these features. This is outlined in a paper by Kashtan and Alon from 2005, where they aim to...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Not-Useless Advice For Dealing With Things You Don't Want to Do, published by Rafael Harth on April 4, 2022 on LessWrong.

Sometimes, you want to do something hard, like working on a project, but you can't get yourself to do it in the moment. I think most advice one reads about this problem (at least outside of LW) is useless, hence the title. The aspiration of this post will be to say at least one thing you haven't thought of before.

[Epistemic status: self-experimentation & modelling; no empirical evidence.]

Here is our basic model:

There's [you], and there's [something you want to achieve], but the process of achieving it is unpleasant, so we've drawn a metaphorical "Unpleasantness Barrier".

This image immediately suggests a bunch of ways how to proceed. Let's start with one of the less obvious ones.

1. Avoid the Barrier

1.1. Just don't do it

I am continuously amazed at how often people tell me they will do something only to then not do it. If this is the option you will end up going with, it is almost strictly better to decide this early rather than resolving to do it and failing. Perhaps you couldn't have known how hard it would be at first, but in most cases, who are we kidding? As trivial as this sounds, I think it is one of the most underutilized strategies on this list. Quite often, you can, in fact, just not do the thing.

Related: Should considered harmful; Taboo Should

1.2. Get Around the Barrier

Don't like calling people? Text them instead. Don't like cleaning your house? Hire someone. Don't like your job? Look for another one.

This strategy is arguably even more underutilized, though probably less so among this audience. It's not always applicable, however, which is true for most items in this list. E.g., perhaps what you want to do is study something, in which case getting someone to do it for you won't help at all.

2. Overcome the barrier using BRUTE STRENGTH

You have some tools to do hard things. Let's look at them in turn.

2.1. Motivation

Sometimes you are motivated and can do things that would usually be hard.

This is what most advice focuses on. It's also probably the least useful item on this list. It's not that motivation isn't helpful when you have it, but it's infamously uncontrollable, and telling people a variant of "just be motivated" probably is unhelpful. I mean, just look at that gun; does this really look like something that's going to work when you stop believing in it?

Generally, no amount of intellectual understanding will get you motivated on demand.

That said, I do think motivation deserves a special mention for creative tasks like writing a novel. Not because motivation is reliable in those cases (I wish!) but because it's far more necessary. If I'm not motivated to write fiction, it generally doesn't help to force myself into doing it anyway because even if I do, what I produce won't be any good.

2.2. Willpower

It is not clear how willpower works or what it is, but it is a fact that people sometimes do hard things that they're not motivated for, so something else is at work here.

Willpower is far more reliable than motivation, but it seems to have a limited supply. It's unclear how to cultivate it (though there's been a decent amount of discussion on LW), but repeated success is probably helpful, so if nothing else, I recommend not resolving to do things with a high probability of failure-to-try.

In general, I think "don't ever expect motivation to work, just rely on willpower for things you really must get done" is good advice.

2.3. Penalties

Tim Urban from WaitButWhy talks about this problem using the metaphor of the rational decision-maker (stick figure), the instant gratification monkey, and the panic monster. The general situation is this:

But when the panic monster shows up, we get this instead:

Tim Urban argues that this hap...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Google's new 540 billion parameter language model, published by Matthew Barnett on April 4, 2022 on LessWrong.

Google just announced a very large language model that achieves SOTA across a very large set of tasks, mere days after DeepMind announced Chinchilla, and their discovery that data-scaling might be more valuable than we thought.

Here's the blog post, and here's the paper. I'll repeat the abstract here, with a highlight in bold,

Large language models have been shown to achieve remarkable performance across a variety of natural language tasks using few-shot learning, which drastically reduces the number of task-specific training examples needed to adapt the model to a particular application. To further our understanding of the impact of scale on few-shot learning, we trained a 540-billion parameter, densely activated, Transformer language model, which we call Pathways Language Model (PaLM).

We trained PaLM on 6144 TPU v4 chips using Pathways, a new ML system which enables highly efficient training across multiple TPU Pods. We demonstrate continued benefits of scaling by achieving state-of-the-art few-shot learning results on hundreds of language understanding and generation benchmarks. On a number of these tasks, PaLM 540B achieves breakthrough performance, outperforming the finetuned stateof-the-art on a suite of multi-step reasoning tasks, and outperforming average human performance on the recently released BIG-bench benchmark. A significant number of BIG-bench tasks showed discontinuous improvements from model scale, meaning that performance steeply increased as we scaled to our largest model. PaLM also has strong capabilities in multilingual tasks and source code generation, which we demonstrate on a wide array of benchmarks.

We additionally provide a comprehensive analysis on bias and toxicity, and study the extent of training data memorization with respect to model scale. Finally, we discuss the ethical considerations related to large language models and discuss potential mitigation strategies.

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Ukraine Post #8: Risk of Nuclear War, published by Zvi on April 4, 2022 on LessWrong.

It seems worth going through the exercise of estimating the probability of nuclear war, and in particular the probability of it causing one’s death. If the probability gets high enough, one can strongly consider being elsewhere or otherwise doing something about it.

Note that all scoring rules and wagers are essentially useless here. You can look back and decide whether your reasoning was good, but saying ‘I was right’ is meaningless.

As a baseline to work from, this EA forum post presents multiple perspectives on nuclear war risk in terms of the danger of being in London, with the author of the post modeling risk as relatively high, versus some superforecaster predictions that modeled risk as relatively low. The forecasts are divided into steps.

Will there be a conventional exchange between NATO and Russia?

Will there be a nuclear exchange?

If there is a nuclear exchange will it hit London?

If it does will you have to get out?

If you don’t get out, will you die?

(Alternate path) Background risk of accidental nuclear war, which is higher when everyone is on alert.

If you multiply the odds of each step together, you get the level of danger from being in London. The level of danger in New York City should be similar.

Aside from the chances of the bomb killing you if it lands, these differences all point in the same direction, with two of them being on different sides of 50%. It is an interesting exercise to read the arguments, and to decide on one’s own opinion on each leg.

Probability of conventional war

There are two ways to get a conventional war. Russia could attack NATO and cause NATO to invoke Article 5 without an intervention in Ukraine, or NATO could decide to intervene in Ukraine.

We have a Metaculus market on whether a NATO country will invoke Article 5 by the end of the year and it is sitting at 5%. That seems reasonable.

I think there is a modest chance that Article 5 is technically invoked but there is nominal fighting, and there is a similar chance that NATO does something crazy and intervenes first for whatever reason. 5% here seems fine.

Dynamics of nuclear deterrence have changed

In the Cold War, both sides felt NATO had woefully inadequate conventional forces, and would need to resort to either lose or use nuclear weapons within weeks if the Warsaw Pact invaded Western Europe.

This has now reversed. Russia’s conventional forces have been exposed. Both sides now believe (correctly) that NATO has vastly superior conventional forces, and could easily repel a conventional Russian attack on NATO. At worst, Russia could make temporary gains in the Baltics.

Thus, I put the chances of NATO dropping the first nuclear weapon at epsilon. NATO has no reason to open Pandora’s box when it can win a conventional war.

I wouldn’t quite put the chance of Putin not knowing that NATO will never drop the first nuclear weapon at epsilon, but I remain very highly confident he knows this. Russia would have to use nuclear weapons first, knowing NATO would never use one first.

I would also put the chances of conventional NATO military invasion of Belarus or Russia, beyond at most necessary incidental incursions during a war over the Baltics or Poland, to be essentially epsilon, for the same reason. Russia is credibly threatening to respond to such attacks with nuclear weapons, even if ‘they started it’ and we have no interest in putting that to the test. Even in a hot conventional NATO/Russia war caused by a Russian invasion, I would expect us not to put troops even into Ukraine because we would not have to.

Putin thinks the West is weak. In some senses he is wrong, but in an important sense he is very right. One does not need to threaten the West with the mass destruction of its cities to ma...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Call For Distillers, published by johnswentworth on April 4, 2022 on The AI Alignment Forum.

Many technical alignment researchers are bad-to-mediocre at writing up their ideas and results in a form intelligible to other people. And even for those who are reasonably good at it, writing up a good intuitive explanation still takes a lot of work, and that work lengthens the turn-time on publishing new results. For instance, a couple months ago I wrote a post which formalized the idea of abstractions as redundant information, and argued that it’s equivalent to abstractions as information relevant at a distance. That post came out about two months after I had the rough math worked out, because it took a lot of work to explain it decently - and I don’t even think the end result was all that good an explanation! And I still don’t have a post which explains well why that result is interesting.

I think there’s a lot of potential space in the field for people who are good at figuring out what other researchers’ math is saying intuitively, and why it’s interesting, and then communicating that clearly - i.e. the skill of distillation. This post will briefly sketch out what two kinds of distillation roles might look like, what skills are needed, and talk about how one might get started in such a role.

Two Distiller Roles

The two types of distiller role I’ll sketch are:

“Independent” distiller: someone who works independently, understanding work published by other researchers and producing distillations of that work.

“Adjunct” distiller: someone who works directly with one researcher or a small team, producing regular write-ups of what the person/team is thinking about and why.

These two roles add value in slightly different ways.

An independent distiller’s main value-adds are:

Explaining the motivation and intended applications

Coming up with new examples

Boiling down the “key intuitive story” behind an argument

Showing how the intuitive story fits into the context of the intended applications

I expect the ability to come up with novel examples and boil down the core intuitive story behind a bunch of math are the rate-limiting skills here.

Rob Miles is a good example of an existing independent distiller in the field. He makes YouTube videos intuitively explaining various technical results and arguments. Rob’s work is aimed somewhat more at a popular audience than what I have in mind, but it’s nonetheless been useful for people in the field.

I expect an adjunct distiller’s main value-adds are:

Writing up explanations, examples, and intuitions, similar to the independent distiller

Saving time for the technical researcher/team; allow more specialization

Providing more external visibility/legibility into the research process and motivation

Accelerating the research process directly by coming up with good examples and intuitive explanations

I expect finding a researcher/team to work with is the rate-limiting step to this sort of work.

Mark Xu is a good example of an existing adjunct distiller. He’s worked with both Evan Hubinger and Paul Christiano, and has written up decent distillations of some of their thoughts. I believe Mark did this with the aim of later doing technical research himself, rather than mostly being a distiller. That is a pretty good strategy and I expect it to be a common pathway, though naturally I expect people who aim to specialize in distillation long-term will end up better at distillation.

What Kind Of Skills Are Needed?

I expect the key rate-limiting skills are:

Ability to independently generate intuitive examples when reading mathematical arguments, or having a mathematical discussion

Ability to extract the core intuitive story from a mathematical argument

Writing/drawing skills to clearly convey technical intuitions to a wider audience

Ability to do most of...