Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Cognitive Risks of Adolescent Binge Drinking, published by Elizabeth on July 20, 2022 on LessWrong.

The takeaway

Our goal was to quantify the cognitive risks of heavy but not abusive alcohol consumption. This is an inhernetly difficult task: the world is noisy, humans are highly variable, and institutional review boards won’t let us do challenge trials of known poisons. This makes strong inference or quantification of small risks incredibly difficult. We know for a fact that enough alcohol can damage you, and even levels that aren’t inherently dangerous can cause dumb decisions with long term consequences. All that said. when we tried to quantify the level of cognitive damage caused by college level binge drinking, we couldn’t demonstrate an effect. This doesn’t mean there isn’t one (if nothing else, “here, hold my beer” moments are real), just that it is below the threshold detectable with current methods and levels of variation in the population.

Motivation

In discussions with recent college graduates I (Elizabeth) casually mentioned that alcohol is obviously damaging to cognition. They were shocked and dismayed to find their friends were poisoning themselves, and wanted the costs quantified so they could reason with them (I hang around a very specific set of college students). Martin Bernstorff and I set out to research this together. Ultimately, 90-95% of the research was done by him, with me mostly contributing strategic guidance and somewhere between editing and co-writing this post.

Problems with research on drinking during adolescence

Literature on the causal medium- to long-term effects of non-alcoholism-level drinking on cognition is, to our strong surprise, extremely lacking. This isn’t just our poor research skills; in 2019, the Danish Ministry of Health attempted a comprehensive review and concluded that:

“We actually know relatively little about which specific biological consequences a high level of alcohol intake during adolescence will have on youth”.

And it isn’t because scientists are ignoring the problem either. Studying medium- and long-term effects on brain development is difficult because of the myriad of confounders and/or colliders for both cognition and alcohol consumption, and because more mechanist experiments would be very difficult and are institutionally forbidden anyway (“Dear IRB: we would like to violently poison some teenagers for four years, while forbidding the other half to engage in standard college socialization”). You could randomize abstinence, but we’ll get back to that.

One problem highly prevalent in alcohol literature is the abstinence bias. People who abstain from alcohol intake are likely to do so for a reason, for example chronic disease, being highly conscientious and religious, or a bad family history with alcohol. Even if you factor out all of the known confounders, it’s still vanishingly unlikely the drinking and non-drinking samples are identical. Whatever the differences, they’re likely to affect cognitive (and other) outcomes.

Any analysis comparing “no drinking” to “drinking” will suffer from this by estimating the effect of no alcohol + confounders, rather than the effect of alcohol. Unfortunately, this rules out a surprising number of studies (code available upon request).

Confounding is possible to mitigate if we have accurate intuition about the causal network, and we can estimate the effects of confounders accurately. We have to draw a directed acyclic graph with the relevant causal factors and adjust analyses or design accordingly. This is essential, but has not permeated all of epidemiology (yet), and especially for older literature, this is not done. For a primer, Martin recommends “Draw Your Assumptions” on edX here.

Additionally, alcohol consumption is a politically live topic, and papers are likely to...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: [AN #173] Recent language model results from DeepMind, published by Rohin Shah on July 21, 2022 on The AI Alignment Forum.

HIGHLIGHTS

Scaling Language Models: Methods, Analysis & Insights from Training Gopher (Jack W. Rae et al) (summarized by Rohin): This paper details the training of the Gopher family of large language models (LLMs), the biggest of which is named Gopher and has 280 billion parameters. The algorithmic details are very similar to the GPT series (AN #102): a Transformer architecture trained on next-word prediction. The models are trained on a new data distribution that still consists of text from the Internet but in different proportions (for example, book data is 27% of Gopher’s training data but only 16% of GPT-3’s training data).

Like other LLM papers, there are tons of evaluations of Gopher on various tasks, only some of which I’m going to cover here. One headline number is that Gopher beat the state of the art (SOTA) at the time on 100 out of 124 evaluation tasks.

The most interesting aspect of the paper (to me) is that the entire Gopher family of models were all trained on the same number of tokens, thus allowing us to study the effect of scaling up model parameters (and thus training compute) while holding data constant. Some of the largest benefits of scale were seen in the Medicine, Science, Technology, Social Sciences, and the Humanities task categories, while scale has not much effect or even a negative effect in the Maths, Logical Reasoning, and Common Sense categories. Surprisingly, we see improved performance on TruthfulQA (AN #165) with scale, even though the TruthfulQA benchmark was designed to show worse performance with increased scale.

We can use Gopher in a dialogue setting by prompting it appropriately. The prompt specifically instructs Gopher to be “respectful, polite, and inclusive”; it turns out that this significantly helps with toxicity. In particular, for the vanilla Gopher model family, with more scale the models produce more toxic continuations given toxic user statements; this no longer happens with Dialogue-Prompted Gopher models, which show slight reductions in toxicity with scale in the same setting. The authors speculate that while increased scale leads to an increased ability to mimic the style of a user statement, this is compensated for by an increased ability to account for the prompt.

Another alternative the authors explore is to finetune Gopher on 5 billion tokens of dialogue to produce Dialogue-Tuned Gopher. Interestingly, human raters were indifferent between Dialogue-Prompted Gopher and Dialogue-Tuned Gopher.

Read more: Blog post: Language modelling at scale: Gopher, ethical considerations, and retrieval

Training Compute-Optimal Large Language Models (Jordan Hoffmann et al) (summarized by Rohin): One application of scaling laws (AN #87) is to figure out how big a model to train, on how much data, given some compute budget. This paper performs a more systematic study than the original paper and finds that existing models are significantly undertrained. Chinchilla is a new model built with this insight: it has 4x fewer parameters than Gopher, but is trained on 4x as much data. Despite using the same amount of training compute as Gopher (and lower inference compute), Chinchilla outperforms Gopher across a wide variety of metrics, validating these new scaling laws.

You can safely skip to the opinion at this point – the rest of this summary is quantitative details.

We want to find functions N(C) and D(C) that specify the optimal number of parameters N and the amount of data D to use given some compute budget C. We’ll assume that these scale with a power of C, that is, N(C) = k_N C^a and D(C) = k_D C^b, for some constants a, b, k_N, and k_D. Note that since total compute increases linearly with both N (since each...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: EAs should use Signal instead of Facebook Messenger, published by Peter S. Park on July 21, 2022 on The Effective Altruism Forum.

Acknowledgements: I am extremely grateful to Michael Einhorn for this idea, and to Samuel Chen, Stephen Fowler, and Luise Wöhlke for their feedback on this post.

tl;dr. Facebook Messenger is dangerously insecure. It only takes one data breach of one insufficiently careful EA to harm the movement. Thus, all EAs should use Signal instead of Facebook Messenger for their internal messages.

The Post: In my experience, groupchats among effective altruists usually happen on either Slack (for work-related settings) or Facebook Messenger (for social settings in general). I've heard that the EA communities of Germany and the U.K. also use Signal, which I propose is a more impactful choice.

As a rule of thumb, all Facebook Messenger conversations between effective altruists should be moved to Signal.

How it works: "Signal is designed to never collect or store any sensitive information. Signal messages and calls cannot be accessed by us or other third parties because they are always end-to-end encrypted, private, and secure." (quoted from their website)

By default, Facebook Messenger is not end-to-end encrypted. Messages sent on the default version of Messenger are stored on Facebook's servers, which can be accessed by Facebook, law enforcement, and hackers. And Facebook has confirmed that it spies on Messenger conversations.

There is a (highly unknown) encrypted version of Facebook Messenger called Secret Conversations. There is also WhatsApp, a Facebook-owned message app which claims to have end-to-end encryption. However, even messages on WhatsApp can be accessed by Facebook and by law enforcement. This is compelling evidence that Facebook cannot be trusted with our messages.

Moreover, using any of these Facebook-owned message apps implicitly endorses, and potentially results in training data for, Facebook's current trajectory. This is a trajectory that can plausibly result in a castastrophe (even human extinction) due to an unaligned artificial intelligence developed by Facebook AI in the future.

In contrast, Signal is a nonprofit whose mission is to provide open-source privacy technology to the public. We know that Signal does not store any data beyond a name and phone number (optional), the time of account creation, and the last log-in time. This has been verified by legally binding subpoenas.

Question: "What if I have nothing to hide?" This belief would most likely be overconfident. If effective altruists' messages are hacked, taken out of context, and publicly revealed, it could substantially and even permanently harm the movement. Consider the example of John Podesta, chair of Hillary Clinton's 2016 presidential campaign. Many of his emails, including those that made Clinton and her campaign look bad, were obtained by hackers in a data breach and published in Wikileaks. This event is likely to have at least partially contributed to Clinton's loss to Donald Trump.

Question: "What if I'm very careful about what I say on Facebook Messenger?" Even if you are very careful, other effective altruists may not be. It only takes one data breach of one insufficiently careful EA to harm the movement. So, effective altruists who perpetuate our community's use of Facebook Messenger are forcing a painful choice between risking reputational harms (e.g., from a data breach) and sacrificing epistemics (e.g., if the whole EA community walks on eggshells in order to not look bad in the event of a data breach). We can avoid this painful choice by agreeing as a community to use Signal.

Question: "Isn't Facebook's event function very useful to make EA events known to the public?" I completely agree, and think it may often be a worthwhile tradeoff to use Facebook's ...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: If you're unhappy, consider leaving, published by Justis on July 20, 2022 on The Effective Altruism Forum.

When I talk to random people around my town, and ask how they're doing, a large fraction of them open with "well, the world sucks, but..."They do this whether or not they, personally, are about to report that they're doing well. It feels like a necessary caveat to them. To me, this is a sign that they're in a social context that cuts against their thriving. Does the world suck? Very hard to say with confidence. The world is very big, and there's no clear objective standard. But carrying around "the world sucks" is heavy, and makes it harder to enjoy life.

So I'd like to tell them to consider just... not thinking that anymore? The belief probably isn't paying rent. It's probably not rigorously considered. Just this cached thing, sitting around, making life slightly worse.

But I don't, because they're random acquaintances and their world models are their business. Same with you, reader of this post. But I'm going to make a recommendation to you, anyway. Consider leaving the organized EA movement for a while.

Wait, what?

If your reaction is "but this movement is great! I'm really happy and energized and ready to make the biggest difference I can with all my new exciting friends" then yeah, keep doing your thing! I probably don't have anything to offer you. And for what it's worth, I also agree the movement is great, and I am personally somewhat involved right now.

But I think it's pretty clear that lots of people in the EA orbit are persistently unhappy. And the solution to being persistently unhappy with a social arrangement or memeplex, usually, is to leave.

Famously, this often doesn't occur to the person suffering for a long time, if ever, even if it looks like the obvious correct choice from the outside.

What do I mean by leaving?

I don't mean "stop caring about doing good". If you've taken the Giving What We Can pledge, for example, I'd definitely keep it. What I mean is something more like "stop expecting EA to provide for any of your core needs." This includes, at least:

Social - have non-EA friends. Ideally have some be local. Talk about other stuff with them, mostly.

Financial - do not rely on EA funding sources for income that you couldn't do without. Don't apply for EA jobs.

Emotional - do not have a unidimensional sense of self-worth that boils down to "how am I scoring on a vague, amorphous impact scale as envisioned in EA terms".

Actually, that's probably about it. If you're persistently unhappy, and don't feel fulfilled socially, financially, or emotionally, stop doing those three things.

What are the upsides?

First, consider yourself. It's easy to take this idea too far, but the person you have the greatest responsibility for - except perhaps your children or in some niche situations your spouse - is yourself. You have direct access to your emotional state. If you're not flourishing, it is first and foremost your responsibility to address that. And addressing it is worth something, even in the EA framework.

Second, consider bias. I think there's a common story which goes something like:

Well I need to make sure I make a good impression on people at orgs X and Y, since I might want to work for them. But they have close relationships with Z and A. So I need to be rubbing elbows with those people, too. To do that it'll help for me to live in the most expensive city in the world, which makes it really important I don't irritate C and D, because I'll need substantial funding just to not burn through savings.

I didn't properly appreciate this for a long time, but I keep seeing people be nervous to like, seriously criticize EA, because their (aspirational) livelihoods depend on it.

This is very bad for epistemology. It may be the single worst thing for ep...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: The value of content density, published by Joey on July 20, 2022 on The Effective Altruism Forum.

Different communities have different norms when it comes to writing and content production. One stylistic difference the EA community has, is that forum posts are often comparatively long, deep and advanced. I really like aspects of this and I think that having a relatively high level of depth can be very nice to offset the other mediums of communication EA uses (e.g., facebook). However, I wish EA forum authors would give content density a little bit more consideration.

Different books have different levels of content density. Some have a strong concept on almost every page (a good example of this would be Ray Dalio's Principles), while other books could be summarized fairly easily into a single, short blog post (I think Carol Dweck’s Mindset falls into this category). There is often a strong social pressure to make something ~book length, so even if it's a bit concept-light it will get stretched to meet a word count. In the world of blog posts, no such limit exists but there are social norms that tend to affect the average duration. I think for entertainment blog posts there can be a lot of value in longer-form writing (e.g., Wait But Why’s writing), but for blog posts that are aiming to get a concept across (like this one) a leaner form of writing would be better. I do agree we should try to make writing entertaining, just not with huge length increases as a trade off. (A good example of both high content density + low wordcount).

Why we should aim for higher content density

Time limited: Many EA forum readers are time limited and thus it is often a worthwhile trade to write a post that covers 90% of the value and is half as long. I often see blog posts that have a good concept or two, but due to how long they are they get far less engagement and readership than they would otherwise.

Counterfactuals: There is quite a volume of posts on the EA forum and this directly ties into most readers' opportunity cost. Oftentimes the trade-off might literally be reading two EA forum posts vs one that is double the length.

Approachability: EAs are a fairly technical bunch and although the use of jargon has been written about, having a high volume of content with relatively sparse concepts also makes it harder for new people to get up to speed. It can make the forum as a whole seem more intimidating than it has to be.

Possible heuristics

Length: The rule of thumb I use for blog posts is ~0.5-1 page per major concept and 1-3 pages per blog post. This kind of cap creates a strong force toward brevity and conveying concepts concisely. It does sometimes require major editing (the quote “I would have written you a shorter letter but did not have the time” comes to mind).

TL;DR and summaries: I think the EA forum makes great use of these and they are worth including in almost every post that is over one page.

Top concept leveraging: I think blog posts often try to cover a large number of topics that could easily be broken into multiple posts. For example, this data on value drift post being separate from this strategies for preventing value drift post makes each piece of writing much more individually referable and digestible.

More information

Some examples of content-dense writing include:

EA Dedicates

Critiques of EA that I want to read

A summary of every Replacing Guilt post

Further reading: Steven Pinker has some really well-articulated and pointed suggestions for community writing norms, many of which I think would apply directly to the EA forum.

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Global Animal Slaughter Statistics & Charts: 2022 Update, published by JLRiedi on July 19, 2022 on The Effective Altruism Forum.

Faunalytics has produced an update to their Global Animal Slaughter Statistics and Charts, which now includes numbers from 2019 and 2020 and documents how many animals are slaughtered for food every year based on United Nations FAO data.

Globally, animal slaughter is outpacing population growth. Pig and sheep slaughter is trending upward, cow slaughter is trending downward, and chicken slaughter for 2020 was down by 2 billion from 2019 but still trending upward overall compared to every year before 2018. Fish slaughter is more volatile, but looks like it could be plateauing.

For animal advocates looking to direct their attention and resources more effectively, the statistics and charts in this valuable visual resource offer a helpful picture of how many animals have been consumed and how consumption has changed over the past 60 years.

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: How to Diversify Conceptual Alignment: the Model Behind Refine, published by Adam Shimi on July 20, 2022 on The AI Alignment Forum.

This work has been done while at Conjecture

Tl;dr: We need far more conceptual AI alignment research approaches than we have now if we want to increase our chances to solve the alignment problem. However, the conceptual alignment field remains hard to access, and what feedback and mentorship there is focuses around few existing research directions rather than stimulating new ideas. This model lead to the creation of Refine, a research incubator for potential conceptual alignment researchers funded by the LTFF and hosted by Conjecture. Its goal is to help conceptual alignment research grow in both number and variety, through some minimal teaching and a lot of iteration and feedback on incubatees’ ideas. The first cohort has been selected, and will run from August to October 2022. In the bigger picture, Refine is an experiment within Conjecture to find ways of increasing the number of conceptual researchers and improve the rate at which the field is making productive mistakes.

The Problem: Not Enough Varied Conceptual Research

I believe that in order to solve the alignment problem, we need significantly more people attacking it from a lot different angles.

Why? First because none of the current approaches appears to yield a full solution. I expect many of them to be productive mistakes we can and should build on, but they don't appear sufficient, especially with shorter timelines.

In addition, the history of science teaches us that for many important discoveries, especially in difficult epistemic situations, the answers don't come from one lone genius seeing through the irrelevant details, but instead from bits of evidence revealed by many different takes and operationalizations (possibly unified and compressed together at the end). And we should expect alignment to be hard based on epistemological vigilance.

So if we accept that we need more people tackling alignment in more varied ways, why are we falling short of that ideal? Note that I will focus here on conceptual researchers, as they are the source of most variations on the problem, and because they are so hard to come by.

I see three broad issues with getting more conceptual alignment researchers working on wildly different approaches:

(Built-in Ontological Commitments) Almost all current attempts to create more conceptual alignment researchers (SERI MATS, independent mentoring...) rely significantly on mentorship by current conceptual researchers. Although this obviously comes with many benefits, it also leads to many ontological commitments being internalized when one is learning the field. As such, it's hard to go explore a vastly different approach because the way you see the problem has been moulded by this early mentorship.

(Misguided Requirements) I see many incorrect assumptions about what it takes to be a good conceptual researcher floating around, both from field-builders and from potential candidates. Here's a non-exhaustive list of the most frustrating ones

You need to know all previous literature on alignment (the field has more breadth than depth, and so getting a few key ideas is more important than knowing everything)

You need to master maths and philosophy (a lot of good conceptual work only uses basic maths and philosophy)

You need to have an ML background (you can pick up the relevant part and just work on approaches different to pure prosaic alignment)

(No Feedback) If you want to start on your own, you will have trouble getting any feedback at all. The AF doesn't provide much feedback even for established researchers, and it has almost nothing in store for newcomers. Really, the main source of feedback in the field is asking other researchers, but when you start...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Applications are open for CFAR workshops in Prague this fall!, published by JohnSteidley on July 19, 2022 on LessWrong.

tldr: CFAR’s running updated versions of our mainline rationality workshop this fall in the Prague area. Apply here.

CFAR workshops are back! After a nearly three-year hiatus during COVID, we're going to be running a series of four workshops this fall in the Czech Republic.

So what is a CFAR workshop again?

These workshops are four-and-a-half-day intensive retreats that are designed around the rationality techniques CFAR developed in its 10+ year history. These techniques vary: some we pulled straight from academic literature (looking at you TAPs), others we adapted from outside practices (Gendlin’s Focusing), while some we developed wholesale (hello Double Crux, Goal Factoring, and all that which is still unnamed). Our goal with this workshop is to create an environment in which participants are able to gain new insight into their minds and their decision-making processes.

Why would you come to a CFAR workshop?

Because you want to! That’s the most important part.

But maybe also you’ve got a sense of stuckness, and a new frame or tool might help you shift. Maybe it’s exactly the opposite of stuck; you’re moving and changing, and you want to be deliberate about how you move forward. Maybe you’re just intrigued and excited about this thing you’ve heard about from your friends. No guarantees from us on the result, but folks often walk away with insight, new fascinating friends, and some outstanding ideas. Most participants answer that they were glad they came (the only trick is figuring out how to select yourself into the correct reference class.)

We think that the workshop would be particularly valuable if you:

Are enthusiastic about rationality and you want to nerd out

Have an exploratory/playful/experimental mindset - you like to try things!

Are looking for an environment where you can have open and honest conversations about what you care about

A caveat, about this workshop...

It’s experimental. Which, if you’re familiar with CFAR’s history, is true of much of our workshops. If the material isn’t alive and real and practiced by our instructors, we don’t believe it will be for participants. And since it’s been a significant nearly-3-years since our last workshop, we and our beliefs have shifted a bit. It’ll still be fairly recognizable for those familiar with our applied rationality, but we want to set your expectations. The most notable shift is we’re saying goodbye to our problem-oriented approach (colloquially known as “bugs”) and focusing on the perspective of asking: what do you actually want and how can we move toward that? We’ll also be incorporating a holistic approach to your goals and needs, an understanding of the limits of improved productivity, an orientation to uncertainty, and more.

Where and when?

There will be four workshops this fall in Prague, Czech Republic (precisely: in a village about 90 min from Prague). All workshops start at 6pm on arrival day and finish with breakfast at 9am on departure day.

September 29 - October 4

October 20 - October 25

November 3 - November 8

November 16 - November 21 (to be confirmed)

There’s also going to be the Prague Fall Season (more info soon!), full of fun and interesting events that will be happening concurrently. Consider coming early or staying late if you want to spend some time taking part in that or just exploring.

If going to Prague is too inconvenient or expensive for you, the odds of spring 2023 workshops in California are higher than ever (but no promises at this point).

How do I get in on this?

Apply here. We’ll be processing applications on a rolling basis. We’re aiming to get back to everyone within 2 weeks of their application. Depending on the demand, we might ask you to provide ...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Applications are open for CFAR workshops in Prague this fall, published by John Steidley on July 20, 2022 on The Effective Altruism Forum.

tldr: CFAR’s running updated versions of our mainline rationality workshop this fall in the Prague area. Apply here.

CFAR workshops are back! After a nearly three-year hiatus during COVID, we're going to be running a series of four workshops this fall in the Czech Republic.

So what is a CFAR workshop again?

These workshops are four-and-a-half-day intensive retreats that are designed around the rationality techniques CFAR developed in its 10+ year history. These techniques vary: some we pulled straight from academic literature (looking at you TAPs), others we adapted from outside practices (Gendlin’s Focusing), while some we developed wholesale (hello Double Crux, Goal Factoring, and all that which is still unnamed). Our goal with this workshop is to create an environment in which participants are able to gain new insight into their minds and their decision-making processes.

Why would you come to a CFAR workshop?

Because you want to! That’s the most important part.

But maybe also you’ve got a sense of stuckness, and a new frame or tool might help you shift. Maybe it’s exactly the opposite of stuck; you’re moving and changing, and you want to be deliberate about how you move forward. Maybe you’re just intrigued and excited about this thing you’ve heard about from your friends. No guarantees from us on the result, but folks often walk away with insight, new fascinating friends, and some outstanding ideas. Most participants answer that they were glad they came (the only trick is figuring out how to select yourself into the correct reference class.)

We think that the workshop would be particularly valuable if you:

Are enthusiastic about rationality and you want to nerd out

Have an exploratory/playful/experimental mindset - you like to try things!

Are looking for an environment where you can have open and honest conversations about what you care about

A caveat, about this workshop...

It’s experimental. Which, if you’re familiar with CFAR’s history, is true of much of our workshops. If the material isn’t alive and real and practiced by our instructors, we don’t believe it will be for participants. And since it’s been a significant nearly-3-years since our last workshop, we and our beliefs have shifted a bit. It’ll still be fairly recognizable for those familiar with our applied rationality, but we want to set your expectations. The most notable shift is we’re saying goodbye to our problem-oriented approach (colloquially known as “bugs”) and focusing on the perspective of asking: what do you actually want and how can we move toward that? We’ll also be incorporating a holistic approach to your goals and needs, an understanding of the limits of improved productivity, an orientation to uncertainty, and more.

Where and when?

There will be four workshops this fall in Prague, Czech Republic (precisely: in a village about 90 min from Prague). All workshops start at 6pm on arrival day and finish with breakfast at 9am on departure day.

September 29 - October 4

October 20 - October 25

November 3 - November 8

November 16 - November 21 (to be confirmed)

There’s also going to be the Prague Fall Season (more info soon!), full of fun and interesting events that will be happening concurrently. Consider coming early or staying late if you want to spend some time taking part in that or just exploring.

If going to Prague is too inconvenient or expensive for you, the odds of spring 2023 workshops in California are higher than ever (but no promises at this point).

How do I get in on this?

Apply here. We’ll be processing applications on a rolling basis. We’re aiming to get back to everyone within 2 weeks of their application. Depending on the demand, we might ...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Bounded complexity of solving ELK and its implications, published by Rubi on July 19, 2022 on The AI Alignment Forum.

This post was written for the SERI MATS program. I thank Evan Hubinger and Leo Gao for their mentorship in the program. Further thanks go to Simon Marshall and Leo Gao (again) for specific comments regarding the content of this post.

The Eliciting Latent Knowledge (ELK) problem was first introduced by Paul Christiano, Mark Xu, and Ajeya Cotra. Arguments concerning the complexity of solving the problem and the resulting consequences of that have been made by Abram Demski and Leo Gao. This post aims to synthesize their thoughts in an accessible way, and extend them with my own original content. I assume familiarity with the initial ELK report, but not with subsequent posts.

Epistemic status: 90% confident that I am accurately representing the thoughts of Abram and Leo at the time they wrote their posts, 70% confident my own arguments do not contain major logical flaws.

Introduction

We would like to solve the Eliciting Latent Knowledge (ELK) problem in the worst case scenario, or at least show that the task is practically impossible so that we can revise our expectations. One of the worst cases for ELK is when the direct reporter is arbitrarily computationally complex. As this case would still allow for approximating the direct reporter within certain tolerances, we extend it further to when the direct reporter and all other sufficiently close approximation are arbitrarily computationally complex. While Paul Christiano has suggested that the complexity of the predictor bounds the complexity of the reporter, we will show later that this only holds in a narrow formulation of the ELK problem.

Abram Demski argues in "ELK Computational Complexity: Three Levels of Difficulty" that if the direct reporter can be arbitrarily computationally complex, a training process to find it is intractable. To get around this issue, it is necessary to find an upper bound on computational complexity, conditional on the predictor, thus ruling out the worst case scenario. Leo Gao's post "Observations about ELK" focuses on a limited case of ELK and arrives at marginally more optimistic results, arguing that proposals which do not depend on the predictor are unable to find the direct translator in the worst case scenario, which rules out most current proposals but leaves open the possibility of a new approach that can somehow get around the issues.

This point summarizes and expands on Abram’s and Leo’s posts regarding the possibility and implications of arbitrarily computationally complex direct reporters. I make the case that, under relatively minor assumptions, it is possible to bound the computational complexity of answering any particular question that has an answer. However, I then go on to show that Leo’s argument against the tractability of proposals not depending on the predictor continues to apply even when computational complexity is bounded.

Question Nets

In the ELK report, it is assumed that both the human and the AI make predictions by doing inference in a Bayes net. While this may not be literally true, it is a useful approximation because it is often the case that there is an analogous Bayes net for however the agent is modeling the world. This is especially likely to hold true for sufficiently advanced agents, such as a human or an AI that can model the world at a superhuman level, so the Bayes net analogy is a safe one to make.

Once we acknowledge that doing inference in a Bayes net is only an analogy, we can stretch the analogy further. In particular, think of a human’s Bayes net as consisting exactly of a node for every question they could conceive to ask. The information contained at each of these question nodes is the human’s beliefs over the distribution of a...