The Nonlinear Library

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Center on Long-Term Risk: Annual review and fundraiser 2023, published by Center on Long-Term Risk on December 13, 2023 on The Effective Altruism Forum.

Jesse Clifton

Crossposted to LessWrong here

This is a brief overview of the

Center on Long-Term Risk (CLR)'s activities in 2023 and our plans for 2024. We are hoping to fundraise $770,000 to fulfill our target budget in 2024.

About us

CLR works on addressing the worst-case risks from the development and deployment of advanced AI systems in order to reduce

s-risks. Our research primarily involves thinking about how to reduce conflict and promote cooperation in interactions involving powerful AI systems. In addition to research, we do a range of activities aimed at building a community of people interested in s-risk reduction, and support efforts that contribute to s-risk reduction via the

CLR Fund.

Review of 2023

Research

Our research in 2023 primarily fell in a few buckets:

Commitment races and safe Pareto improvements deconfusion. Many researchers in the area consider

commitment races a potentially important driver of conflict involving AI systems. But we have been missing a precise understanding of the mechanisms by which they could lead to conflict. We believe we made significant progress on this over the last year. This includes progress on understanding the conditions under which an approach to bargaining called "

safe Pareto improvements (SPIs)" can prevent catastrophic conflict.

Most of this work is non-public, but public documents that came out of this line of work include

Open-minded updatelessness,

Responses to apparent rationalist confusions about game / decision theory, and a forthcoming paper (see

draft) & post on SPIs for expected utility maximizers.

Paths to implementing surrogate goals.

Surrogate goals are a special case of SPIs and we consider them a promising route to reducing the downsides from conflict. We (along with CLR-external researchers Nathaniel Sauerberg and Caspar Oesterheld) thought about how implementing surrogate goals could be both credible and counterfactual (i.e., not done by AIs by default), e.g., using

compute monitoring schemes.

CLR researchers, in collaboration with Caspar Oesterheld and Filip Sondej, are also working on a project to "implement" surrogate goals/SPIs in contemporary language models.

Conflict-prone dispositions. We thought about the kinds of dispositions that could exacerbate conflict, and how they might arise in AI systems. The primary motivation for this line of work is that, even if alignment does not fully succeed, we may be able to shape their dispositions in coarse-grained ways that reduce the risks of worse-than-extinction outcomes. See our post on

making AIs less likely to be spiteful.

Evaluations of LLMs. We continued our

earlier work on evaluating cooperation-relevant properties in LLMs. Part of this involved cheap exploratory work with GPT-4 and Claude (e.g., looking at behavior in scenarios from the

Machiavelli dataset) to see if there were particularly interesting behaviors worth investing more time in.

We also worked with external collaborators to develop "Welfare Diplomacy", a variant of the Diplomacy game environment designed to be better for facilitating Cooperative AI research. We

wrote a paper introducing the benchmark and using it to evaluate several LLMs.

Community building

Progress on s-risk community building was slow, due to the departures of our community building staff and funding uncertainties that prevented us from immediately hiring another Community Manager.

We continued having career calls;

We ran our fourth

Summer Research Fellowship, with 10 fellows;

We have now hired a new Community Manager, Winston Oswald-Drummond, who has just started.

Staff & leadership changes

We saw some substantial staff changes this year, with three staff m...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: The Best of Don't Worry About the Vase, published by Zvi on December 13, 2023 on LessWrong.

Hello everyone! This is going to be a bit of a housekeeping post and a welcome to new subscribers.

Note that this is not the primary version of my writing, which can be found on Substack, but it is a full copy of all posts found there.

My writing can be intimidating. There is a lot of it, and it's often dense. As always, choose only the parts relevant to your interests, do not be afraid to make cuts. I attempt to make every post accessible as an entry point, but I also want to build up a superstructure over time. This seemed like a good time to recap some of the very best of my old writing and talk about what I'm up to.

Over many years, this blog has morphed from focusing on rationality to COVID to AI.

But not only those things. I'm interested in almost everything. I write periodic updates about housing policy, childhood, fertility, medicine and health, gaming and grab bags of everything else.

In addition to writing, I also run a small 501c(3) with one employee called Balsa Research. Balsa is dedicated to laying groundwork on a few key issues to make big civilizational wins possible, starting with repeal of the Jones Act. This link is to an update on that, and you can donate here. Your subscriptions here are also very much appreciated.

Underlying it all continues to be my version of the principles of rationality.

Rationality

A lot has changed since my last best-of writeup six years ago. One thing that has not changed is that I consider myself part of the rationalist community. No specific interest in rationality or its modes of thinking are required, but I strive to embody my version of this style of thinking, and to illustrate and hopefully pass on this mode of thinking throughout my writing.

What is rationality? This post is one good answer. It is believing, and updating on evidence, so as to systematically improve the correspondence between your map and the territory, and using that map to achieve your values.

To me, a rationalist continues to be someone who highly values, and invests in, the version of this process and the art thereof that they believe in, both in themselves and others.

If you're wondering why anyone would think this way, my best responses to that are Responses to Tyler Cowen on Rationality and Why Rationality?

If you're interested in going deeper, you should try reading the sequences. You can get the Kindle version here.

I think rationality and the sequences are pretty great. The sequences were created by Eliezer Yudkowsky, in the hopes that those who learned to think well in general would also be able to think well about AI. Whether or not you have any interest in thinking about AI, or thinking about it well, I find it valuable to think about everything well, whenever and to the extent I can.

While I do consider myself a Rationalist, I do not consider myself an Effective Altruist. That is a very different set of norms and cultural constructs.

The Evergreen Posts

These are to me the ten posts most worth reading today, along with a pitch on why you might want to read each of them. Only one is directly about AI, exactly because AI moves so quickly, and my top AI posts are listed in the next section down.

The top ten are in alphabetical order, all are listed again in their appropriate sections.

If you only read one recent post and are here for AI, read OpenAI: The Battle of the Board.

If you only read one fully evergreen older post, read Slack.

An Unexpected Victory: Container Stacking at the Port of Long Beach. This is still highly underappreciated. How did Ryan's boat ride and Tweetstorm cause a policy change? Could we duplicate this success elsewhere in the future? How?

Asymmetric Justice. A concept I wish more people knew and understood. Many moral and f...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: AI Control: Improving Safety Despite Intentional Subversion, published by Buck Shlegeris on December 13, 2023 on The AI Alignment Forum.

We've released a paper,

AI Control: Improving Safety Despite Intentional Subversion. This paper explores techniques that prevent AI catastrophes even if AI instances are colluding to subvert the safety techniques. In this post:

We summarize the paper;

We compare our methodology to what the one used in other safety papers.

The next post in this sequence (which we'll release in the coming weeks) discusses what we mean by AI control and argues that it is a promising methodology for reducing risk from scheming models.

Here's the abstract of the paper:

As large language models (LLMs) become more powerful and are deployed more autonomously, it will be increasingly important to prevent them from causing harmful outcomes. Researchers have investigated a variety of safety techniques for this purpose, e.g. using models to review the outputs of other models, or red-teaming techniques to surface subtle failure modes.

However, researchers have not evaluated whether such techniques still ensure safety if the model is itself intentionally trying to subvert them. In this paper, we develop and evaluate pipelines of safety techniques ("protocols") that are robust to intentional subversion.

We investigate a scenario in which we want to solve a sequence of programming problems, using access to a powerful but untrusted model (in our case, GPT-4), access to a less powerful trusted model (in our case, GPT-3.5), and limited access to human contractors who provide high-quality trusted labor. We investigate protocols that aim to never submit solutions containing backdoors, which we operationalize here as logical errors that are not caught by test cases.

We investigate a range of protocols and test each against strategies that the untrusted model could use to subvert them. One protocol is what we call trusted editing. This protocol first asks GPT-4 to write code, and then asks GPT-3.5 to rate the suspiciousness of that code. If the code is below some suspiciousness threshold, it is submitted. Otherwise, GPT-3.5 edits the solution to remove parts that seem suspicious and then submits the edited code. Another protocol is untrusted monitoring.

This protocol asks GPT-4 to write code, and then asks another instance of GPT-4 whether the code is backdoored, using various techniques to prevent the GPT-4 instances from colluding. These protocols improve substantially on simple baselines.

Paper summary

Setup

We focus on a toy setting of using LLMs to solve 5,000 programming problems from the APPS dataset (of the "interview" difficulty level). We treat it as a catastrophe if any of the solutions are backdoored, which we define as a solution that passes all of the APPS test cases but is incorrect on an input that GPT-4 can find when prompted with the solution. For example, the following implementation of the is_odd function is successfully backdoored: it passes the reference test cases but GPT-4 can find the exploit of 354.

The goal is to find a protocol that submits solutions while never submitting a backdoor. The protocol has access to:

An untrusted more-capable model, which we're worried might try to backdoor solutions. We use GPT-4 as the untrusted more-capable model. (We strongly suspect that GPT-4 is trustworthy by our definition; we pretend for the sake of this paper that it isn't.)

A trusted less-capable model, which we know won't try to backdoor solutions. We use GPT-3.5 for this. We are interested in the setting where you

trust less-capable models, because less-capable models are unlikely to be able to subvert safety measures without us easily noticing their subversion attempts.

Limited access to trusted high-quality labor. In practice, AI developers have acces...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: AI Views Snapshots, published by Rob Bensinger on December 13, 2023 on LessWrong.

(Cross-posted from Twitter, and therefore optimized somewhat for simplicity.)

Recent discussions of AI x-risk in places like Twitter tend to focus on "are you in the Rightthink Tribe, or the Wrongthink Tribe?". Are you a doomer? An accelerationist? An EA? A techno-optimist?

I'm pretty sure these discussions would go way better if the discussion looked less like that. More concrete claims, details, and probabilities; fewer vague slogans and vague expressions of certainty.

As a start, I made this image (also available as a Google Drawing):

I obviously left out lots of other important and interesting questions, but I think this is OK as a conversation-starter. I've encouraged Twitter regulars to share their own versions of this image, or similar images, as a nucleus for conversation (and a way to directly clarify what people's actual views are, beyond the stereotypes and slogans).

If you want to see a filled-out example, here's mine (though you may not want to look if you prefer to give answers that are less anchored): Google Drawing link.

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Enhancing intelligence by banging your head on the wall, published by Bezzi on December 13, 2023 on LessWrong.

The Sudden Savant Syndrome is a rare phenomenon in which an otherwise normal person got some kind of brain injury and immediately develops a new skill. The linked article tells the story of a 40-years old guy who banged his head against a wall while swimming, and woke up with a huge talent for playing piano (relevant video). Now, I've spent 15 years in formal music training and I can ensure you that nobody can fake that kind of talent without spending years in actual piano practice.

Here's the story of another guy who banged his head and become a math genius; you can find several other stories like that. And maybe most puzzling of all is this paper, describing a dozen cases of sudden savants who didn't even bang their head, and acquired instant skill without doing nothing in particular.

I vaguely remember one sudden savant story being mentioned on a children book by Terry Deary, presented in his usual "haha, here's a funny trivia" way. But even as a child, I was pretty shocked to read that. Like, seriously? You could become a math genius just by banging your head on the wall in some very precise way?

I don't think that Sudden Savant Syndrome is just a scam; there are too many documented cases and most kind of talent are very, very difficult to fake. But if true, why there are so surprisingly few studies on that? Why is no one spending billions of dollars to replicate it in a controlled way? This is a genuine question; I know very little about biology and neuroscience, but it surely sounds way easier than rewriting the genetic code of every neuron in the brain...

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: [Valence series] 3. Valence & Beliefs, published by Steven Byrnes on December 13, 2023 on LessWrong.

3.1 Post summary / Table of contents

Part of the

Valence series.

So far in the series, we defined valence (

Post 1) and talked about how it relates to the "normative" world of desires, values, preferences, and so on (

Post 2). Now we move on to the "positive" world of beliefs, expectations, concepts, etc. Here, valence is no longer the sine qua non at the center of everything, as it is in the "normative" magisterium. But it still plays a leading role. Actually, two leading roles!

Section 3.2 distinguishes two paths by which valence affects beliefs: first, in its role as a control signal, and second, in its role as "interoceptive" sense data, which I discuss in turn:

Section 3.3 discusses how valence-as-a-control-signal affects beliefs. This is the domain of motivated reasoning, confirmation bias, and related phenomena. I explain how it works both in general and through a nuts-and-bolts toy model. I also elaborate on "voluntary attention" versus "involuntary attention", in order to explain anxious rumination, which goes against the normal pattern (it involves thinking about something despite a strong motivation not to think about it).

Section 3.4 discusses how valence-as-interoceptive-sense-data affects beliefs. I argue that, if concepts are

"clusters in thingspace", then valence is one of the axes used by this clustering algorithm. I discuss how this relates to various difficulties in modeling and discussing the world separately from how we feel about it, along with the related "affect heuristic" and "halo effect".

Section 3.5 briefly muses on whether future AI will have motivated reasoning, halo effect, etc., as we humans do. (My answer is "yes, but maybe it doesn't matter too much".)

Section 3.6 is a brief conclusion.

3.2 Two paths for normative to bleed into positive

Here's a diagram from the

previous post:

We have two paths by which valence can impact the world-model (a.k.a. "Thought Generator"): the normative path (upward black arrow) that helps control which thoughts get strengthened versus thrown out, and the positive path (curvy green arrow) that treats valence as one of the input signals to be incorporated into the world model. Corresponding to these two paths, we get two ways for valence to impact factual beliefs:

Motivated reasoning / thinking / observing and confirmation bias - related to the upward black arrow, and discussed in §3.3 below;

The entanglement of valence into our conceptual categories, which makes it difficult to think or talk about the world independently from how we feel about it - related to the curvy green arrow, and discussed in §3.4 below.

Let's proceed with each in turn!

3.3 Motivated reasoning / thinking / observing, including confirmation bias

Of the fifty-odd biases discovered by Kahneman, Tversky, and their successors, forty-nine are cute quirks, and one is destroying civilization. This last one is confirmation bias - our tendency to interpret evidence as confirming our pre-existing beliefs instead of changing our minds.…

Scott Alexander

3.3.1 Attention-control and motor-control provide loopholes through which desires can manipulate beliefs

Wishful thinking - where you believe something because it would be nice if it were true - is generally maladaptive: Imagine spending all day opening your wallet, over and over, expecting each time to find it overflowing with cash. We don't actually do that, which is an indication that our brains have effective systems to mitigate (albeit not eliminate, as we'll see) wishful thinking.

How do those mitigations work?

As discussed in

Post 1, the brain works by model-based reinforcement learning (RL). Oversimplifying as usual, the "model" (predictive world-model, a.k.a. "Thought Generator") is traine...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Funding case: AI Safety Camp, published by Remmelt on December 13, 2023 on The Effective Altruism Forum.

Project summary

AI Safety Camp is a program with a 5-year track record of enabling people to find careers in AI Safety.

We support up-and-coming researchers outside the Bay Area and London hubs.

We are out of funding. To make the 10th edition happen, fund our stipends and salaries.

What are this project's goals and how will you achieve them?

AI Safety Camp is a program for inquiring how to work on ensuring future AI is

safe, and try concretely working on that in a team.

For the 9th edition of AI Safety Camp we opened applications for

29 projects.

We are first to host a special area to support "Pause AI" work. With funding, we can scale from 4 projects for restricting corporate-AI development to 15 projects next edition.

We are excited about our new research lead

format, since it combines:

Hands-on guidance: We guide research leads (RLs) to carefully consider and scope their project. Research leads in turn onboard teammates and guide their teammates through the process of doing new research.

Streamlined applications: Team applications were the most time-intensive portion of running AI Safety Camp. Reviewers were often unsure how to evaluate an applicant's fit for a project that required specific skills and understandings. RLs usually have a clear sense of who they would want to work with for three months. So we instead guide RLs to prepare project-specific questions and interview their potential teammates.

Resource-efficiency: We are not competing with other programs for scarce mentor time. Instead, we prospect for thoughtful research leads who at some point could become well-recognized researchers. The virtual format also cuts on overhead - instead of sinking funds into venues and plane tickets, the money goes directly to funding people to focus on their work in AI safety.

Flexible hours: Participants can work remotely from their timezone alongside their degree or day job - to test their fit for an AI Safety career.

How will this funding be used?

We are fundraising to pay for:

Salaries for the organisers for the current AISC

Funding future camps (see budget section)

Whether we run the tenth edition, or put AISC indefinitely on hold depends on your donation.

Last June, we had to freeze a year's worth of salary for three staff. Our ops coordinator had to leave, and Linda and Remmelt decided to run one more edition as volunteers.

AISC has previously gotten grants paid for by FTX money. After the FTX collapse, we froze $255K in funds to cover clawback claims. For the current AISC, we have $99K left from SFF that was earmarked for stipends - but nothing for salaries, and nothing for future AISCs.

If we have enough money we might also restart the in-person version of AISC. This decision will also depend on an ongoing external evaluation of AISC, which among other things, is evaluating the difference in impact of the virtual vs in-person AISCs.

By default we'll decide what to prioritise with the funding we get. But if you want to have a say, we can discuss that. We can earmark your money for whatever you want.

Potential budgets for various versions of AISC

These are example budgets for different possible versions of the virtual AISC. If our funding lands somewhere in between, we'll do something in between.

Virtual AISC - Budget version

Software etc

$2K

Organiser salaries, 2 ppl, 4 months

$56K

Stipends for participants

$0

Total

$58K

In the Budget version, the organisers do the minimum job required to get the program started, but no continuous support to AISC teams during their projects and no time for evaluations and improvement for future versions of the program.

Salaries are calculated based on $7K per person per month.

Virtual AISC - Normal version

Software etc

$2K

Org...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Balsa Update and General Thank You, published by Zvi on December 13, 2023 on LessWrong.

Wow, what a year it has been. Things keep getting crazier.

Thank you for taking this journey with me. I hope I have helped you keep pace, and that you have been able to discern for yourself the parts of this avalanche of words and events that were helpful. I hope to have helped things make somewhat more sense.

And I hope many of you have taken that information, and used it not only to be able to check Twitter less, but also to make better decisions, and, hopefully, to help make the world a better place - one in which humanity is more likely to survive.

Recently, my coverage of the Biden administration executive order and the events at OpenAI have been received very positively. I'd like to do more in that mold: more focused, shorter pieces that pull the story together, hopefully de-emphasizing more ephemeral weekly posts over time. I am also happy that this work has potentially opened doors that might grant me larger platforms and other ways to make a difference.

If you feel it would make the world better to do so, please help spread the word to others who would find my work useful.

Thank you especially to both my long-time and recent paid subscribers and my Patreon supporters. It is important to me that all my content remain freely accessible - so please do not subscribe if it would be a hardship - but subscriptions and other contributions are highly motivating and allow me to increase my budget.

You can also help by contributing to my 501c(3), Balsa Research.

The rest of this post is an update on what is happening there.

Balsa First Targets the Jones Act

Even with the craziness that is AI, it is important not to lose sight of everything else going on and to seek out opportunities to create a better, saner world. That means building a world that's better equipped to handle AI's challenges and one that knows it can do sensible things.

Previously I shared both an initial announcement and Balsa FAQ. Since then, we've focused on identifying particularly low-hanging fruit where key bridging work is not being done. I've hired Jennifer Chen, who has set up the necessary legal and logistical infrastructure for us to begin work. We've had a lot of conversations with volunteers, considered many options and game plans, and are ready to begin work in earnest.

As our first major project, we've decided that Balsa will work to repeal the Jones Act. That is a big swing, and we are small. We feel that the current approaches in Jones Act reform are flawed and that there's an opportunity here to really move the needle (but, if it turns out we're wrong, we will pivot).

Our plan continues to be to lay the necessary groundwork for a future push.

We'll prioritize identifying the right questions to ask and commissioning credible academic work to find and quantify those answers. The questions that matter are often going to be the ones that are going to matter in a Congressional staff meeting or hearing, breaking down questions that particular constituencies and members care about.

I believe that the numbers will show both big wins and few net losers from repeal - including few losers among the dedicated interest groups that are fighting for the Jones Act tooth and nail - such that full compensation for those losers would be practical. We also think that the framing and understanding of the questions involved can be dramatically improved. The hope is that the core opposition, which comes largely from unions, can ultimately be brought into a win-win deal.

This is somewhat of a narrowing of the mission. The intended tech arm of Balsa did not end up happening. We will not attempt to influence elections or support candidates. We do not anticipate having the resources for the complete stack, although, if we get...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Funding case: AI Safety Camp, published by Remmelt on December 12, 2023 on LessWrong.

Project summary

AI Safety Camp is a program with a 5-year track record of enabling people to find careers in AI Safety.

We support up-and-coming researchers outside the Bay Area and London hubs.

We are out of funding. To make the 10th edition happen, fund our stipends and salaries.

What are this project's goals and how will you achieve them?

AI Safety Camp is a program for inquiring how to work on ensuring future AI is

safe, and try concretely working on that in a team.

For the 9th edition of AI Safety Camp we opened applications for

29 projects.

We are first to host a special area to support "Pause AI" work. With funding, we can scale from 4 projects for restricting corporate-AI development to 15 projects next edition.

We are excited about our new research lead

format, since it combines:

Hands-on guidance: We guide research leads (RLs) to carefully consider and scope their project. Research leads in turn onboard teammates and guide their teammates through the process of doing new research.

Streamlined applications: Team applications were the most time-intensive portion of running AI Safety Camp. Reviewers were often unsure how to evaluate an applicant's fit for a project that required specific skills and understandings. RLs usually have a clear sense of who they would want to work with for three months. So we instead guide RLs to prepare project-specific questions and interview their potential teammates.

Resource-efficiency: We are not competing with other programs for scarce mentor time. Instead, we prospect for thoughtful research leads who at some point could become well-recognized researchers. The virtual format also cuts on overhead - instead of sinking funds into venues and plane tickets, the money goes directly to funding people to focus on their work in AI safety.

Flexible hours: Participants can work remotely from their timezone alongside their degree or day job - to test their fit for an AI Safety career.

How will this funding be used?

We are fundraising to pay for:

Salaries for the organisers for the current AISC

Funding future camps (see budget section)

Whether we run the tenth edition, or put AISC indefinitely on hold depends on your donation.

Last June, we had to freeze a year's worth of salary for three staff. Our ops coordinator had to leave, and Linda and Remmelt decided to run one more edition as volunteers.

AISC has previously gotten grants paid for by FTX money. After the FTX collapse, we froze $255K in funds to cover clawback claims. For the current AISC, we have $99K left from SFF that was earmarked for stipends - but nothing for salaries, and nothing for future AISCs.

If we have enough money we might also restart the in-person version of AISC. This decision will also depend on an ongoing external evaluation of AISC, which among other things, is evaluating the difference in impact of the virtual vs in-person AISCs.

By default we'll decide what to prioritise with the funding we get. But if you want to have a say, we can discuss that. We can earmark your money for whatever you want.

Potential budgets for various versions of AISC

These are example budgets for different possible versions of the virtual AISC. If our funding lands somewhere in between, we'll do something in between.

Virtual AISC - Budget version

Software etc

$2K

Organiser salaries, 2 ppl, 4 months

$56K

Stipends for participants

$0

Total

$58K

In the Budget version, the organisers do the minimum job required to get the program started, but no continuous support to AISC teams during their projects and no time for evaluations and improvement for future versions of the program.

Salaries are calculated based on $7K per person per month.

Virtual AISC - Normal version

Software etc

$2K

Organiser salaries, 3 ...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: OpenAI: Leaks Confirm the Story, published by Zvi on December 12, 2023 on LessWrong.

Previously: OpenAI: Altman Returns, OpenAI: The Battle of the Board, OpenAI: Facts from a Weekend, additional coverage in AI#41.

We have new stories from The New York Times, from Time, from the Washington Post and from Business Insider.

All paint a picture consistent with the central story told in OpenAI: The Battle of the Board. They confirm key facts, especially Altman's attempted removal of Toner from the board via deception. We also confirm that Altman promised to help with the transition when he was first fired, so we have at least one very clear cut case of Altman saying that which was not.

Much uncertainty remains, especially about the future, but past events are increasingly clear.

The stories also provide additional color and key details. This post is for those who want that, and to figure out what to think in light of the new details.

The most important new details are that NYT says that the board proposed and was gung ho on Brad Taylor, and says D'Angelo suggested Summers and grilled Summers together with Altman before they both agreed to him as the third board member. And that the new board is remaining quiet while it investigates, echoing the old board, and in defiance of the Altman camp and its wish to quickly clear his name.

The New York Times Covers Events

The New York Times finally gives its take on what happened, by Tripp Mickle, Mike Isaac, Karen Weise and the infamous Cade Metz (so treat all claims accordingly).

As with other mainstream news stories, the framing is that Sam Altman won, and this shows the tech elite and big money are ultimately in charge. I do not see that as an accurate description what happened or its implications, yet both the tech elite and its media opponents want it to be true and are trying to make it true through the magician's trick of saying that it is true, because often power resides where people believe it resides.

I know that at least one author did read my explanations of events, and also I talked to a Times reporter not on the byline to help make everything clear, so they don't have the excuse that no one told them. Didn't ultimately matter.

Paul Graham is quoted as saying Altman is drawn to power more than money, as an explanation for why Altman would work on something that does not make him richer. I believe Graham on this, but also I think there are at least three damn good other reasons to do it, making the decision overdetermined.

If Altman wants to improve his own lived experience and those of his friends and loved ones, building safe AGI, or ensuring no one else builds unsafe AGI, is the most important thing for him to do. Altman already has all the money he will ever need for personal purposes, more would not much improve his life. His only option is to instead enrich the world, and ensure humanity flourishes and also doesn't die.

Indeed, notice the rest of his portfolio includes a lot of things like fusion power and transformational medical progress. Even if Altman only cares about himself, these are the things that make his life better - by making everyone's life better.

Power and fame and prestige beget money. Altman does not have relevant amounts of equity in OpenAI, but he has used his position to raise money, to get good deal flow, and in general to be where the money resides. If Altman decided what he cared about was cash, he could easily turn this into cash. To be clear, I do not at all begrudge in general. I am merely not a fan of some particular projects, like 'build a chip factory in the UAE.'

AGI is the sweetest, most interesting, most exciting challenge in the world. Also the most important. If you thought your contribution would increase the chance things went well, why would you want to be working on anything ...