Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Disagreement with bio anchors that lead to shorter timelines, published by Marius Hobbhahn on November 16, 2022 on The AI Alignment Forum.

This would have been a submission to the FTX AI worldview prize. I’d like to thank Daniel Kokotajlo, Ege Erdil, Tamay Besiroglu, Jaime Sevilla, Anson Ho, Keith Wynroe, Pablo Villalobos and Simon Grimm for feedback and discussions. Criticism and feedback are welcome. This post represents my personal views.

The causal story for this post was: I first collected my disagreements with the bio anchors report and adapted the model. This then led to shorter timelines. I did NOT only collect disagreements that lead to shorter timelines. If my disagreements would have led to longer timelines, this post would argue for longer timelines.

I think the bio anchors report (the one from 2020, not Ajeya’s personal updates) puts too little weight on short timelines. I also think that there are a lot of plausible arguments for short timelines that are not well-documented or at least not part of a public model. The bio anchors approach is obviously only one possible way to think about timelines but it is currently the canonical model that many people refer to. I, therefore, think of the following post as “if bio anchors influence your timelines, then you should really consider these arguments and, as a consequence, put more weight on short timelines if you agree with them”. I think there are important considerations that are hard to model with bio anchors and therefore also added my personal timelines in the table below for reference.

My best guess bio anchors adaption suggests a median estimate for the availability of compute to train TAI of 2036 (10th percentile: 2025, 75th percentile: 2052). Note that this is not the same as predicting the widespread deployment of AI. Furthermore, I think that the time “when AI has the potential to be dangerous” is earlier than my estimate of TAI because I think that this poses a lower requirement than the potential to be economically transformative (so even though the median estimate for TAI is 2036, I wouldn’t be that surprised if, let’s say 2033 AIs, could deal some severe societal harm, e.g. > $100B in economic damage).

You can find all material related to this piece including the colab notebook, the spreadsheets and the long version in this google folder.

Executive summary

I think some of the assumptions in the bio anchors report are not accurate. These disagreements still apply to Ajeya’s personal updates on timelines. In this post, I want to lay out my disagreements and provide a modified alternative model that includes my best guesses.

Important: To model the probability of transformative AI in a given year, the bio anchors report uses the availability of compute (e.g. see this summary). This means that the bio anchors approach is NOT a prediction for when this AI has been trained and rolled out or when the economy has been transformed by such an ML model, it merely predicts when such a model could be trained. I think it could take multiple (I guess 0-4) years until such a model is engineered, trained and actually has a transformative economic impact.

My disagreements

You can find the long version of all of the disagreements in this google doc, the following is just a summary.

I think the baseline for human anchors is too high since humans were “trained” in very inefficient ways compared to NNs. For example, I expect humans to need less compute and smaller brains if we were able to learn on more data or use parallelization. Besides compute efficiency, there are further constraints on humans such as energy use, that don’t apply to ML systems. To compensate for the data constraint, I expect human brains to be bigger than they would need to be without them. The energy constraint could imply that human brains a...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Eirik Mofoss named "Young leader of the year" in Norway, published by Jorgen Ljones on November 16, 2022 on The Effective Altruism Forum.

Written by Jørgen Ljønes, managing director of Gi Effektivt and co-founder of the Norwegian effective altruism movement.

Summary

Eirik Mofoss was recently named “Young leader of the year” by the largest business newspaper in Norway. He is the co-founder of the Norwegian effective altruism movement and—as a member of Giving What We Can—donates 20 percent of his income to organisations that, through documented effectiveness, make a positive difference in the world. The award provided lots of valuable attention and traffic to EA Norway and Gi Effektivt, our donation plattform. We wanted to share and celebrate this, not only as a recognition of Eirik, but also for all the Norwegian EAs who has made this possible.

Background

Dagens Næringsliv (Norwegian for "Today's Business"), commonly known as DN, is the third-largest daily newspaper in Norway. Every year, DN nominates 30 people under the age of 30 who, through good ideas, entrepreneurial or leadership skills, represent guiding figures for others, and want to make the world more sustainable. First, DN’s readership nominates a selection of people, then an independent jury picks the final 30, based on the UN's 17 sustainable development goals (SDGs). The jury, together with readers, then selects a single winner, among people whose accomplishments have yielded some kind of measurable impact in the realm of sustainable development.

The nomination of Eirik as a candidate for the award

Eirik Mofoss co-founded the first Norwegian branch of the Effective Altruism movement at NTNU in 2014, and has remained true to his ideals ever since. Eirik has committed to donating 20 percent of his income for the rest of his life, and has—through his nomination—imparted his mode of working and living on a readership of over 150 000 people. He has also widely publicised his altruistic ideals through his earlier work, as adviser to both the Conservative Party in Parliament, as well as adviser to Norad (Norwegian Agency for Development Cooperation). According to the jury:

Eirik Mofoss examines our fundamental assumptions about what aid currently is, and our vision for what it could be; how one can—and should—contribute, as a private citizen.

DN’s first interview with Eirik Mofoss quickly became their most read “30 under 30”-interview ever, and the newspaper’s most read piece in 2022.

Eirik Mofoss is 29 years old, born in Tromsø, in the north of Norway, and has a master's degree in industrial economics and technology management from NTNU. Mofoss has previously worked as a political and financial adviser for the Conservative Party of Norway, and as a business development manager for Visma. Almost ten years ago, he became a member of Giving What We Can, and committed to donating 20 percent of his income for the rest of his life. Mofoss co-founded the first effective altruism group in Norway in 2014 while studying at NTNU, and has later co-founded the foundation Gi Effektivt that fundraises for charities recommended by GiveWell.

The jury’s reasoning for naming Eirik "Young leader of the year"

Eirik Mofoss shows us that our direct actions still yield impactful results during a time when many feel increasingly powerless. Mofoss builds bridges, and has skillfully integrated his personal and working lives. He seeks, through his activism and modes of thinking, to elevate the discourse around sustainability—away from individualistic critiques, and toward systemic ones.

Mofoss’ generation has been raised in the global labour aristocracy. Despite this, some of them have recognized the extent to which their privileges and positions can be used for good. Mofoss challenges how we as individuals can contribute, and works to ...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Cruxes for nuclear risk reduction efforts - A proposal, published by Sarah Weiler on November 16, 2022 on The Effective Altruism Forum.

This is my attempt to give an overview of debates and arguments relevant to the question of how to mitigate nuclear risks effectively. I list a number of crucial questions that I think need to be answered by anyone (individual or group) seeking to find their role as a contributor to nuclear risk mitigation efforts. I give a high-level overview of the cruxes in Figure 1:

These questions are based on my moderately extensive engagement with the nuclear risk field; they are likely not exhaustive and might well be phrased in a less-than-optimal way — I thus welcome any feedback for how to improve the list found below. I hope that this list can help people (and groups) reflect on the cause of nuclear risk reduction by highlighting relevant considerations and structuring the large amount of thinking that has gone into the topic already. I do not provide definitive answers to the questions listed, but try to outline competing responses to each question and flesh out my own current position on some of them in separate posts/write-ups (linked to below).

The post consists of the following sections:

Setting the stage: some background on my CERI research project

Outline of my work on nuclear issues prior to the summer fellowship

Summary of work by others with some similarity to mine

A defense of the value of my project and the output presented here

Main body: list of cruxes in the nuclear risk debate

Substantive cruxes: questions to determine which nuclear risks to work on and how to do so

Sub-cruxes: questions to help tackle the cruxes above

Meta-level cruxes: methodological and epistemological questions

Links and references

Setting the stage

For a couple of months, I have been engaged in an effort to disentangle the nuclear risk cause area, i.e., to figure out which specific risks it encompasses and to get a sense for what can and should be done about these risks. I took several stabs at the problem, and this is my latest attempt to make progress on this disentanglement goal.

My previous attempts to disentangle nuclear risk

While I had some exposure to nuclear affairs during my studies of global politics at uni (i.e., at least since 2018) and have been reading about the topic throughout the last few years, I’ve been engaging with the topic more seriously only since the beginning of this year (2022), when I did a part-time research fellowship in which I decided to focus on nuclear risks.

For that fellowship, I started by brainstorming my thoughts and uncertainties about nuclear risk as a problem area that I might want to work on (resulting in a list of questions and my preliminary thoughts on them), did a limited survey of the academic literature on different intellectual approaches to the topic of nuclear weapons, and conducted a small-scale empirical investigation into how three different non-profits in the nuclear risk field (the Nuclear Threat Initiative, the International Campaign to Abolish Nuclear Weapons, and the RAND Corporation) conceptualize and justify their work on nuclear risk (resulting in a sketch of the theory of change. of each organization, constructed based on the information they provide on their websites).

During ten weeks over this summer (Jul-Sep 2022), my participation in the Cambridge Existential Risk Initiative — a full-time, paid and mentored research fellowship — has allowed me to dedicate more time to this project and to test out a few more approaches to understanding the nuclear risk field. I spent around three weeks with the goal of compiling a list of organizations working on nuclear risk issues, collecting information on their self-described theory of change, and categorizing the organizations in a broad typolog...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Current themes in mechanistic interpretability research, published by Lee Sharkey on November 16, 2022 on The AI Alignment Forum.

This post gives an overview of discussions - from the perspective and understanding of the interpretability team at Conjecture - between mechanistic interpretability researchers from various organizations including Conjecture, Anthropic, Redwood Research, OpenAI, and DeepMind as well as some independent researchers. It is not a review of past work, nor a research agenda. We're thankful for comments and contributions from Neel Nanda, Tristan Hume, Chris Olah, Ryan Greenblatt, William Saunders, and other anonymous contributors to this post, which greatly improved its quality. While the post is a summary of discussions with many researchers and received comments and contributions from several, it may nevertheless not accurately represent their views.

The last two to three years have seen a surge in interest in mechanistic interpretability as a potential path to AGI safety. Now there are no fewer than five organizations working on the topic (Anthropic, Conjecture, DeepMind, OpenAI, Redwood Research) in addition to numerous academic and independent researchers.

In discussions about mechanistic interpretability between a subset of researchers, several themes emerged. By summarizing these themes here, we hope to facilitate research in the field more broadly.

We identify groups of themes that concern:

Object-level research topics in mechanistic interpretability

Research practices and tools in mechanistic interpretability

Field building and research coordination in mechanistic interpretability

Theories of impact for mechanistic interpretability

Object-level research topics in mechanistic interpretability

Solving superposition

Anthropic’s recent article on Toy Model of Superposition laid out a compelling case that superposition is a real phenomenon in neural networks. Superposition appears to be one of the reasons that polysemanticity happens, which makes mechanistic interpretability very difficult because it prevents us from telling simple stories about how features in one layer are constructed from features in previous layers.

A solution to superposition will look like the ability to enumerate all the features that a network represents, even if they’re represented in superposition. If we can do that, then we should be able to make statements like “For all features in the neural network, none violate rule X” (and more ambitiously, for "no features with property X participate in circuits which violate property Y"). Researchers at Anthropic hope this might enable ‘enumerative safety’, which might allow checking random samples or comprehensive investigations of safety-critical parts of the model for unexpected and concerning components. There are many potential reasons researchers could fail to achieve enumerative safety, including failing to solve superposition, scalability challenges, and several other barriers described in the next section.

Anthropic outlined several potential solutions to superposition in their article. Very briefly, these strategies are:

Create models without superposition.

Find a sparse overcomplete basis that describes how features are represented in models with superposition. This will likely involve large scale solutions to sparse coding.

Hybrid approaches in which one changes models, not resolving superposition, but making it easier for a second stage of analysis to find a sparse overcomplete basis that describes it.

Multiple organizations are pursuing these strategies. Researchers in all organizations are keen to hear from people interested in working together on this problem. However, there is a range of views among researchers on how central superposition is as a problem and how tractable it is.

Barriers beyond superposit...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Unpacking "Shard Theory" as Hunch, Question, Theory, and Insight, published by Jacy Reese Anthis on November 16, 2022 on The AI Alignment Forum.

I read several shard theory posts and found the details interesting, but I couldn't quite see the big picture. I'm used to hearing "theory" refer to a falsifiable generalization of data. It is typically stated as a sentence, paragraph, list, mathematical expression, or diagram early in a scientific paper. Theories range from extremely precise like Newton's three laws of motion to extremely broad like Frankish's consciousness illusionism (i.e., consciousness is an illusion).

I have also been generally confused about what it means to "solve alignment" before AGI arrives given that there is not (yet) consensus around any pre-AGI formalization of the alignment problem itself: Wouldn't any proposed solution still have a significant number of people (say, >10% of Alignment Forum users) who think that it doesn't even pose the problem the right way? What should our "theories" even be aiming for?

With help from Nathan Helm-Burger, I think I now better understand what's referred to as shard theory and want to share my understanding as an exercise in problem and solution formulation in alignment. I think "shard theory" refers to four sequential components: a Shard Hunch that motivates a two-part Shard Question, the first part of which is currently being answered by the gradual development of an actual Shard Theory of human values, which hopefully provides answers to the second part with Shard Insight that can be implemented in AI systems to facilitate alignment. Namely:

Shard Hunch: A human brain is a general intelligences, and its intentions and behavior are reasonably aligned with its many shards of values (i.e., little bits of "contextual influences on decision-making"). Maybe something like that alignment can work for AI too!

Shard Question: How does the human brain ensure alignment with its values, and how can we use that information to ensure the alignment of an AI with its designers' values?

Shard Theory: The brain ensures alignment with its values by doing A, B, C, etc.

Shard Insight: We can ensure the alignment of an AI with its designers' values by doing X, Y, Z, etc. mapped from shard theory.

This is exciting! Now when I read shard theory research, I feel like I properly understand it as gradually filling in A, B, C, X, Y, Z, etc. For example, Assumptions 1, 2, and 3 in "The Shard Theory of Human Values" are examples of A, B, and C, and the two arguments in "Reward is Not the Optimization Target" and "Human Values & Biases are Inaccessible to the Genome" are examples of X and Y. I also think the specific idea of a "shard" is less central to these claims than I thought; it seems the first of those posts could parsimoniously replace "shard" with "value" (in the dictionary sense) with very little meaning lost, and the latter two posts don't even use the word. I wonder if something like "Brain-Inspired Alignment" would be a clearer label, at least until a central concept like shards emerges in the research.

Shard research is also at a very early stage, so it is inevitably less focused on stating and validating the falsifiable, non-trivial claims that could be an actual shard theory (which is usually what we discuss in science) and instead seems to mostly be developing a language for eventually specifying shard theory—much like how Rubin's potential outcomes (POs) and Pearl's directed acyclic graphs (DAGs) were important developments in causality research because they allowed for the clear statement of falsifiable, nontrivial causal theories. Pope and Turner also use the terms "paradigm" and "frame," which I think are more fitting for what they have done so far than shard "theory" per se though less specific than "language."

For ...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: User-Controlled Algorithmic Feeds, published by jefftk on November 12, 2022 on LessWrong.

I've been

playing with

Mastodon more, and the main way you see what other people have posted is by reading your feed. This consists of everything any of the people you follow has posted or boosted. If you're always going to read everything that's come out since your last visit, which is the strategy I use for email, then order doesn't matter very much. But for social media a strategy of reading everything means following very few people, spending more time on the site than you probably want to, or falling behind. What would be better?

The way mainstream social networks (Facebook, Twitter, etc) handle this is with prioritization: run an algorithm that considers every post it could potentially show you and picks the best according to some inscrutable and highly proprietary ranking, optimized for a combination of engagement (getting you to post/comment/like), retention (getting you to keep using the site), monetization (getting you to view and interact with ads), and other things. It's common to complain about the algorithm, especially when it fails to show you things you really would have wanted to see or when what it's optimizing for is clearly not what's best for you.

Mastodon's approach is very different: its prioritization algorithm is

"newest first". This is the ideal algorithm if what matters most is breaking news, but since that's not most of what I care about it has some downsides:

I'd rather see Pat's only post of the week than Sam's 100th.

I'm more interested in the things the people I follow post than the things they boost.

I want to read everything Alex posts because it's relevant to my work, consistently interesting, or they're someone I'm close to. But if this were my bar for following people I'd miss a lot of things by people who I do still often like reading.

If I only have a short amount of time available, all things being equal I'd rather a Jo post than a Rowan one, but this is not a strong preference.

The problem with a traditional social network's algorithmic feed algorithm isn't that it uses an algorithm but that (a) it's goals are not your goals and (b) you can't tweak it. Imagine a feed prioritized according to:

Every post gets assigned a number, and are shown highest to lowest.

Once a post has been shown to you it's removed from the pool of eligible posts.

You control the rules that assign numbers to posts.

Rules include assigning points for recency, how rare it is for this person to post, how often you favorite this person's posts, whether the post is a boost, hashtags, etc.

You have a modifier for each person you follow and there's a button or menu option on each post for bumping that person's priority up or down.

Starting off with recency as the only active contributor to score and then letting people click "show more / show less" would be a good place to start, and fully backwards compatible: if you don't use the feature your feed is unchanged.

(For more discussion along these lines see Harris's post and Issue

#3782.)

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: [Linkpost] Sam Harris on "The Fall of Sam Bankman-Fried", published by michel on November 16, 2022 on The Effective Altruism Forum.

Sam Harris just shared a 20 min podcast with his thoughts on the FTX crash, Sam Bankman-Fried (SBF), and effective altruism. I think he had a good take, and I'm glad he shared it.

Highlights (written retrospectively; hope I'm not misrepresenting):

Sam Harris discusses how he didn't expect what looks like serious wrongdoing – and likely fraud – from SBF, who he had glorified on his podcast in December 2021 for an episode on Earning to Give. Even listening to the podcast back again, Sam Harris didn't detect any deviousness in SBF.

He acknowledges there's a lot we don't know yet about SBF's character and how long this has been going on.

Sam mentions that he's steered clear of the EA community because he has always found it "cult-like"

But he defends EA principles and says that this whole situation means "exactly zero" for his commitment to those ideas.

He compares criticizing EA ideas in light of FTX to criticizing the scientific method in light of elite scientists faking results.

He extends his defense to utilitarianism and consequentialism: if you're critiquing consequentialism ethics on the basis that it led to bad consequences you're "seriously confused."

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Internal communication framework, published by rosehadshar on November 15, 2022 on LessWrong.

Usually when we think about humans and agency, we focus on the layer where one human is one agent.

We can also shift focus onto other layers and treat those entities as agents instead - for example, corporations or nations.

Or, we can take different parts within a human mind as agenty -- which is a lens we explore in this post.

In this post, we first introduce a general, pretty minimalist framework for thinking about the relationship between different parts of the mind (Internal Communication Framework, or ICF). Based on this framework, we build a basic technique that can be used to leverage ICF, and which we believe can be helpful for things like:

Resolving inner conflicts

Building internal alignment

Increasing introspective access

We have shared ICF with a small number of people (~60-80) over a few years, and are writing this post partly to create a written reference for ICF, and partly to make the framework accessible to more people.

The framework

Some background context:

ICF is rooted in thinking about hierarchical agency (i.e. how agency, and strategic dynamics, play out composite agents and their parts).

It is mostly an attempt to generalise from a class of therapy schools and introspection techniques working with parts of the mind, and to make the framework cleaner from a theoretical perspective. (More on the relationship between ICF and some particular techniques from that class here

ICF was developed in 2018 by Jan Kulveit with help from Nora Amman, in the context of the Epistea Summer Experiment and then CFAR instructorship training.

ICF makes two core assumptions:

The human mind is made of parts

Interactions between parts, and between parts and the whole, work best when the interactions are cooperative and kind

There is a possibly interesting discussion about the epistemic status of these assumptions, which is not the point of this post, so we invite you to interpret them in whatever spirit you like - as a model which is somewhat true, a model which is wrong but useful, a metaphor for processing your experience.

The human mind is made of parts

These parts have different goals and models of the world, and don't always agree. They are still all part of your mind.

If we make this assumption that the mind is made of parts, it becomes less obvious what we mean by terms like ‘you’ or ‘the self’. One metaphor for thinking about this is to view the whole set of parts as something like a ‘council’, which collectively has control over something like a ‘speaker’, which is the voice creating your internal stream of words, the appearance of a person, your sense of ‘self’...

Often, the whole is the most intuitive and helpful level of abstraction, and ‘you’ will make perfect sense. Sometimes, for example when experiencing inner conflict, ‘you’ will be confusing, and it will be more productive to work at the level of the parts instead.

Interactions between parts, and between parts and the whole, work best when the interactions are cooperative and kind

By cooperation, we mean something like choosing ‘cooperate’ in various games, in a game theoretic sense.

While game theory has formalised the notion of cooperation between agents at the same level of abstraction, we don’t have a similar formal model for positive interactions between a whole and its parts. We will refer to these interactions as kindness.

You may disagree with this assumption in the form of a normative claim. Descriptively, we think that cooperative and kind relations between parts, and between parts and the whole, tend to lead to more constructive interactions, and longer-term make people happier, more aligned and more agentic.

A few overall notes:

Different layers of agency can compete for power - e.g. for space to ...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Open Phil is seeking applications from grantees impacted by recent events, published by Bastian Stern on November 16, 2022 on The Effective Altruism Forum.

We (Open Phil) are seeking applications from grantees affected by the recent collapse of the FTX Future Fund (FTXFF) who fall within our long-termist focus areas (biosecurity, AI risk, and building the long-termist EA community). If you fit the following description, please fill out this application form.

We’re open to applications from:

Grantees who never received some or all of their committed funds from FTXFF.

Grantees who received funds, but want to set them aside to return to creditors or depositors.

We think there could be a number of complex considerations here, and we don’t yet have a clear picture of how we’ll treat requests like these. We’d encourage people to apply if in doubt, but to avoid making assumptions about whether you’ll be funded (and about what our take will end up being on what the right thing to do is for your case). (Additionally, we’re unsure if there will be legal barriers to returning funds.) That said, we’ll do our best to respond to urgent requests quickly, so you have clarity as soon as possible.

Grantees whose funding was otherwise affected by recent events.

Please note that this form does not guarantee funding. We intend to evaluate applications using the same standard as we would if they were coming through one of our other longtermist programs — we will evaluate whether they are a cost-effective way to positively influence the long-term future. As described in Holden’s post, we expect our cost-effectiveness “bar” to rise relative to what it has been in the past, so unfortunately we expect that some of the applications we receive (and possibly a sizeable fraction of them) will not be successful. That said, this is a sudden disruptive event and we plan to take into account the benefits of stability and continuity in our assessment.

We’ll prioritize getting back to applicants who indicate time sensitivity and whose work seems highly likely to fall above our updated bar. If we’re unsure whether an application is above our new bar, we’ll do our best to get back within the indicated deadline (or within 6 weeks, if the application isn’t time-sensitive), but we may take longer as we reevaluate where the bar should be.

We’re aware that others may want to help out financially. If you would like to identify yourself as a potential donor either to this effort or to a different one aimed at impacted FTXFF grantees, you can get in contact with us at [email protected]. We sincerely appreciate anyone who wants to help, but due to logistical reasons we can only respond to emails from people who think there’s a serious chance they’d be willing to contribute over $250k.

We’ve left an option on the form to explain specific circumstances – we can imagine many ways that recent events could be disruptive. (For example, if FTXFF had committed to funding a grantee that planned to regrant some of those funds, anyone anticipating a regrant could be affected.)

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: I think EA will make it through stronger, published by lincolnq on November 16, 2022 on The Effective Altruism Forum.

status: kind of rambly but I wanted to get this out there in case it helps

This week's events triggered in me some soul-searching, wondering whether effective altruism even makes sense as a coherent thing anymore.

The reason I thought EA might break up or dissolve was something like: EA mostly attracted naive maximizer-types ("do the Most Good, using reasoning"), but now it's obvious that the idea of maximizing goodness doesn't work in practice--we have a really clear example of where trying to do that fails (SBF if you attribute pure motives to him); as well as a lot of recent quotes from EA luminaries saying that you shouldn't do that. I didn't see what else holds us together besides the maximizing thing.

But I was kind of ignoring the reasoning thing! I thought about it, and I think that we can make minimal changes: The framing I like is "Do good, using REASONING". With capital letters :)

I think deleting "the most" is a change we should have made a long time ago; few of the important people in EA were claiming that they were doing the most good anyway. And EA at its core is about reasoning: reasoning carefully, using evidence; thinking about first-order and second-order effects; comparing options in front of you; argument and debate. The simpler phrasing of this new mission is intended to make reasoning stand out.

If this direction is adopted, I have the following hopes:

that EA will become a "bigger tent," accepting of more types of people doing more types of good things in the world and reasoning about them. e.g., we'll welcome anyone who is trying to do good, and is open to talking through the 'why' behind what they are doing

that naive utilitarian maximizers will go away or be a bit more humble :)

that people will put more emphasis on developing and relying on their own reasoning processes, and rely less on the reasoning of others to make big decisions in their lives.

that cause prioritization will get less emphasis, especially career cause prioritization (I think the maximizing thingy regularly causes people to make bad career decisions)

(Some color on the final one: I've had a blog post brewing for a long time against strong career cause prio but haven't really managed to write it up in a convincing way. e.g., I think AI is a bad career direction for a lot of people, but young EAs are convinced to try it anyway because AI is held up as the priority path and they'll have so much more impact if they make it. This seems bad for lots of reasons which I will try to write up in a post if I can ever figure out how to articulate them.)

Anyway, I think the above hopes, if they pan out, will make the community stronger. And, though I am normally loath to argue about optics, I do think this change would counter most of the arguments that you regularly see in news media against EA principles (such as that EA is about dangerous maximizing, or that it's only for elites, or that young people's careers are affected in unstable/chaotic ways when they encounter EA).

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.