Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Binding Fuzzies and Utilons Together, published by eleni on December 4, 2022 on The Effective Altruism Forum.

"A single death is a tragedy; a million deaths is a statistic."

Attributed to Joseph Stalin.

(Epistemic status: speculative)

In this post, we argue that producing more emotionally appealing materials related to EA cause areas could yield some beneficial results. We present some possible research designs to assess the empirical validity of our claims, and call for EAs with a background in psychology to lead this research project.

Introduction

In Purchase Fuzzies and Utilons Separately, Yudkowsky argues that, if we want maximize our impact, we should be explicit about which of our "good deeds" we do with the intention of really helping the world to become a better place, and which are done in order for us to feel good about ourselves. By clearly separating these two types of actions, the argument goes, we could have both a larger positive impact on the world and get to feel even better about ourselves. We think that this is very valuable advice at the individual level. However, when considering the strategic approach the EA movement as a whole should have, we believe that merging warm fuzzies and utilons together, to some extent, might possibly be a good idea.

Some ideas in the Effective Altruism movement are a deliberate attempt to correct a market failure in the "market" of doing good. Some cause areas and interventions, due to a wide range of psychological biases, are less emotionally appealing than others, which means that, when left unchecked, they would receive less resources than what would be optimal, and the opposite holds for the more emotionally appealing causes. The approach EA usually seems to take is to acknowledge this phenomenon ("neglectedness" in the ITN framework) and try to reallocate resources to where they would be the most efficient, without directly taking into account the emotional appeal of the cause. In this post, we argue that a further step in correcting this failure could be to explicitly targeting emotional appeal when designing the marketing of these neglected cause areas, in a way that would attenuate, in part, the very characteristics that make the cause neglected on the first place.

We hypothesize that while trying to focus explicitly on improving the emotional appeal of EA cause areas could make the movement easier to promote to non-EAs, a significant part of the benefits of this strategy could come from the effect of increased engagement of those who are already EAs. If this hypothesis is valid, increasing the emotional appeal of marketing materials in EA organizations could prove to be a cost-effective intervention.

In this post, we will develop this general idea in more detail. We will specify what kinds of interventions would fit what we have in mind, and we will present some potential upsides that these interventions might have. We have a simple idea in mind that should be empirically verified before being put into practice, and so we also tentatively present some research ideas that EAs with the necessary skills could pursue. We will also present some objections to our ideas and points of uncertainty as an important factor for our belief on the importance on further research. Finally, neither of us have a background in psychology, so take what is written here with a grain of salt.

Binding Fuzzies and Utilons

The hypothesis to be presented in this article is the following: explicitly focusing on increasing the emotional appeal in the marketing of EA organizations could prove to be a powerful tool to help move more resources towards these areas. As we will discuss in the next section, these could be both monetary and human resources, and we believe this effect could hold both for non-EAs and for EAs.

A striking anecdotal ...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Our 2022 Giving, published by Jeff Kaufman on December 4, 2022 on The Effective Altruism Forum.

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Summary of a new study on out-group hate (and how to fix it), published by AllAmericanBreakfast on December 4, 2022 on LessWrong.

This is a summary of the 2022 paper Individuals prefer to harm their own group rather than help an opposing group. I spent about an hour reading the study and writing the post as I went. If I made mistakes in my interpretation, please let me know in the comments.

Abstract excerpt:

Individuals prefer to harm their own group rather than provide even minimal support to an opposing group across polarized issues (abortion access, political party, gun rights).

Individuals preferred to subtract more than three times as much from their own group rather than support an opposing group, despite believing that their in-group is more effective with funds.

Identity concerns drive preferences in group decision-making

Individuals believe that supporting an opposing group is less value-compatible than harming their own group.

Summary

Let’s say you’re a highly partisan voter. Maybe you’re a Democrat, with a strong dislike of Republican politics. Now, let’s play a game of Would You Rather. Let’s say a mustache-twiddling psychologist comes up to you and offers you a choice. On the table is a pair of checks. You look more closely. The checks are made out to the Republican Party and the Democratic Party, each for $1,000,000!

“You’ve got to choose,” says the psychologist. “Either I make both of these donations, or tear up both checks. What do you want me to do?”

You hem and haw, then think about what each party would spend the money on. Democrats are the party of the poor. A little money can go a long way. The Republicans are the party of the rich, and they get so much money that an extra $1,000,000 won’t help them. It would make sense to have the psychologist donate both checks - on balance, the extra money will probably help the Democrats more than the Republicans.

So you’re surprised when the words out of your mouth are, “tear up both checks.”

OK, that’s not the exact methodology of this study, but it gets the point across. This study showed that participants are foolishly spiteful in lose-lose situations. Here are the individual study results:

In lose-lose situations, do people irrationally focus on hurting their opponents, even at a greater cost to their own side? On both sides of the aisle, offered a choice of giving their opponents $1 or having their own side lose $1, they’ll take the loss about 75% of the time, even if they think their own side can make better use of the money. So yes, people will sacrifice a greater benefit to their own side to avoid helping the enemy.

How big is this effect? The researchers found that people would rather lose more than $3 than give $1 to their political opponents.

How much of this is an identity thing, as opposed to an attempt to be strategic? People feel like giving $1 to the opposition harms their political identity more than losing $1 for their own side. Even if they think their own side uses the money more effectively than their opponents, they’re not persuaded to change their donations on that basis.

What if we reinforce people’s partisan identities? Does that enhance the effect? Yes it does. Reinforce people’s identity as a Democrat, and they’ll be even more likely to choose to sacrifice $1 for the Democrats in order to avoid giving $1 to the Republicans, and vice versa.

What if people thought it was normal to focus on avoiding hurting their own side, even if it helps their opponents? This makes a huge difference. If people think a strong majority of their own side would rather let the other side have $1 in order to avoid losing $1 for their own side, they’ll fit in with the perceived norm.

What if nobody would find out? The researchers found that this doesn’t matter at all. People make pretty much the same choic...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Take 3: No indescribable heavenworlds., published by Charlie Steiner on December 4, 2022 on The AI Alignment Forum.

As a writing exercise, I'm writing an AI Alignment Hot Take Advent Calendar - one new hot take, written every day for 25 days. Or until I run out of hot takes.

Some people think as if there are indescribable heavenworlds. They're wrong, and this is important to AI alignment.

This is an odd accusation given that I made up the phrase "indescribable heavenworld" myself, so let me explain. It starts not with heavenworlds, but with Stuart Armstrong writing about the implausibility of indescribable hellworlds.

A hellworld, is, obviously, a bad state of the world. An indescribable hellworld is a state of the world where where everything looks fine at first, and then you look closer and everything still looks fine, and then you sit down and think about it abstractly and it still seems fine, and then you go build tools to amplify your capability to inspect the state of the world and they say it's fine, but actually, it's bad.

If the existence of such worlds sounds plausible to you, then I think you might enjoy and benefit from trying to grok the metaethics sequence.

Indescribable hellworlds are sort of like the reductio of an open question argument. Open question arguments say that no matter what standard of goodness you set, if it's a specific function of the state of the world then it's an open question whether that function is actually good or not (and therefore moral realism). For a question to really be open, it must be possible to get either answer - and indescribable hellworlds are what keep the question open even if we use the standard of all of human judgment, human cleverness, and human reflectivity.

If you read Reducing Goodhart, you can guess some things I'd say about indescribable hellworlds. There is no unique standard of "explainable," and you can have worlds that are the subject of inter-standard conflict (even supposing badness is fixed), which can sort of look like indescribable badness. But ultimately, the doubt over whether some world is bad puts a limit on how hellish it can really be, sort of like harder choices matter less. A preference that can't get translated into some influence on my choices is a weak preference indeed.

An indescribable heavenworld is of course the opposite of an indescribable hellworld. It's a world where everything looks weird and bad at first, and then you look closer and it still looks weird and bad, and you think abstractly and yadda yadda still seems bad, but actually, it's the best world ever.

Indescribable heavenworlds come up when thinking about what happens if everything goes right. "What if" - some people wonder - "the glorious post-singularity utopia is actually good in ways that are impossible for human to comprehend? That would, by definition, be great, but I worry that some people might try to stop that glorious future from happening by trying to rate futures using their present preferences / judgment / cleverness / reflectivity. Don't leave your fingerprints on the future, people!"

No indescribable heavenworlds. If a future is good, it's good for reasons that make sense to me - maybe not at first glance, but hopefully at second glance, or after some abstract thought, or with the assistance of some tools whose chain of logic makes sense to me. If the future realio trulio seems weird and bad after all that work, it's not secretly great, we probably just messed up.

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: SFF Speculation Grants as an expedited funding source, published by Andrew Critch on December 3, 2022 on The Effective Altruism Forum.

Hi everyone, SFF has received numerous emails recently from organizations interested in expedited funding. I believe a number of people here already know about SFF Speculation Grants, but since we've never actually announced our existence on the EA Forum before:

The Survival and Flourishing Fund has a means of expediting funding requests at any time of year, via applications to our Speculation Grants program:

SFF Speculation Grants are expedited grants organized by SFF outside of our biannual grant-recommendation process (the S-process). “Speculation Grantors” are volunteers with budgets to make these grants. Each Speculation Grantor’s budget grows or increases with the settlement of budget adjustments that we call “impact futures” (explained further below). Currently, we have a total of ~20 Speculation Grantors, with a combined budget of approximately $4MM. Our process and software infrastructure for funding these grants were co-designed by Andrew Critch and Oliver Habryka.

For instructions on how to apply, please visit the link above.

For general information about the Survival and Flourishing Fund, see:

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Logical induction for software engineers, published by Alex Flint on December 3, 2022 on The AI Alignment Forum.

This work was supported by the Monastic Academy for the Preservation of Life on Earth and the Long Term Future Fund.

Outline

This post is an explanation of the theory of logical induction developed by Garrabrant et al

I formulate the theory in a way that should make sense to software engineers and those with a software engineering mindset.

I will go through a full implementation of the logical induction algorithm in Python, and I will use it to explain the basic theory of logical induction, including the core logical induction algorithm.

I will give type signatures for all concepts and will work through all algorithms in terms of a sequence of processing steps.

The Python code for this guide is here.

The Colab notebook for this guide is here.

The index of type signatures for this guide is here.

Motivation

Logical induction is a theory, published in 2016 by Scott Garrabrant, Tsvi Benson-Tilsen, Andrew Critch, Nate Soares, and Jessica Taylor, about how to build machines that maintain uncertainty about the world and update those beliefs in light of evidence. It does this by assigning numbers between 0 and 1 to claims about the world, just as probability theory does, but it makes different guarantees about the internal relationship between those numbers. Whereas probability theory guarantees that its numbers will obey the sum and product rules of probability, logical induction guarantees that the evolution of its numbers over time will obey the logical induction criterion.

Many people have heard that logical induction is about having uncertainty in purely logical facts. It is true that logical induction shows how to construct algorithms that maintain uncertainty in purely logical facts, but in my view this is not really the point of logical induction. The point of logical induction, in my view, is that it is always computable, even when reasoning about contradictory, uncomputable, or self-referential questions. Its capacity to maintain uncertainty about purely logical facts is actually a by-product of the computability of logical induction.

Logical induction addresses the same basic problem that probability theory addresses. Logical induction and probability theory, therefore, are two different answers to the question: what is a reasonable formal method for quantifying uncertainty and updating it in light of evidence? Probability theory and logical induction both provide concrete operationalizations of "quantified uncertainty" (henceforth "credence"), and what it means for a set of credences to be "reasonable".

Probability theory says that credences are "reasonable" if it is impossible for someone to bet against you in a way that is expected to make money, independent of the true state of the world (a Dutch book). Logical induction says that credences are "reasonable" if it is impossible for someone to bet against you in a way that makes more and more money over time with no corresponding down-side risk. The probability theory formulation is the stronger guarantee; its drawback is that it is not in general computable. The logical induction formulation is computable, and in this guide we will walk through a general purpose algorithm for computing credences given complicated, even self-referential, world models.

At its core, the theory of logical induction consists of two things:

A set of proofs showing that if you assign credences in a way that is consistent with the logical induction operationalization of uncertainty then your credences are guaranteed to exhibit certain common-sense desirable properties such as consistency over time, unbiasedness over time, converging to well-calibrated limits in a timely manner.

An algorithm that assigns credences in a way th...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Revisiting EA's media policy, published by Arepo on December 3, 2022 on The Effective Altruism Forum.

Epistemic status: This post is meant to be a conversation starter rather than a conclusive argument. I don’t assert that any of the concerns in it are overwhelming, only that we have too quickly adopted a set of media communication practices without discussing their trade-offs.

Also, while this was in draft form, Shakeel Hashim, CEA’s new head of communications, made some positive comments on the main thesis suggesting that he agreed with a lot of my criticisms and planned to have a much more active involvement with the media. If so, this post may be largely redundant - nonetheless, it seems worth having the conversation in public.

CEA adheres to what they call the fidelity model of spreading ideas, which they formally introduced in 2017, though my sense is it was an unofficial policy well before that. In a low-fidelity nutshell, this is the claim that EA ideas are somewhat nuanced and media reporting often isn’t, and so it’s generally not worth pursuing - and often worth actively discouraging - media communication unless you’re a) extremely confident the outlet in question will report the ideas exactly as you describe them and b) you’re qualified to deal with the media.

In practice, because CEA pull many strings, this being CEA policy makes it de facto EA policy. ‘Qualified to deal with the media’ seems to mean ‘CEA-sanctioned’, and I have heard of at least one organisation being denied CEA-directed money in part because it was considered too accommodating of the media. Given that ubiquity, I think it’s worth discussing the policy more depth. We have five years of results to look back on and, to my knowledge, no further public discussion of the subject. I have four key concerns with the current approach:

It’s not grounded in research

It leads to a high proportion of negative coverage for EA

It assumes a Platonic ideal of EA

It contributes to the hero-worship/concentration of power in EA

Elaborating each.

Not empirically grounded

The article assumes that low-fidelity spreading of EA ideas is necessarily bad, but doesn’t give any data beyond some very general anecdotes to support this. There’s an obvious trade-off to be had between a small number of people doing something a lot like what we want and a larger number doing something a bit like what we want, and it’s very unclear which has higher expectation.

To see the case for the alternative, we might compare the rise of the animal rights movement in the wake of Peter Singer’s original argument for animal welfare. The former is a philosophically mutated version of the latter, so on fidelity model reasoning would have been something that’s ‘similar but different’ - apparently treated on the fidelity model as undesirable. Similarly, the emergence of reducetarianism/flexitarianism looks very like what the fidelity model would consider to be a ‘diluted’ version of the practice of veganism Singer advocated. My sense is that both of these have nonetheless been strong net positives for animal welfare.

High proportion of negative coverage

If you have influence over a group of supporters and you tell them not to communicate with the media, one result you might anticipate is that a much higher proportion of your media coverage comes from the detractors, who you don’t have influence over. Shutting out the media can also be counterproductive - they’re (mostly) human, and so tend to deal more kindly with people who deal more kindly with them. I have three supporting anecdotes, one admittedly eclipsing the others:

FTXgate

At the time of writing, if you Google ‘effective altruism news’, you still get something like this.

Similarly, if you look at Will’s Tweetstorm decrying SBF’s actions, the majority of responses are angrily negativ...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: CEA “serious incident report”?, published by Pagw on December 3, 2022 on The Effective Altruism Forum.

CNBC reported on 30/11/22 that the Centre for Effective Altruism had filed a '“serious incident report” tied to “the collapse of FTX”' with the Charity Commission for England and Wales. It says reasons for this could include 'the “loss of your charity’s money or assets” or “harm to your charity’s work or reputation”'. Here is more information about serious incident reporting.I have not seen any comment about this from CEA or anywhere else. Whilst the FTX fallout is serious, I wasn't aware of clear direct implications for CEA, so this seems like new information. Readers of the Forum may be donors to CEA and CEA is still soliciting donations through its website, so I think it's important to get a clear public picture of any problems. Clarifying that there is no problem relevant for the public would also be helpful if that's the case. Does anybody have knowledge about this that they could share?

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: MrBeast's Squid Game Tricked Me, published by lsusr on December 3, 2022 on LessWrong.

I recently watched Squid Game—both the original Netflix series and MrBeast's real life version. Then I watched every video I could find about how MrBeast's Squid Game was made. I was surprised to learn that MrBeast used a whole bunch of CGI (computer-generated imagery). I had taken its realty at face value.

There are four places I know of where MrBeast used CGI.

The sky above the big dirt field wasn't originally sky. The scenes were actually filmed in an indoor stadium.

The smoke effects of players who got eliminated were added in post-production.

A dust effect where a woman fell was CGI.

The big platforms for Tug-of-War and Glass Bridge weren't real at all. Both events took place close to the ground. In those scenes, almost everything the players didn't physically touch was CGI. Even the empty spaces between glass panels in Glass Bridge weren't real. They were solid black squares that got removed in post-production.

The sky, smoke and dust are all plausible. But the big giant platforms had to be CGI. Using such platforms in a real-life game would be ludicrously dangerous, especially without safety harnesses. Real platforms would be unnecessarily expensive too compared to replacing backgrounds, which is part VFX artists' standard toolkit.

Why didn't I notice?

First of all, the VFX team SoKrispyMedia is very good at VFX. But that alone isn't sufficient.

My experience of watching MrBeast's Squid Game reminds me of a magic trick routine I used to perform. To the audience it looked like two separate magic tricks but they actually worked together. First I would perform a series of tricks that used a real regular deck of cards. Everything was pure sleight-of-hand. Once my audience had gotten used to the idea that I'm doing everything legitimately I would switch to a set of gimmicked (fake) cards. The effect was much stronger than when I started out with the gimmicked (fake) cards.

MrBeast (perhaps unintentionally) pulled off a similar trick. When I watch television and movies, I'm constantly scrutinizing each scene to figure out how the filmmakers created it. But most of MrBeast's videos are completely real. When I watched MrBeast's Squid Game it did not occur to me to question anything, even when the images onscreen were unrealistic. I wasn't on-guard for CGI trickery.

Unexamined Beliefs

Most of our beliefs never go through a thorough examination. There's just too much stuff going on. It's infeasible to question whether every rock and tree you look at is real. Perhaps you pick up a rock and question its legitimacy once or twice, but you can't do it for every rock you walk by every day. There isn't enough time in the day.

One way to define beliefs is to ask "what would you bet money on". But contemplating whether you would bet money on an unexamined belief modifies that belief the way forcing a quantum superposition into one of its eigenvectors destroys the initial state. The number of beliefs that go through a rational analysis is a tiny fraction of our total beliefs. The beliefs you would bet money on are an even smaller selection. Nearly all of our beliefs are (in their natural state) not things we would bet money on.

The problem of identifying what to question is a more fundamental challenge than reconsidering cached thoughts and thinking outside your comfort zone. Cached thoughts have been examined at least once (or, at the very least, processed into words), which is more examination than most of the beliefs flowing through a human brain. As for thinking outside your comfort zone, someone raised in a religious household who is merely 50% confident in Creationism is already most of the way to abandoning it.

Identifying the small bits of information that are even worth thinking about i...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Some notes on common challenges building EA orgs, published by Ben Kuhn on December 3, 2022 on The Effective Altruism Forum.

(Follow-up to: Want advice on management/organization-building?)

After posting that offer, I've chatted with a few people at different orgs (though still have bandwidth for more if other folks are interested, as I find them pretty fun!) and started to notice some trends in what kinds of management problems different orgs face. These are all low-confidence—I'd like to validate them against a broader set of orgs—but I thought I'd write them up to see if they resonate with people.

Observations

Many orgs skew extremely junior

The EA community skews very young, and highly engaged EAs (like those working at orgs) skew even younger. Anecdotally, I was the same age or older than almost everyone I talked to, many of whom were the most experienced person in their org. By comparison, at Wave almost the entire prod/eng leadership team is my age or older. (Note that this seems to be less true in the most established/large/high-status orgs, e.g. Open Phil.)

This isn't a disaster, especially since the best of the junior people are very talented, but it does lead to a set of typical problems:

having to think through how to do everything from first principles rather than copy what's worked from elsewhere

managers needing to spend a fair amount of time providing basic "how to be a functioning employee" support to junior hires

managers not providing that support and the junior hires ending up being less effective, or growing less quickly, than they would at an org that could provide them more support

(At Wave, we've largely avoided hiring people with <2 years of work experience onto the prod/eng team for this reason. Of course, that's easier for us than for many EA orgs, so I'm not suggesting this as a general solution.)

It also leads to two specific subproblems:

Many managers are first-time managers

Again, this isn't a disaster, since many of these first-time managers are very smart, hardworking and kind. But, first-time managers have a few patterns of mistakes they tend to make, mostly related to "seeing around corners" or making decisions whose consequences play out on a long time horizon (understandably, since they often haven't worked as a manager for long enough to have seen long-time-horizon decisions play out!). This is things like:

Providing enough coaching and career growth for their team

Giving feedback that's difficult to give

Making sure their reports are happy in their roles and not burning out

The last point seems especially underrated in EA, I suspect because people are unusually focused on "doing what's optimal, not what's fun." That's a good idea to a large extent, but even for many people who are largely motivated by impact, they can be massively more or less productive depending on how much their day-to-day work resonates with them. But I suspect many EAs, like me, are reluctant to admit that this applies to us too and we're not purely impact-maximizing robots.

Almost all managers-of-managers are first-timers

Going from managing individual contributors to managing managers is a fairly different skillset in many ways. Most people in EA orgs that are managing managers seem to be "career EAs" for whom it's their first manager-of-managers role. As above, new managers of managers tend to make some predictable mistakes, e.g.:

Not planning far enough ahead for what hires they'll need to make to support their org's growth

Hiring/promoting the wrong people into management roles (having become a good manager doesn't necessarily mean you'll be good at coaching/evaluating other managers who fail in different ways than you did!)

Not noticing team dysfunction, either due to not having systems in place (e.g. skip-level 1:1s) or not knowing that something is ...