Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Red-teaming contest: demographics and power structures in EA, published by TheOtherHannah on August 31, 2022 on The Effective Altruism Forum.

Summary

In this post we look at two common high-level criticisms of the EA movement, and evaluate them against specific examples from the global health and development cause area.

These high-level criticisms are [Section 1]:

EA’s narrow demographic is bad for the movement, and

EA has power structures that mirror (neo)colonialism, paternalism and institutional racism.

We evaluate these through three case studies [described in Section 2], and we identify three issues that we believe are restricting the EA movement's effectiveness and growth [Section 3]. These identified issues are:

3.1 Demographic homogeneity (specifically, low cultural/geographic diversity) is limiting the EA community’s ability to design and deliver effective, innovative solutions. [confidence: relatively strong, size of impact: medium in non-longtermist cause areas]

3.2 Decision-makers in some EA orgs do not have enough information or robust processes for addressing their blindspots, and this decreases the accuracy of their models. [confidence: medium, size of impact: relatively small]

3.3 Power structures within EA organisations have parallels to colonialism and institutional racism, and this is damaging the reputation and credibility of the EA movement. [confidence: medium, size of impact: difficult to determine, and debatable]

We believe these issues are limiting EA’s ability to become a truly global movement that inspires innovative and effective solutions for the world's most pressing problems [Section 4].

Finally, we make recommendations for the EA community and some EA orgs to adopt in the near-term future [Section 5]. Broadly, our recommendations seek to:

Increase the sharing of EA tools with people in different geographic areas, without prescribing the end goal for those we share them with.

Increase conversations within EA about colonialism and institutional racism, with the aim that community members can better identify and avoid biases and power structures that perpetuate them.

Epistemic status:

We have considered the high-level criticisms from a variety of angles, and this post includes only the issues we think are the most persuasive. None of us are experts in the fields investigated, so we anticipate that there will be some mistakes or blindspots of our own contained in this critique. If you think we've completely missed work that EA is doing in these areas, perhaps this signals that the EA culture we've been exposed to doesn’t have enough mention of these aspects of EA.

Section One: Introducing the original high-level criticisms

1.1 EA’s narrow demographic is bad for the movement

1.2 EA has power structures that mirror (neo)colonialism, paternalism and institutional racism.

We will give a quick overview of these common high-level criticisms and state what we hope to add to the discussion.

1.1 EA’s narrow demographic is bad for the movement

The demographic of people who respond to the EA survey has stayed relatively consistent over recent years. Respondents are over 70% each of White, male and young, and live in the same set of five high-income, western countries (the US, the UK, Germany, Australia, and Canada) that were most common in previous years.

People commonly comment on the importance of demographic diversity, and how to increase it. The Centre for Effective Altruism (CEA) has a statement on why diversity, equity and belonging are important in an EA context, and the steps they are taking to increase these. It’s really good and you should check it out. This discussion is missing:

concrete examples that demonstrate a theory of change from “increased diversity” to “better ability to improve the lives of others”, and

Systems-change (outside o...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Sequencing Intro, published by jefftk on August 29, 2022 on LessWrong.

I've been working in computational bio for a

couple months now and I've been learning a lot. There's still a ton I don't know, but I'm currently at a stage where I've put some pieces together while still remembering what it was like not to understand them, which is often a good time to try to write introductory stuff. Trying to explain things is also a good way of making sure I understand them myself. So, here's an dump of what I've been learning:

The biological world primarily stores information with nucleic acids. These are series of nucleotides, often called

bases: A, C, G, and T. For example, a

strand of nucleotides could look like:

The two main kinds of nucleic acid are DNA and RNA. They differ in a few ways, but from a computational perspective they're very similar. Physical RNA will have U instead of T, though in sequencing data you'll often see it with with T anyway.

Each base has a complement: A bonds with T, and G with C. A nucleic acid that comprises two bonded strands is called double stranded. Each base in one strand will be bonded to its complement:

CCGACTCTGTCACGGGTCTAGCAATGTGGTAAGCA

This is the famous double helix and makes for a more stable structure than single stranded nucleic acid.

Going from a physical nucleic acid to a sequence on a computer is sequencing, and the reverse is synthesis. I'm only going to talk about the former; I haven't learned much about the latter.

The most common sequencing method today is Next

Generation Sequencing, commonly called Illumina sequencing after the main vendor. Bases are dyed and the machine reads their colors. The output of sequencing is a large number of short reads. Each read is a sequence of 50-300 bases, usually around 150. In setting up the sequencing run you choose how many bases to read, and different applications will make the most sense with different lengths. Accuracy drops off as you read farther along the strand. Note the lengths we're talking about are way less than the length of a full nucleic acid, which is generally at least thousands of bases. Not getting the full picture is a big downside of this kind of sequencing.

Let's get some real data to play with. When people publish a paper that depends on sequencing they generally upload their raw data to the NIH's National Center for

Biotechnology Information (NCBI). Here's a paper I've been looking at recently, which sequenced wastewater: RNA

Viromics of Southern California Wastewater and Detection of SARS-CoV-2

Single-Nucleotide Variants. If you look down to the "Data availability" section, you'll see:

Raw sequencing data have been deposited on the NCBI Sequence Read

Archive under accession number PRJNA729801, and representative code can be found at.

The GitHub link is helpful for getting metadata (what does each sample represent?) and understanding how they processed it (what tools did they use and how?), but for now we're looking for sequencing data. The accession number is "PRJNA729801", and while we could click through and download it from the NCBI, the user interface on the European mirror (European Nucleotide Archive) is much better. We go to

their landing page and enter the accession number:

This takes us to a page that describes the data:

We want to sanity check the title to make sure we didn't end up with the wrong data set, and "Metatranscriptomic sequencing of Southern

California wastewater" sounds about right.

Scrolling down there are links:

We could download all of this data, but it would be about 80GB compressed. For now, let's just download a single

fastq.gz file, at ~150MB: SRR14530724_1.fastq.gz.

These files are generally both very large and very repetitive, so they're a natural candidate for compression. The most common option is gzip, and that's what they've us...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Will MacAskill Media for WWOTF - Full List, published by James Aitchison on August 30, 2022 on The Effective Altruism Forum.

Never before have we had the chance to enjoy so much Will MacAskill. He seems to have been everywhere. What a superb and exhaustive job he and his team have done to promote 'What We Owe The Future.'

So far I have counted 16 podcasts, 18 articles and 6 other bits and pieces. I have listed all these below with links and brief comments.

Podcast Appearances

The 80,000 Hours Podcast with Rob Wiblin. A warm and comprehensive three-hour discussion.

Making Sense Podcast with Sam Harris. Harris is strongly supportive, MacAskill particularly inspiring on the sweep of history.

Mindscape Podcast with Sean Carroll. Carroll asks questions about utilitarianism, metaethics and population ethics which MacAskill handles well.

The Ezra Klein Show Podcast. A fine conversation on long-termism. Klein structures the discusion around ‘Three simple sentences: Future people count. There could be a lot of them. And we can make their lives better.’ Good discussions about history - the contingent nature of the abolition of slavery and that certain times have plasticity.

Tim Ferriss Podcast. A lively discussion with much humour and several gems from MacAskill. Includes recommendation of Joseph Henrich's 'The Secret of Our Success.'

Deep Dive with Ali Abdaal Podcast. A relaxed, friendly and wide-ranging three-hour conversation. Covers a lot of ground including EA psychology and MacAskill’s work methods. This is a high-quality YouTube production as well as a podcast and is my favourite among the appearances.

Conversations with Tyler Podcast . Tyler Cowan’s questioning focuses on the limits of utilitarianism.

The Lunar Society Podcast with Dwarkesh Patel. Mainly on the contingency of moral progress.

Global Dispatch Podcast with Mark Goldberg. Discussion on longtermism and the United Nations. Goldberg enthusiastic about the UN adopting some longtermist thinking.

Modern Wisdom Podcast with Chris Williamson. An accessible discussion of longtermism.

Conversations with Coleman with Coleman Hughes Includes population ethics, economic growth and moral change.

Daily Stoic Podcast with Ryan Holiday. Mainly on altruism and moral change.

Kera Think with Krys Boyd. 30 minutes conversation.

Freakanomics Podcast with Steve Levitt. Discussion mainly on the economic themes in WWOTF, which MacAskill handles very well.

1a Podcast on NPR. David Gurn discussion on EA as a life changing philosophy. Includes comments from Sofya Lebedeva and Spencer Goldberg.

Ten Percent Happier Podcast with Dan Harris. A warm discussion on donations, EA and longtermism.

There are transcripts for the podcasts by Wiblin, Carroll, Klein, Cowan, Patel, Goldberg and Levitt.

Articles and Book Reviews

The New Yorker: The Reluctant Prophet of Effective Altruism by Gideon Lewis-Kraus. A fine 10,000-word article profiling MacAskill and setting out the history of the EA movement. The author spent several days with his subject and covers MacAskill as an individual and the ideas and dynamics of the movement. MacAskill comments on the article in this Twitter Thread

Time: Want to Do More Good? This Movement might have the Answer by Naina Bajekal . A beautifully written and inspiring profile of MacAskill and the EA movement.

Vox: How Effective Altruism Went from a Niche Movement to a Billion-Dollar Force by Dylan Matthews. A well-informed and thoughtful article on EA’s evolution by an EA insider.

Wired: The Future Could be Blissful - If Human's Don't Go Extinct First. Shorter interview with Will Macaskill by Matt Reynolds.

New York Times: The Case for Longtermism by Will MacAskill A Guest Essay adapted from the book.

BBC Futures: What is Longtermism and Why Does it Matter? by Will MacAskill. Another essay based on the book.

Foreig...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Survey of NLP Researchers: NLP is contributing to AGI progress; major catastrophe plausible, published by Sam Bowman on August 31, 2022 on The AI Alignment Forum.

I was part of a group that ran a PhilPapers-style survey and metasurvey targeting NLP researchers who publish at venues like ACL. Results are here (Tweet-thread version). It didn't target AGI timelines, but had some other questions that could be of interest to people here:

NLP is on a path to AGI: 58% agreed that Understanding the potential development of artificial general intelligence (AGI) and the benefits/risks associated with it should be a significant priority for NLP researchers.

Related: 57% agreed that Recent developments in large-scale ML modeling (such as in language modeling and reinforcement learning) are significant steps toward the development of AGI.

AGI could be revolutionary: 73% agreed that In this century, labor automation caused by advances in AI/ML could plausibly lead to economic restructuring and societal changes on at least the scale of the Industrial Revolution.

AGI could be catastrophic: 36% agreed that It is plausible that decisions made by AI or machine learning systems could cause a catastrophe this century that is at least as bad as an all-out nuclear war.

46% of women and 53% of URM respondents agreed.

The comments suggested that people took a pretty wide range of interpretations to this, including things like OOD robustness failures leading to weapons launches.

Few scaling maximalists: 17% agreed that Given resources (i.e., compute and data) that could come to exist this century, scaled-up implementations of established existing techniques will be sufficient to practically solve any important real-world problem or application in NLP.

The metasurvey responses predicted that 47% would agree to this, so there are fewer scaling maximalists than people expected there to be.

Optimism about ideas from cognitive science: 61% agreed that It is likely that at least one of the five most-cited systems in 2030 will take clear inspiration from specific, non-trivial results from the last 50 years of research into linguistics or cognitive science.

This strikes me as very optimistic, since it's pretty clearly false about the most cited systems today.

Optimism about the field: 87% agreed that On net, NLP research continuing into the future will have a positive impact on the world.

32% of respondents who agreed that NLP will have a positive future impact on society also agreed that there is a plausible risk of global catastrophe.

Most NLP research is crap: 67% agreed that A majority of the research being published in NLP is of dubious scientific value.

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Toby Ord’s The Scourge, Reviewed, published by ColdButtonIssues on August 30, 2022 on The Effective Altruism Forum.

Cross-posted from Cold Button Issues.

On a recent Wednesday afternoon, I was having lunch with a friend and mentioned that some of my friends donated money to buy bed nets to prevent malaria. “Waste of time,” my friend said.

“Do bed nets not work?” I asked.

“Oh, that’s not it. Rather, people in developing countries just don’t have moral standing. People say that all lives are of equal worth, but they’d rather go on a nice vacation than donate enough to save a life through an effective charity, which I’ve heard costs $5,000.”

She then showed me her new emerald brooch (market value: $5,001). “And even if we thought people were just dumb about charity, we would still expect politics to revolve around global poverty if all lives were of equal worth, and it doesn’t. Deep down, we all know foreigners don’t matter.”

Then I ate my veal entrée. People have told me that animal suffering is wrong, but since they seemed to rarely act on that claim, I had concluded that animal suffering was no big deal.

Anyways, this is a review of Toby Ord’s The Scourge.

Ord’s Argument

Moral indifference justifies itself. Or at least that is Ord’s argument when it comes to embryos.

Some argue that human embryos have just as much moral worth as the normal, walking around type humans. Some religious people use the phrase “womb to tomb,” indicating that a human life is a human life, from conception to death.

Some argue against embryos having equal value to walking around type humans, by appeals to intuition or incredulity- it’s a clump of cells! Others point out that embryos lack some features that might make people morally valuable- desires, consciousness, self-identity, friendships with others.

Ord instead poses a thought experiment: if a new disease emerged that caused the vast majority of human deaths and targeted the young, would responding to such a nightmarish pandemic be the instant priority of all humanity? Yet if human embryos matter just as much as normal humans and we now know that the majority of embryos suffer spontaneous abortions, why is there no outcry? Why do people claim that life expectancy in the United States is near 80 years of age, instead of acknowledging that life expectancy from conception is remarkably low? Ord produces this helpful life expectancy chart from conception.

If embryos really mattered, people would care about the vast number of embryos that die naturally. Ord argues that the “marked lack of curiosity about what is claimed to be of immense importance suggests that even now, few people really believe that full moral status begins at conception.”

People barely care about this, so embryos matter little or none. Therefore, Ord argues, appeals to the value of human embryos to argue against abortion or IVF fail.

Extensions

An obvious extension of Ord’s claim is the field of global poverty. Many people say that your moral worth doesn’t depend on where you live or that your obligation to others doesn’t depend on a shared race, nationality, regional proximity, or language. However, almost everyone acts like lives are not of equal value, ergo, they are not.

The number of animals suffering in factory farms is immense. Only a small number of people are vegans and only a trivial number of people are full-time animal welfare activists or factory farm saboteurs. Ergo, animal suffering is no big deal.

Now and then, some people will say we should care about future generations. While people do appear willing to make some sacrifices for their children and grandchildren it doesn’t normally extend much further than that. As I rested in my mansion’s east wing (paid for by mortgaging my great-grandchildren’s future!), I read a book another friend had recommended, coin...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: A subjective account of what it's like to join an EA-aligned org without previous EA knowledge, published by lukasj10 on August 30, 2022 on The Effective Altruism Forum.

TL;DR

We (Healthier Hens (HH), a CE-incubated EA-aligned animal welfare charity) have hired a mid-career Country Manager for our operation in Kenya. He had no previous EA knowledge or experience. We asked him 15 questions about his experience entering the EA world, learning about the key concepts and the community in general. This post is an overview of his subjective experiences including overarching themes such as what seemed common and different. Some of the benefits felt upon entering EA and his perspective on EA awareness and community. Finally, we reflect on the above from the organisation’s standpoint.

Introduction

After having chosen our pilot country of operations, we hired a Country Manager (CM) based in Kenya, to lead our efforts on the ground. Despite having previous experience working in the humanitarian and animal welfare sectors, he had no previous knowledge of Effective Altruism (EA). We thought that learning about his experience jumping into the EA world could be useful for other EA-aligned orgs hiring externally and upcoming ones, considering what challenges it could bring.

We carried out a semi-structured interview, asking 15 questions (plus follow-ups) to understand what his experience was like as an applicant and employee. This post is a summary of the responses that our questions elicited, presented under the umbrella of several overarching themes that had come up. Readers are advised that this is a highly subjective account of early professional engagement with EA concepts and working principles. We are, however, very keen on encouraging other orgs that have gone through similar hiring processes to engage their new employees with similar inquiries to better understand how we can transition more smoothly, since the focus on attracting talent is definitely here to stay.

Differences and similarities - the before and after

Among notable differences between regular non-profit programs and those stemming from EA, the focus on positive impact in the (at least relative) long-term stood out. Having seen several short-term interventions come and go in the region, our CM is inspired by the community’s attempts to seek measurable and quantifiable ways to achieve change. In the case of Animal Welfare (AW), he was yearning for solutions going beyond one-dimensional, 5-year “band-aid solution” interventions. EA thinking of how to have a long-lasting, multigenerational effect with positive flow through effects made sense, even when considering the experiences and frustrations of conventional NGO stakeholders.

Using the ITN framework was also a fresh breath of air - our CM found regular interventions reliant on emotions and public opinion far too often. Data should permit arriving at better decisions. This was a major motivator when considering applying for the job. Despite counterfactuals being a new concept to the entire animal welfare community in Kenya, the more he had learnt about it, the more he wanted to help lead an organisation that could potentially have a positive impact on millions of hens worldwide. A surprising discovery for him was to find out that Open Wing Alliance (OWA) is part of the EA community. It then made great sense why the group advocates for the cage-free transition, and how impactful that can be.

Regarding the recruitment process, our CM found it surprisingly thought-provoking and useful - each part of it taught him even more about the organisational concept and ultimate final impact that we seek. In past recruitment experiences, there were practically no stages based on theoretical or project management skills, with most focus on face-to-face discussions. He experie...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Worlds Where Iterative Design Fails, published by johnswentworth on August 30, 2022 on The AI Alignment Forum.

In most technical fields, we try designs, see what goes wrong, and iterate until it works. That’s the core iterative design loop. Humans are good at iterative design, and it works well in most fields in practice.

In worlds where AI alignment can be handled by iterative design, we probably survive. So long as we can see the problems and iterate on them, we can probably fix them, or at least avoid making them worse.

By the same reasoning: worlds where AI kills us are generally worlds where, for one reason or another, the iterative design loop fails. So, if we want to reduce X-risk, we generally need to focus on worlds where the iterative design loop fails for some reason; in worlds where it doesn’t fail, we probably don’t die anyway.

Why might the iterative design loop fail? Most readers probably know of two widely-discussed reasons:

Fast takeoff: there will be a sudden phase shift in capabilities, and the design of whatever system first undergoes that phase shift needs to be right on the first try.

Deceptive inner misalignment: an inner agent behaves well in order to deceive us, so we can’t tell there’s a problem just by trying stuff and looking at the system’s behavior.

. but these certainly aren’t the only reasons the iterative design loop potentially fails. This post will mostly talk about some particularly simple and robust failure modes, but I’d encourage you to think on your own about others. These are the things which kill us; they’re worth thinking about.

Basics: Hiding Problems

Example/Analogy: The Software Executive

Imagine that a software company executive, concerned about the many errors coming from the software, creates a new incentive scheme: software developers get a monetary reward for changes which decrease the rate of error messages showing up on the manager’s dashboard, and get docked for changes which increase the rate of error messages.

As Tyler Cowen would say: “solve for the equilibrium”. Obvious equilibrium here: the developers stop throwing error messages when they detect a problem, and instead the software just fails silently. The customer’s experience remains the same, but the manager’s dashboard shows fewer error messages. Over time, the customer’s experience probably degrades, as more and more problems go undetected.

In the short run, the strategy may eliminate some problems, but in the long run it breaks the iterative design loop: problems are not seen, and therefore not iterated upon. The loop fails at the “see what goes wrong” step.

Why RLHF Is Uniquely Terrible

The software executive’s strategy is the same basic idea as Reinforcement Learning from Human Feedback (RLHF). AI does something, a human looks at what happened to see if it looks good/bad, and the AI is trained on the human’s feedback. Just like the software executive’s anti-error-message compensation scheme, RLHF will probably result in some problems actually being fixed in the short term. But it renders the remaining problems far less visible, and therefore breaks the iterative design loop. In the context of AI, RLHF makes it far more likely that a future catastrophic error will have no warning signs, that overseers will have no idea that there’s any problem at all until it’s much too late.

Note that this issue applies even at low capability levels! Humans overlook problems all the time, some of those mistakes are systematic, and RLHF will select for places where humans systematically overlook problems; that selection pressure applies even when the neural net lacks great capabilities.

This is the core reason why I consider RLHF uniquely terrible, among alignment schemes. It is the only strategy I know of which actively breaks the iterative design loop; it makes probl...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: How likely is deceptive alignment?, published by Evan Hubinger on August 30, 2022 on The AI Alignment Forum.

The following is an edited transcript of a talk I gave. I have given this talk at multiple places, including first at Anthropic and then for ELK winners and at Redwood Research, though the version that this document is based on is the version I gave to SERI MATS fellows. Thanks to Jonathan Ng, Ryan Kidd, and others for help transcribing that talk. Substantial edits were done on top of the transcription by me. Though all slides are embedded below, the full slide deck is also available here.

Today I’m going to be talking about deceptive alignment. Deceptive alignment is something I'm very concerned about and is where I think most of the existential risk from AI comes from. And I'm going to try to make the case for why I think that this is the default outcome of machine learning.

First of all, what am I talking about? I want to disambiguate between two closely related, but distinct concepts. The first concept is dishonesty. This is something that many people are concerned about in models, you could have a model and that model lies to you, it knows one thing, but actually, the thing it tells you is different from that. So this happens all the time with current language models—we can, for example, ask them to write the correct implementation of some function. But if they've seen humans make some particular bug over and over again, then even if in some sense it knows how to write the right function, it's going to reproduce that bug. And so this is an example of a situation where the model knows how to solve something and nevertheless lies to you. This is not what I'm talking about. This is a distinct failure mode. The thing that I want to talk about is deceptive alignment which is, in some sense, a subset of dishonesty, but it's a very particular situation.

So deceptive alignment is a situation where the reason that your model looks aligned on the training data is because it is actively trying to look aligned for instrumental reasons, which is very distinct. This is a situation where the thing that is causing your model to have good performance is because it is trying to game the training data, it actively has a reason that it wants to stick around in training. And so it’s trying to get good performance in training for the purpose of sticking around.

Ajeya Cotra has a really good analogy here that I think is helpful for understanding the difference between these two classes. So you can imagine that you are a child and you've inherited a massive business. And you have to determine who's going to run the business for you. There's a bunch of candidates that you're trying to evaluate. And those candidates fall into three categories. You have the saints, which are people that really just want to help you, run things effectively, and accomplish what you want. You have the sycophants, which want to make you happy, satisfy the letter of your instructions, make it so that the business looks like it's doing well from your perspective, but don't actually want to fundamentally help you. And you have the schemers, people who want to use the control of the business for their own purposes, and are only trying to get control of it and pretend that they're doing the right thing, so that they can eventually get something later.

For our purposes, we're concerned primarily with the schemers and that is the deceptive alignment category.

So I would say in this situation that the sycophants are examples of dishonesty where they would say a bunch of false facts to you about what was happening to convince you that things were going well, but they don't have some ulterior motive. The schemers, they have some ulterior motive, they have something that they want to accomplish. And they're activ...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Notes on how prizes may fail and how to reduce the risk of them failing, published by Peter Wildeford on August 30, 2022 on The Effective Altruism Forum.

A lot of organizations in EA are working on prizes right now. I'm skeptical that they will work well without some particular effort - here are my some thoughts on them.

Though looking at the OP cause prioritization prize, Effective Ideas Prize, and the EA Criticism Prize, it seems like a lot of prizes are actually going pretty well so far.

(This is an experiment where ordinarily I would've spent time to make this a longer post, but after a few months I'm clearly not going to make time for this so instead I'm just going to post the outline to my post without working hard to turn it into a proper post. Let me know what you think about this format and I may do it more. Further note that this a personal post written on my own and not as a result of my role at Rethink Priorities.)

Why prizes are good

It’s easier to evaluate retrospectively than prospectively - you only have to pay for the things you like (or at least whatever you like most).

It lets people audition for a role, potentially letting talent shine that wouldn't otherwise get recognized ex ante.

They raise the salience of a particular need of grantmaker(s).

They're helpful for providing explicit encouragement to get people to do things they probably would've been happy to do anyway.

They provide credentials people can use in the future (e.g., "I was winner of {PRIZE}" goes on the resume).

Ways prizes fail

Prizes are often not large enough to produce sufficient expected value

Even if the expected value is good, people are not (an should not be) risk neutral with regard to personal finances

Even if the expected value is good, people frequently do not have the startup capital to float to get the prize

There are illusions of transparency where prize makers think they have clearly articulated what they want but they don’t, which leads to a lot of wasted time and disappointment

There is counterparty risk where someone taking on the personal risk of doing the prize may do well but then not be evaluated the way they expect

Ideas on how to make prizes better

Explicitly attempt to calculate expected value for participants and then offer a large enough prize

Be open to increasing the prize amount

In addition to the prize, offer to evaluate people/teams prospectively as best you can and give them start up capital to attempt the prize

Have smaller milestones that result in prizes to ensure people are on the same page and to reduce risk (the OP cause prioritization prize is great at this with honorable mentions)

Offer lots of smaller prizes to honorable mentions/attempts to smooth out risk

Invest a lot of time upfront to internally understand and externally communicate what you want and how you will judge things

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: EA & LW Forums Weekly Summary (21 Aug - 27 Aug 22'), published by Zoe Williams on August 30, 2022 on LessWrong.

This is also posted on the EA forum: see here.Supported by Rethink Priorities

Sunday August 21st - Saturday August 27th

The amount of content on the EA and LW forums has been accelerating. This is awesome, but makes it tricky to keep up with! The below hopes to help by summarizing popular (>40 karma) posts each week. It also includes announcements and ideas from Twitter that this audience might find interesting. This will be a regular series published weekly - let me know in the comments if you have any feedback on what could make it more useful!If you'd like to receive these summaries via email, you can subscribe here.

Methodology

This series originated from a task I did as Peter Wildeford’s executive research assistant at Rethink Priorities, to summarize his weekly readings. If your post is in the ‘Didn’t Summarize’ list, please don’t take that as a judgment on its quality - it’s likely just a topic less relevant to his work. I’ve also left out technical AI posts because I don’t have the background knowledge to do them justice.

My methodology has been to use this and this link to find the posts with >40 karma in a week for the EA forum and LW forum respectively, read / skim each, and summarize those that seem relevant to Peter. Those that meet the karma threshold as of Sunday each week are considered (sometimes I might summarize a very popular later-in-the-week post in the following week’s summary, if it doesn’t meet the bar until then). For twitter, I skim through the following lists: AI, EA, Forecasting, National Security (mainly nuclear), Science (mainly biosec).

I’m going through a large volume of posts so it’s totally possible I’ll get stuff wrong. If I’ve misrepresented your post, or you’d like a summary edited, please let me know (via comment or DM).

EA Forum

Philosophy and Methodologies

Critque’s of MacAskill’s ‘Is it Good to Make Happy People?’

Discusses population asymmetry, the viewpoint that a new life of suffering is bad, but a new life of happiness is neutral or only weakly positive. Post is mainly focused on what these viewpoints are and that they have many proponents vs. specific arguments for them. Mentions that they weren’t well covered in Will’s book and could affect the conclusions there.

Presents evidence that people’s intuitions tend towards needing significantly more happy people than equivalent level of suffering people for a tradeoff to be ‘worth it’ (3:1 to 100:1 depending on question specifics), and that therefore a big future (which would likely have more absolute suffering, even if not proportionally) could be bad.

EAs Underestimate Uncertainty in Cause Prioritization

Argues that EAs work across too narrow a distribution of causes given our uncertainty in which are best, and that standard prioritizations are interpreted as more robust than they really are.As an example, they mention that 80K states “some of their scores could easily be wrong by a couple of points” and this scale of uncertainty could put factory farming on par with AI.

The Repugnant Conclusion Isn’t

The repugnant conclusion (Parfit, 1984) is the argument that enough lives ‘barely worth living’ are better than a much smaller set of super duper awesome lives. In one description of it, Parfit said the barely worth it lives had ‘nothing bad in them’ (but not much good either).

The post argues that actually makes those lives pretty awesome and non-repugnant, because nothing bad is a high bar.

A Critical Review of Givewell’s 2022 Cost-effectiveness Model

NB: longer article - only skimmed it so I may have missed some pieces.

Suggestions for cost-effectiveness modeling in EA by a health economist, with Givewell as a case study. The author believes the overall approach ...