Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: The elephant in the bednet: the importance of philosophy when choosing between extending and improving lives, published by MichaelPlant on November 18, 2022 on The Effective Altruism Forum.

Michael Plant, Joel McGuire, and Samuel Dupret

Summary

How should we compare the value of extending lives to improving lives? Doing so requires us to make various philosophical assumptions, either implicitly or explicitly. But these choices are rarely acknowledged or discussed by decision-makers, all of them are controversial, and they have significant implications for how resources should be distributed.

We set out two crucial philosophical issues: (A) an account of the badness of death, how to determine the relative value of deaths at different ages, and (B) locating the neutral point, the place on the wellbeing scale at which life is neither good nor bad for someone. We then illustrate how different choices for (A) and (B) alter the cost-effectiveness of three charities which operate in low-income countries, provide different interventions, and are considered to be some of the most cost-effective ways to help others: Against Malaria Foundation (insecticide-treated nets), GiveDirectly (cash transfers), and StrongMinds (group therapy for depression). We assess all three in terms of wellbeing-adjusted life years (WELLBYs) and explain why we do not, and cannot, use standard health metrics (QALYs and DALYs) for this purpose. We show how much cost-effectiveness changes by shifting from one extreme of (reasonable) opinion to the other. At one end, AMF is 1.3x better than StrongMinds.

At the other, StrongMinds is 12x better than AMF. We do not advocate for any particular view. Our aim is simply to show that these philosophical choices are decision-relevant and merit further discussion.

Our results are displayed in the chart below, which plots the cost-effectiveness of the three charities in WELLBYs/$1,000.

StrongMinds and GiveDirectly are represented with flat, dashed lines because their cost-effectiveness does not change under the different assumptions. The changes in AMF’s cost-effectiveness are a result of two varying factors. One is using different accounts of the badness of death, that is, ways to assign value to saving lives at different ages; these three accounts go by unintuitive names in the philosophical literature, so we’ve put a slogan in brackets after each one to clarify their differences: deprivationism (prioritise the youngest), the time-relative interest account (prioritise older children over infants), and Epicureanism (death isn’t bad for anyone – prioritise living well, not living long). We also consider including two variants of the time-relative interest account (TRIA); on these, life has a maximum value at the ages of either 5 or 25.

The other factor is where to locate the neutral point, the place at which someone has overall zero wellbeing, on a 0-10 life satisfaction scale; we assess that as being at each location between 0/10 and 5/10. As you can see, AMF’s cost-effectiveness changes a lot. It is only more cost-effective than StrongMinds if you adopt deprivationism and place the neutral point below 1.

1. Introduction

How should we compare the value of extending lives to improving lives? Let’s focus our minds with a real choice. On current estimates, for around $4,500, you can expect to save one child’s life by providing insecticide-treated nets (ITNs). Alternatively, that sum could provide a $1,000 cash transfer to four-and-a-half families living in extreme poverty ($1,000 is about a year’s household income). The cost of both choices is the same, but the outcomes differ. Which one will do the most good?

This is a difficult and discomforting ethical question. How might we answer it? And how much would different answers change the priorities?

There are various m...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Does Sam make me want to renounce the actions of the EA community? No. Does your reaction? Absolutely., published by GoodEAGoneBad on November 18, 2022 on The Effective Altruism Forum.

Alright, you finally broke me. No honey, all vinegar.

I have spent close to ten years in EA. Over those ten years, I have worked extremely hard and invested in our community. I have organized and helped newcomers and junior members when I’m exhausted, when I’m facing family crises, when the last thing I want to do is get on the phone or train. Instead of going to the private sector and making money, I have stuck around doing impactful jobs that I largely hate as a way of dedicating myself to helping others and this community to thrive. I don’t regret this. Do FTX or Sam Bankman-Fried's actions change my assessment of my actions? No. Do the reactions I have seen here? Yes, I’m afraid so.

The things that draw me to the EA community are, above anything else, its commitment to supporting one another and working together as a team to reduce suffering. Throughout the years, I have held friends as they cried because a project they were doing failed, celebrated their successes, and watched again and again as EAs do the brave thing and are then there for one another. Through this, I have built friendships and relationships that are the joys of my life.

Now the new catechism. Do I condemn fraud? Yes. Of course, I do. This is a stupid question EAs keep performatively asking and answering. Everyone opposes fraud, there is no one on the other side of this issue. Sam’s actions were awful and I condemn them. Do I believe we should circle our wagons and defend Sam? No. However, there is a huge difference between condemning his actions while rallying together to support one another through this awful time and what I see happening here which I believe can best be described as a witch-hunt against everything and everyone that ever intersected with Sam or his beliefs.

Over the last few days, posters on this forum, Twitter, and Facebook have used this scandal to attack and air every single grievance they have ever had against Effective Altruism or associated individuals and organizations. Even imaginary ones. Especially imaginary ones. This has included Will MacAskill and other thought leaders for the grave sin of not magically predicting that someone whose every external action suggested that he wanted to work with us to make the world a better place, would YOLO it and go Bernie Madoff. The hunt has included members of Sam’s family for the grave sin of being related to him. It has included attributing the cause of Sam’s actions to everything from issues with diversity and inclusivity, lack of transparency in EAG admissions, the pitfalls of caring if we all get eviscerated by a nuke or rogue AI, and, of course, our office spaces and dating habits.

Like Stop. There are lessons to be learned here and I would have been fully down for learning them and working together to fix them with all of you. But why exactly should I help those in the community who believe that the moral thing to do when someone is on their knees is to curb stomp them while yelling “I should have been admitted to EAG 2016!”? Why should I expose myself further by doing ambitious things (No I don’t mean fraud- that’s not an ambitious thing that’s a --- criminal--- thing) when if I fail people are going to make everything worse by screaming “I told you so” to signal that they never would have been such a newb? Yeah. No. The circle I’m drawing around who is and is not in my community is getting dramatically redrawn. This is not because one person or company made a series of very bad decisions, it's because so many of your actions are those of people I will not invest in further and who I don't want anywhere near my life or life’s work.

I’ll k...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: EA is a global community - but should it be?, published by Davidmanheim on November 18, 2022 on The Effective Altruism Forum.

Without trying to wade into definitions, effective altruism is not just a philosophy and a plan of action, it’s also a community. And that means that community dynamics are incredibly important in shaping both the people involved, and the ideas. Healthy communities can make people happier, more effective, and better citizens locally and globally - but not all communities are healthy. A number of people have voiced concerns about the EA community in the recent past, and I said at the time that I think that we needed to take those concerns seriously. The failure of the community to realize what was happening with FTX isn’t itself an indictment of the community - especially given that their major investors did not know - but it’s a symptom that reinforces many of the earlier complaints.

The solutions seem unclear, but there are two very different paths that would address the failure - either reform, or rethinking the entire idea of EA as a community. So while people are thinking about changes, I’d like to suggest that we not take the default path of least resistance reforms, at least without seriously considering the alternative.

“The community” failed?

Many people have said that the EA community failed when they didn’t realize what SBF was doing. Others have responded that no, we should not blame ourselves. (As an aside, when Eliezer Yudkowsky is telling you that you’re overdoing heroic responsibility, you’ve clearly gone too far.) But when someone begins giving to EA causes, whether individually, or via Founders Pledge, or via setting up something like SFF, there is no-one vetting them for being honest or well controlled.

The community was trusting - in this case, much too trusting. And people have said that they trusted the apparent (but illusory) consensus of EAs about FTX. I am one of them. We were all too trusting of someone who, according to several reports, had a history of breaking rules and cheating others, including an acrimonious split that happened early on at Alameda, and evidently more recently frontrunning. But the people who raised flags were evidently ignored, or in other cases feared being pariahs for speaking out more publicly.

But the idea that I and others trusted in “the community” is itself a problem. Like Rob Wiblin, I generally subscribe to the idea that most people can be trusted. But I wasn’t sufficiently cautious about how trust that applies to “you won’t steal from my wallet, even if you’re pretty sure you can get away with it,” doesn’t scale to “you can run a large business or charity with effectively no oversight.” A community that trusts by default is only sustainable if it is small. Claiming to subscribe to EA ideas, especially in a scenario where you can be paid well to do so, isn’t much of a reason to trust anyone. And given the size of the EA community, we’ve already passed the limits of where trusting others because of shared values is viable.

Failures of Trust

There are two ways to have high trust: naivety, and sophistication. The naive way is what EA groups have employed so far, and the sophisticated way requires infrastructure to make cheating difficult and costly.

To explain, when I started in graduate school, I entered a high-trust environment. I never thought about it, partly because I grew up in a religious community that was high trust. So in grad school, I was comfortable if I left my wallet on my desk when going to the bathroom, or even sometimes when I had an hour-long meeting elsewhere in the building.

I think during my second year, someone had something stolen from their desk - I don’t recall what, maybe it was a wallet. We all received an email saying that if someone took it, they would be expel...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Don't design agents which exploit adversarial inputs, published by Alex Turner on November 18, 2022 on The AI Alignment Forum.

Summary. Consider two common alignment design patterns:

Optimizing for the output of a grader which evaluates plans, and

Fixing a utility function and then argmaxing over all possible plans.

These design patterns incentivize the agent to find adversarial inputs to the grader (e.g. "manipulate the simulated human grader into returning a high evaluation for this plan"). I'm pretty sure we won't find adversarially robust grading rules. Therefore, I think these alignment design patterns are doomed.

In this first essay, I explore the adversarial robustness obstacle. In the next essay, I'll point out how this is obstacle is an artifact of these design patterns, and not any intrinsic difficulty of alignment. Thanks to Erik Jenner, Johannes Treutlein, Quintin Pope, Charles Foster, Andrew Critch, randomwalks, and Ulisse Mini for feedback.

1: Optimizing for the output of a grader

One motif in some AI alignment proposals is:

An actor which proposes plans, and

A grader which evaluates them.

For simplicity, imagine we want the AI to find a plan where it makes an enormous number of diamonds. We train an actor to propose plans which the grading procedure predicts lead to lots of diamonds.

In this setting, here's one way of slicing up the problem:

Outer alignment: Find a sufficiently good grader.

Inner alignment: Train the actor to propose plans which the grader rates as highly possible (ideally argmaxing on grader output, but possibly just intent alignment with high grader output).

This "grader optimization" paradigm ordains that the AI find plans which make the grader output good evaluations. An inner-aligned actor is singlemindedly motivated to find plans which are graded maximally well by the grader. Therefore, for any goal by which the grader may grade, an inner-aligned actor is positively searching for adversarial inputs which fool the grader into spitting out a high number!

In the diamond case, if the actor is inner-aligned to the grading procedure, then the actor isn't actually aligned towards diamond-production. The actor is aligned towards diamond-production as quoted via the grader's evaluations. In the end, the actor is aligned to the evaluations.

I think that there aren't clever ways around this issue. Under this motif, under this way of building an AI, you're not actually building an AI which cares about diamonds, and so you won't get a system which makes diamonds in the limit of its capability development.

Three clarifying points:

This motif concerns how the AI makes decisions—this isn't about training a network using a grading procedure, it's about the trained agent being motivated by a grading procedure.

The grader doesn't have to actually exist in the world. This essay's critiques are not related to "reward tampering", where the actor messes with the grader's implementation in order to increase the grades received. The "grader" can be a mathematical expected utility function over all action-sequences which the agent could execute. For example, it might take the action sequence and the agent's current beliefs about the world, and e.g. predict the expected number of diamonds produced by the actions.

"The AI optimizes for what humanity would say about each universe-history" is an instance of grader-optimization, but "the AI has human values" is not an instance of grader-optimization.

The parable of evaluation-child

an AI should optimize for the real-world things I value, not just my estimates of those things. — The Pointers Problem: Human Values Are A Function Of Humans' Latent Variables

First, a mechanistically relevant analogy. Imagine a mother whose child has been goofing off at school and getting in trouble. The mom just wants her kid to t...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Engineering Monosemanticity in Toy Models, published by Adam Jermyn on November 18, 2022 on The AI Alignment Forum.

Overview

In some neural networks, individual neurons correspond to natural "features" in the input. Such monosemantic neurons are much easier to interpret, because in a sense they only do one thing. By contrast, some neurons are polysemantic, meaning that they fire in response to multiple unrelated features in the input. Polysemantic neurons are much harder to characterize because they can serve multiple distinct functions in a network.

Recently, Elhage+22 and Scherlis+22 demonstrated that architectural choices can affect monosemanticity, raising the prospect that we might be able to engineer models to be more monosemantic. In this work we report preliminary attempts to engineer monosemanticity in toy models.

Toy Model

The simplest architecture that we could study is a one-layer model. However, a core question we wanted to answer is: how does the number of neurons (nonlinear units) affect the degree of monosemanticity? To that end, we use a two-layer architecture:

Features are generated as sparse vectors in a high-dimensional space. They are then run through a (fixed) random projection layer to produce the inputs into our model. We imagine this random projection process as an analogy to the way the world encodes features in our observations.

Within the model, the first layer is a linear transformation with a bias, followed by a nonlinearity. The second layer is a linear transformation with no bias.

Our toy model is most similar to that of Elhage+22, with a key difference being that the extra linear layer allows us to vary the number of neurons independent of the number of features or the input dimension.

We study this two model on three tasks. The first, a feature decoder, performs a compressed sensing reconstruction of features that were randomly and lossily projected into a low-dimensional space. The second, a random re-projector, reconstructs one fixed random projection of features from a different fixed random projection. The third, an absolute value calculator, performs the same compressed sensing task and then returns the absolute values of the recovered features. These tasks have the important property that we know which features are naturally useful, and so can easily measure the extent to which neurons are monosemantic or polysemantic.

Note that we primarily study the regime where there are more features than embedding dimensions (i.e. the sparse feature layer is wider than the input) but where features are sufficiently sparse that the number of features present in any given sample is smaller than the embedding dimension. We think this is likely the relevant limit for e.g. language models, where there are a vast array of possible features but few are present in any given sample.

Key Results

We find that models initialized with zero mean bias (left) find different local minima depending on the learning rate, with more monosemantic solutions and slightly lower loss at higher learning rates. Models initialized with a negative mean bias (right) all find highly monosemantic local minima, and achieve slightly better loss. Note that these models are all in a regime where they have more neurons than there are input features.

Just to hammer home how weird this is, below we've plotted the activations of neurons in response to single-feature inputs. The three models we show get essentially the same loss but are clearly doing very different things!

More generally, we find:

When inputs are feature-sparse, models can be made more monosemantic with no degredation in performance by just changing which loss minimum the training process finds (Section 4.1.1).

More monosemantic loss minima have moderate negative biases in all three tasks, and we are able to use t...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Deontology and virtue ethics as "effective theories" of consequentialist ethics, published by Jan Kulveit on November 17, 2022 on LessWrong.

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Introducing new leadership in Animal Charity Evaluators’ Research team, published by Animal Charity Evaluators on November 17, 2022 on The Effective Altruism Forum.

Introduction

Hello EA Forum! As some of you may already be aware, Animal Charity Evaluators (ACE) has recently expanded our team, most notably in Research. As such, we'd like to introduce ourselves as ACE's new Director of Research and Evaluations Program Manager and share how we intend to engage the EA community via this forum moving forward. We both joined ACE this summer, right as the annual charity evaluation season kicked off. Since we were hired, two new researchers, Alina Salmen, Ph.D. and Max Taylor, have joined, as well as our new Executive Director, Stien van der Ploeg.

Our Backgrounds

Elisabeth Ormandy - Director of Research

I have an academic background in Neuroscience (B.Sc.), Applied Animal Behaviour and Welfare (M.Sc.), and Animal Welfare and Ethics (Ph.D.). I completed a research fellowship in animal policy development for the Canadian Council on Animal Care and held a post-doctoral fellowship position at the University of British Columbia (UBC). I also taught various undergraduate courses at UBC including: Animals and Society, Animals and Global Issues, Scholarly Writing and Argumentation, and Animals, Politics and Ethics. In 2015, I opted to leave academia to co-found the Canadian Society for Humane Science—I served as their Executive Director until I joined ACE. In that role, I gained experience in nonprofit management, impact assessment, and strategic planning, and learned the importance of strengthening the animal advocacy movement.

Alongside my paid work in animal advocacy, I currently serve in a number of volunteer roles, most notably as a Board member for the Association for the Protection of Fur-Bearing Animals.

Vince Mak - Evaluations Program Manager

I have a generalist background—I graduated from the Wharton School at the University of Pennsylvania and spent the beginning of my career in financial services. I discovered effective altruism in 2019, developed an interest in animal advocacy after reading Animal Liberation in 2020, and began doing EA-style charity evaluations across cause areas as a volunteer with SoGive in 2021. Outside of my job at ACE, I currently assist in various capacities with EA research, grantmaking, and movement building.

ACE's Next Steps on the EA Forum

Our intentions

In the past few years, ACE's activity on the EA Forum has been to provide updates on our work and share our thinking. As an EA organization dedicated to transparency and intellectual rigor, we would like to take it a step further and interact more closely with the community that shares these values. As we tinker with our evaluation methodology in the months and years ahead, we plan to invite your feedback during the intermediate stages; we do not want to just inform you, we want to be informed by you. In addition to opportunities to critique our thinking, you can also expect from ACE transparency about our processes, responsiveness to fair and genuine criticism, and a commitment to evolve our thinking in response to new evidence.

Adjustments to our evaluations process so far

This year, our research team altered the way we approach our evaluations, partially thanks to input from people on this forum in the past. For instance, we have now implemented quantitative scoring frameworks for our Programs and Cost Effectiveness evaluation criteria—a marked departure from how we assessed these criteria last year.

We appreciate the EA community's willingness to provide constructive criticism so that we can continually refine our methods. More recently, some of you have already volunteered your time and expertise by sharing proposals for how ACE can improve. In particular, we thank Nuno Sempere for a th...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Announcing the Progress Forum, published by jasoncrawford on November 17, 2022 on LessWrong.

I’d like to invite you to join the Progress Forum, the new online home for the progress community. It's a clone of this site, but with a focus on progress studies and the philosophy of progress.

This forum was pre-announced in January 2022, and quietly opened in April. Although anyone could sign up, we deliberately didn’t make any big announcement about it, aiming first for a small, high-quality community. Now that we have a lot of good content on the site, we’re announcing it more broadly.

The primary goal of this forum is to provide a place for long-form discussion of progress studies. It’s also, like LW, a place to find local clubs and meetups.

The broader goal is to share ideas, strengthen them through discussion and comment, and over the long term, to build up a body of thought that constitutes a new philosophy of progress for the 21st century (and beyond).

I invite you to post:

Essays (original, or cross-posted from your blog)

Drafts, half-baked ideas, and work-in-progress thinking, for feedback

Questions for brainstorming

Local events and community groups

Etc.

And please read and comment on what others have shared.

You can subscribe to Forum posts via email, RSS, or Twitter.

The Forum is sponsored by The Roots of Progress. Huge thanks to the people who worked to create and run it: Lawrence Kestleoot, Andrew Roberts, Sameer Ismail, David Smehlik, Alec Wilson, and Ross Graham. Thanks also to Kris Gulati for nudging this project along, and to Ruth Grace Wong for helpful conversations about community and moderation. Finally, thanks to the LessWrong team for creating this software platform, and especially to Oliver Habryka, Ruby Bloom, Raymond Arnold, JP Addison, James Babcock, and Ben Pace for answering questions and helping us customize this instance of it.

Go check it out.

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Diversification is Underrated, published by Justis on November 17, 2022 on The Effective Altruism Forum.

Note: This is not an FTX post, and I don't think its content hinges on current events. Also - though this is probably obvious - I'm speaking in a strictly personal capacity.

Formal optimization problems often avail themselves of one solution - there can be multiple optima, but by default there tends to be one optimum for any given problem setup, and the highest expected value move is just to dump everything into that optimum.

As a community, we tend to enjoy framing things as formal optimization problems. This is pretty good! But the thing about formal problem setups is they encode lots of assumptions, and those assumptions can have several degrees of freedom. Sometimes the assumptions are just plain qualitative, where quantifying them misses the point; the key isn't to just add another order-of-magnitude (or three) variable to express uncertainty. Rather, the key is to adopt a portfolio approach such that you're hitting optima or near-optima under a variety of plausible assumptions, even mutually exclusive ones.

This isn't a new idea. In various guises and on various scales, it's called moral parliament, buckets, cluster thinking, or even just plain hedging. As a community, to our credit, we do a lot of this stuff.

But I think we could do more, and be more confident and happy about it.

Case study: me

I do/have done the following things, that are likely EA-related:

Every month, I donate 10% of my pre-tax income to the Against Malaria Foundation.

I also donate $100 to Compassion in World Farming, mostly because I feel bad about eating meat.

In my spare time, I provide editing services to various organizations as a contractor. The content I edit is often informed by a longtermist perspective, and the modal topic is probably AI safety.

I once was awarded (part of a) LTFF (not FTX, the EA Funds one) grant, editing writeups on current cutting-edge AI safety research and researchers.

Case study from a causes perspective

On a typical longtermist view, my financial donations don't make that much sense - they're morally fine, but it'd be dramatically better in expectation to donate toward reducing x-risk.

On a longtermist-skeptical view, the bulk of my editing doesn't accomplish much for altruistic purposes. It's morally fine, but it'd be better to polish general outreach communications for the more legible global poverty and health sector.

And depending on how you feel about farmed animals, that smaller piece of the pie could dwarf everything else (even just the $100 a month is plausibly saving more chickens from bad lives than my AMF donations save human lives), or irrelevant (if you don't care about chicken welfare basically at all).

I much prefer my situation to a more "aligned" situation, where all my efforts go the same single direction.

It's totally plausible to me that work being done right now on AI safety makes a really big difference for how well things go in the next couple decades. It's also plausible to me that none of it matters, either because we're doomed in any case or because our current trajectory is just basically fine.

Similarly, it's plausible to me (though I think unlikely) that I learn that AMF's numbers are super inflated somehow, or that its effectiveness collapsed and nobody bothered to check. And it's plausible that in 20 years, we will have made sufficient progress in global poverty and health that there no longer exist donation opportunities in the space as high leverage as there are right now, and so now is a really important time.

So I'm really happy to just do both. I don't have quantitative credences here, though I'm normally a huge fan of those. I just don't think they work that well for the outside view of the portfolio approach - I've ...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: [Closing Nov 20th] University Group Accelerator Program Applications are Open, published by jessica mccurdy on November 17, 2022 on The Effective Altruism Forum.

This post is a reminder that applications are open for the next round of CEA’s University Group Accelerator Program (UGAP)!

Are you a student at a university without a university group? Consider starting one by applying to UGAP!

The University Group Accelerator Program (UGAP) is a program run by the Centre For Effective Altruism that aims to take a university group from a couple of interested organizers to a mid-sized new EA group. It offers regular meetings with an experienced mentor, trainings, and useful resources to run your first intro seminar (or fellowship) program.

Applications for next semester are open!

Learn more and apply here by 11:59pm ET, on November 20th

We are also looking for additional experienced organizers to serve as mentors. You can find out more about and express interest in becoming a UGAP mentor here.

We have had groups from every populated continent participate and are excited to expand!

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.