Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Brain Efficiency: Much More than You Wanted to Know, published by jacob cannell on January 6, 2022 on LessWrong.

What if the brain is highly efficient? To be more specific, there are several interconnected key measures of efficiency for physical learning machines:

energy efficiency in ops/J

spatial efficiency in ops/mm^2 or ops/mm^3

speed efficiency in time/delay for key learned tasks

compute/algorithmic efficiency in circuit size and steps for key learned tasks

learning/data efficiency in samples/observations/bits required to achieve a level of compute efficiency, or per unit thereof

Why should we care? Brain efficiency matters a great deal for AGI timelines and takeoff speeds, as AGI is implicitly/explicitly defined in terms of brain parity. If the brain is about 6 OOM away from the practical physical limits of energy efficiency, then roughly speaking we should expect about 6 OOM of further Moore's Law hardware improvement past the point of brain parity: perhaps two decades of progress at current rates, which could be compressed into a much shorter time period by an intelligence explosion - a hard takeoff.

But if the brain is already near said practical physical limits, then merely achieving brain parity in AGI at all will already require using up most of the optimizational slack, leaving not much left for a hard takeoff - thus a slower takeoff.

In worlds where brains are efficient, AGI is first feasible only near the end of Moore's Law (for non-exotic, reversible computers), whereas in worlds where brains are highly inefficient, AGI's arrival is more decorrelated, but would probably come well before any Moore's Law slowdown.

In worlds where brains are ultra-efficient, AGI necessarily becomes neuromorphic or brain-like, as brains are then simply what economically efficient intelligence looks like in practice, as constrained by physics. This has important implications for AI-safety: it predicts/postdicts the success of AI approaches based on brain reverse engineering (such as DL) and the failure of non-brain like approaches, it predicts that AGI will consume compute & data in predictable brain like ways, and it suggests that AGI will be far more like human simulations/emulations than you'd otherwise expect and will require training/education/raising vaguely like humans, and thus that neuroscience and psychology are perhaps more useful for AI safety than abstract philosophy and mathematics.

If we live in such a world where brains are highly efficient, those of us interested in creating benevolent AGI should immediately drop everything and learn how brains work.

Energy

Computation is an organization of energy in the form of ordered state transitions transforming physical information towards some end. Computation requires an isolation of the computational system and its stored information from the complex noisy external environment. If state bits inside the computational system are unintentionally affected by the external environment, we call those bit errors due to noise, errors which must be prevented by significant noise barriers and or potentially costly error correction techniques.

Thermodynamics

Information is conserved under physics, so logical erasure of a bit from the computational system entails transferring said bit to the external environment, necessarily creating waste heat. This close connection between physical bit erasure and thermodynamics is expressed by the Landauer Limit[1], which is often quoted as

Eb>kBln2

However the full minimal energy barrier analysis involves both transition times and transition probability, and this minimal simple lower bound only applies at the useless limit of 50% success/error probability or infinite transition time.

The key transition error probability α is constrained by the bit energy:

α=e−EbkBT[2][3]

Here's a range of ...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: EAA is overinvesting in corporate welfare reforms, published by kato on January 6, 2022 on The Effective Altruism Forum.

Thank you to James Ozden for feedback on this post.

In this post I argue that corporate welfare reforms (CWRs) are relatively overinvested in by the EA side of the animal movement (which I’ll refer to as “EAA” from here on). Specifically, I believe that while CWRs are good, and in fact one of the most promising approaches we have, our enamoration with them leads us to underinvest in other approaches in a way that is suboptimal.

By corporate welfare reforms, I mean instances where an animal protection NGO lobbies a corporation to transition at least part of their purchasing of a particular animal “product” over to that same “product” but raised in less bad conditions. The most common examples of CWRs are cage-free commitments and the Better Chicken Commitment.

The core claims:

CWRs currently command a significant portion of EAA’s resources.

CWRs, while good, have serious limitations.

A pluralistic movement is more likely to end factory farming, and our current investment level in CWRs is stifling the development and refinement of alternative approaches.

Conclusion:

Thus, EAA ought to either 1) grow the pot of funds to invest more in non-CWR approaches, or 2) transition some of its resources from CWRs to other approaches.

Below I explain my evidence for each claim in greater detail, and conclude with suggesting what we ought to do about this argument.

Claim 1: CWRs currently command a significant portion of EAA’s resources.

James Ozden recently calculated that the three major funders in EAA, OpenPhil, the EA Fund (EAF), and ACE, had spent an estimated 60% of their animal welfare grants in the past 2 years on CWRS. This seems like a lot, and I’ll go on to argue it’s probably too high a proportion. For what proportion of institutional EAA money ought to go to CWRs, see Implications below.

There’s one counterargument I want to address here: Even with this much money going to CWRs, there’s still a ton of money in the animal rights (AR) movement beyond EAA that doesn’t go to CWRs. For instance, PETA brought in $64M in donations in 2020, more than double what the three EAA funders distributed in that time, and it’s safe to say that none of PETA’s income went to fund CWRs. Perhaps then it makes sense for EAA to overinvest in CWRs because the other side of the AR movement is underinvesting in them, so it in a way evens out.

I find this argument somewhat compelling, but there is also a somewhat compelling response: if we believe that the EAA side of the animal movement has certain skills that the rest of the AR movement somewhat lacks, such as using and responding to evidence, it may make sense to avoid over-relying on the rest of the AR movement.

Claim 2: CWRs, while good, have serious limitations.

It’s first worth noting that as an approach to help animals, CWRs are pretty awesome: They are probably the most impactful and the most scalable approach the animal movement has found. The fact that the smart people at these EAA granting organizations invest so heavily in them is evidence of this, as is the fact that the percent of chickens in cage-free housing has gone from 6% to at least 28% since 2015. As far as I’m aware, no other approach can claim such an impressive shift. CWRs also seem to pave the way for impressive legislative reforms, such as the European Commission’s recent historic pledge to ban almost all cages and crates for farmed animals.

But CWRs, like any approach, also have their limitations:

Enforcement is not a given: A victory is not truly a victory until it is implemented and enforced. With cage-free, groups seem to have done well with enforcement. For other campaigns, including the Better Chicken Commitment, I’m less clear how well enforcem...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Animal welfare EA and personal dietary options, published by Rob Bensinger on January 5, 2022 on LessWrong.

When I imagine an animal welfare EA group, I imagine views breaking down something like:

50%: If factory farmed animals are moral patients, it's more likely that they have net-negative lives (i.e., it would better for them not to exist, than to live such terrible lives).

50%: If factory farmed animals are moral patients, it's more likely that they have net-positive lives (i.e., their lives may be terrible, but they aren't so lacking in value that preventing the life altogether is a net improvement).

This seems like a super hard question, and not one that changes the importance of working to promote animal welfare, so naively (absent some argument for a more informative prior) it should have a 50/50 split within animal welfare circles.

(Possibly more effort should go into the net-positive view within EA because it's more neglected by animal welfare activists, who tend to be vegns; but the space as a whole is so neglected that I suspect this shouldn't be a large factor.)

Within the "net-negative" camp, in my unanchored "what would I naively expect?" hypothetical, I then imagine dietary preferences breaking down something like:

10%: Approximate vegnism or approximate reducetarianism. ("Approximate" to allow for carve-outs like bivalves and especially-moral animal products. The group generally strongly encourages all members to have at least one carve-out, because bivalves in particular are such a clear case and dietary purity ethics is a risky attractor to avoid.)

10%: Handshake-itarianism.

20%: Boycott-itarianism.

60%: Anything goes. A normal meat-eating diet, optimized only for health and convenience. This is the standard animal welfare EA diet, because EA is generally about optimizing your positive impact on the world, not about purifying your personal actions of any possible negative impact.

The number would be much higher than 60% on strictly utilitarian grounds, but humans aren't strict utilitarians and it makes sense for people working hard on improving animal lives to develop strong feelings about their own personal relationship to factory farming, or to want to self-signal their commitment in some fashion.

Within the "net-positive" camp, I imagine:

10%: Sentience-maximizing diets. If you think animals in factory farms have net-positive lives, then it makes sense to want to increase the number of animals (by eating the most meat-heavy healthy diet possible) while also working to improve their welfare.

10%: Handshake-itarianism.

20%: Boycott-itarianism.

60%: Anything goes.

Handshake-itarianism observes that the ~vegns and the sentience-maximizers are sort of offsetting each other's efforts, and that it can make more sense for Bob the ~Vegn and Alice the Sentience-Maximizer to pair off and each agree to eat a "compromise" diet (e.g., both eat meat but only on the weekend). This has a few nice consequences:

- Both diets are likely to be healthier, because they'll be more nutritionally diverse/balanced. This makes it likelier that both EAs will be more productive and have better lives.

- The outcome can be closer to an optimal trade between Alice and Bob's values, because it's less likely to be constrained by either individual's personal circumstances or limitations.

Maybe Alice has an easier time sticking to her ideal diet than Bob does, but Bob is more confident in his view than Alice is; in that case, a handshake diet can produce an outcome that's closer to a compromise between Alice and Bob's ideal diets, because less-extreme diets are easier to maintain and because the handshake agreement has more adjustable parameters. (Like, Alice is lactose-intolerant so Bob drinks more milk on Alice's behalf.)

- Trades like this are likely to have positive social and ...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: How To Raise Others’ Aspirations in 17 Easy Steps, published by cmessinger on January 6, 2022 on LessWrong.

Title taken from Tyler Cowen: The high-return activity of raising others’ aspirations

This is taken from a twitter thread (which I also wrote up on my website) that I was - aptly for the content - encouraged to put in non-Twitter places. Thanks for the encouragement!

Things you can say to people in ten seconds that sometimes produce insanely outsized effects:

Yeah, someone should do that. Why not you?

Is there something you could do about that problem in the next five minutes?

That's a great thought - have you written it up somewhere? I'd be excited to share it if so.

Should you write a book / blog?

You want to do that but don't think you will? Do you want to make a concrete plan now?

Do you want me to ask in a week / month if you've done that or how it's going?

Feeling stuck sucks. Want to brainstorm together?

Feeling stuck sucks. Have you spent a five minute timer generating options?

What's the twenty minute / minimum viable product version of this overwhelming-feeling thing?

Do you want me to sit with you while you fill out that application/write that email? Too bad, it's happening.

Is it worth just asking / cold emailing and seeing?

Is there anyone I know who you'd like to be introduced to?

Do you want to just set an alarm on your phone now as a reminder? (from Damon Sasi)

Do you know anyone else who might have struggled with or succeeded at that? Have you talked to them about it? (from Damon Sasi)

Who do you know who you could ask for help from?

You are Allowed (to try, to do, to build, to ask). I strongly recommend this link of things you are allowed to do.

(Or do you not need permission in the first place?)

Do you sort of already know what you're going to do / have your mind made up about this?

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Think about EA alignment like skill mastery, not cult indoctrination, published by electroswing on January 5, 2022 on The Effective Altruism Forum.

People doing meta-EA sometimes jokingly frame their work as “I want to indoctrinate people into EA, sort of like what cults do, but don’t worry! Haha! What we do is fine because EA isn’t a cult.”

I think this is a harmful mindset. When thinking about how to get more people to be EA-aligned, I think skill mastery is a better model than cult indoctrination.

Skill mastery often has these attributes:

Repeated deliberate practice.

“Thinking about it in the shower”, i.e. thinking about it without much effort.

“Gears-level understanding”, i.e. knowing the foundations of the skill and understanding how all the pieces relate.

Cult indoctrination is more about gaslighting people into believing that there can be no other truth than X. They do this by repeating talking points and creating closed social circles.

Accordingly, when thinking about how to get more people to be EA-aligned, here are some good questions to ask:

Can we build structures which enable repeated deliberate practice of EA? Good examples are intro fellowships, book recommendations, and club meetings. Are there more?

Can we get people to “think about EA in the shower”? One way to improve this could be to provide better-written reading materials which pose questions which are amenable to shower thoughts.

Can we encourage more “gears-level understanding” of EA concepts? For example, emphasize the reasons behind x-risk calculations rather than their conclusions.

It is also probably a bad idea for EA to resemble a cult, because cults have bad epistemics. Accordingly, here are some paths to avoid going down:

Repeating talking points: when discussing EA topics with a skeptical non-EA, don’t repeat standard EA talking points if they’re not resonating. It is useless to say “AI Safety is a pressing problem because superintelligent AGI may pose existential risk” if they do not believe superintelligent AGI could ever possibly exist. Instead, you can have a more intellectually honest conversation by first understanding what their current worldview and model of AI is, and building off of this. In other words it is important to adopt good pedagogy: building from the student's foundations, rather than instructing them to memorize isolated facts.

Closed social circles: for example, in the setting of a university group, it is probably a bad idea to create an atmosphere where people new to EA feel out of place.

The central idea here is that promoting gears-level understanding of EA concepts is important. Gears-level understanding often has repeated deliberate practice and shower thoughts as a prerequisite, so skill mastery and gears-level understanding are closely related goals.

I would rather live in a world with people who have their own sound models of x-risk and other pressing problems, even if they substantially differ from the standard EA viewpoint, than a world of people who are fully on board with the standard EA viewpoints but don’t have a complete mastery of the ideas behind them.

Summary: People who try to get more people to be EA-aligned often use techniques associated with cult indoctrination, such as repeating talking points and creating closed social circles. Instead, I think it is more useful to think about EA-alignment as a skill that a person can master. Accordingly, good techniques to employ are repeated deliberate practice, "thinking about it in the shower", and promoting gears-level understanding.

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Prioritization when size matters, published by jh on January 5, 2022 on The Effective Altruism Forum.

TL;DR: I describe some key considerations for prioritization when the potential size of a funding opportunity matters (i.e. its 'room for more funding'). I show how this relates to the ITN framework and I propose new prioritization components, including 'Scalability', as clarification of Tractability, and several different components of 'Neglectedness'.

It would be nice if all we needed to know when prioritizing funding opportunities was their marginal good per dollar. In reality, funders always need to make two decisions: what to fund and how much.

Research, evaluation and execution takes time and effort. When we find a great opportunity we should be even more thrilled if we can put many dollars into it rather than just one. Scalability matters.

However, if we start throwing a lot of dollars at an opportunity then other funders are going to react. Coordination is hard and our dollars are likely to crowd out or 'funge' others. The extent to which this is an issue depends, among other things, on the Neglectedness of the opportunity.

These considerations, and others, arise when we think about the interaction of our funding decisions with others (i.e. as part of an 'equilibrium'). They are even more relevant when we have the ability to put so much funding into an opportunity that it changes the opportunity itself (i.e. is 'non-marginal').

In this post I share a framework that we have developed at the Total Portfolio Project (TPP) to account for these considerations. I'll refer to this as the 'framework'. But, note that it's just one tool among many that we think are useful for assessing opportunities.

We developed this framework out of necessity because of our goal to develop tools for prioritization that work across a wide range of funding opportunities. And when size matters. Now, with the increasing wealth and ambitions of EA, it seems like these considerations are more relevant than ever to the community at large.

Comments, questions and feedback are very welcome.

See this post's twin for an explanation of both ITN and TPP's framework using a mathematical model.

Review of ITN

The Importance, Tractability and Neglectedness (ITN) style frameworks are important tools for estimating the 'value of allocating marginal resources to a problem'. 80,000 hours describes the components of their ITN-style framework as follows:

Importance - If we solved the problem, how good would it be?

Tractability - If we doubled the resources dedicated to solving this problem, what fraction of the problem would we expect to solve?

Neglectedness - How many resources are already going towards solving this problem?

I think these are the right questions, when interpreted carefully, for assessing the value of adding marginal resources to a problem.

However, there is a good deal of confusion and disagreement about how the components should be assessed in practice (e.g. see most of the posts here). Diversity of thought is a good thing. But it seems like many of these disagreements are unnecessary.

Differences of opinion, confusion and debate applies to all three ITN components. But, in my experience, it applies the most to Neglectedness. As noted here: "Ideally, we would like to not simply select causes that are neglected, but select causes that are neglected for reasons other than their impact."

I think these issues are a symptom of ITN not addressing the question of why an opportunity exists in the first place. Thinking about impact in 'equilibrium' requires tackling this question. So, it is a valuable exercise just on this count alone. It also naturally leads to multiple considerations that capture different aspects of what we might mean by 'neglectedness'.

Equilibrium effects

TPP's framework can...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: GovAI Annual Report 2021, published by GovAI on January 5, 2022 on The Effective Altruism Forum.

A Note from Ben Garfinkel, Acting Director

2021 was a year of big changes for GovAI.

Most significantly, we spun out from the University of Oxford and adopted a new non-profit structure. At the same time, after five years within the organisation, I stepped into the role of Acting Director. Our founder, Allan Dafoe, shifted into the role of President. We also created a new advisory board consisting of several leaders in AI governance: Allan, Ajeya Cotra, Tasha McCauley, Toby Ord, and Helen Toner.

Successfully managing these transitions was our key priority for the past year. Over a period of several months — through the hard work, particularly, of Markus Anderljung (Head of Policy), Alexis Carlier (Head of Strategy), and Anne le Roux (Head of Operations) — we completed our departure from Oxford, arranged a new fiscal sponsorship relationship with the Centre for Effective Altruism, revised our organisation chart and management structures, developed a new website and new branding, and fundraised $3.8 million to support our activities in the coming years.

Our fundamental mission has not changed: we are still building a global research community, dedicated to helping humanity navigate the transition to a world with advanced AI. However, these transitions have served as a prompt to reflect on our activities, ambitions, and internal structure. One particularly significant implication of our exit from Oxford has been an increase in the level of flexibility we have in hiring, fundraising, operations management, and program development. In the coming year, as a result, we plan to explore expansions in our field-building and policy development work. We believe we may be well-placed to organise an annual conference for the field, to offer research prizes, and to help bridge the gap between AI governance research and policy.

Although 2021 was a transitional year, this did not prevent members of the GovAI team from producing a stream of new research. The complete list of our research output – given below – contains a large volume of research targeting different aspects of AI governance. One piece that I would like to draw particular attention to is Dafoe et al.’s paper Open Problems in Cooperative AI, which resulted in the creation of a $15 million foundation for the study of cooperative intelligence.

Our key priority for the coming year will be to grow our team and refine our management structures, so that we can comfortably sustain a high volume of high-quality research while also exploring new ways to support the field. Hiring a Chief of Staff will be a central part of this project. We are also currently hiring for Research Fellows and are likely to open a research management role in the future. Please reach out to [email protected] if you think you might be interested in working with us.

I’d like to close with a few thank-yous. First, I would like to thank our funders for their generous support of our work: Open Philanthropy, Jaan Tallinn (through the Survival and Flourishing Fund), DALHAP Investments Limited (through Effective Giving), the Long-Term Future Fund, and the Center for Emerging Risk Research. Second, I would like to thank the Centre for Effective Altruism for its enormously helpful operational support. Third, I would like to thank the Future of Humanity Institute and the University of Oxford for having provided an excellent initial home for GovAI.

Finally, I would like to express gratitude for everyone who’s decided to focus their career on AI governance. It has been incredible watching the field’s growth over the past five years. From many conversations, it is clear to me that many of the people now working in the field have needed to pay significant upfront costs...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Movie review: Don't Look Up, published by Sam Marks on January 4, 2022 on LessWrong.

[There have already been some reviews of Don’t Look Up, for example Quinn Dougherty's, Nicholas Kross's, and Scott Alexander's. I’m posting this anyway because I think I say pretty different things than these reviews; in particular, my impression of the movie was much more positive.]

[Epistemic status: trying to extract morals about x-risk from an allegory, but beware that some parts might accidentally generalize from fictional evidence.]

[Spoilers for Don’t Look Up on the level of: I tell you the entire plot.]

I think of Don’t Look Up as being divided into two parts.

The first part is about a relatively straightforward x-risk scenario: a grad student discovers a comet which will impact the Earth in 6 months, there’s some issues with coordination and getting the powers-the-be to take the problem seriously, but eventually the relevant players recognize the direness of the threat and launch rockets to deflect the comet. But then there’s a twist: actually the comet has lodes of valuable ore, $140 trillion of it. The deflection mission is aborted, and the second part of the movie begins.

The second part represents a much more interesting x-risk scenario: now some players have an incentive to trade off [probability of averting the x-risk] for money. This is more dire, and much more reflective of the likely x-risk scenarios we could face, especially AGI. Accordingly, where the movie’s society managed to pull itself together enough to avert the simple comet scenario in part 1, their levels of coordination and competence are not high enough to avert the modified comet-with-massive-economic value scenario in part 2. The comet impacts Earth and everyone dies.

The transition between parts 1 and 2 is kinda awkward. In the middle of the deflection mission – the launch has gone off flawlessly, the rockets are in the air, and people are already celebrating – someone informs the president that actually the comet is extremely valuable, and they abort the mission. It sort of seems like the movie is trying to communicate something like “okay good job, you defeated the comet on easy mode, but now let’s change things up.” In my mindcanon, the deflection mission goes off perfectly, the Earth is saved, and the movie ends in triumph. Then a new movie starts, Don’t Look Up 2, with the modified premise. I think this is a less confusing way to think about the story. Below I’ll talk about part 1 as if we saw it end in victory, because I think we’re supposed to imagine that happened.

Don’t Look Up: General Thoughts

Before I talk about parts 1 and 2 in more detail, here are some general thoughts about the movie.

How much you enjoy Don’t Look Up probably depends strongly on your expectations going in. I had very low expectations. I expected the movie to present a caricature of x-risk response along the lines of “good guys who want to save the world vs bad guys who are too stupid/greedy/evil to give a shit.” I expected it to be just grounded-in-reality enough to rile me up and provoke public conversation. And I expected it to be ungrounded enough to hit all the wrong notes and ensure the public conversation was infuriating and useless.

Instead I was very pleasantly surprised. Though the movie was necessarily a caricature, it felt like a very realistic caricature. This is probably because it draws heavily on things that Actually Happened during the COVID pandemic. Accordingly, the social and informational dynamics felt very plausible, as did the ways that the people in charge failed (or, sometimes, succeeded).

Relatedly, it seems that many people disliked DLU because they thought it was a one-sided attack on republicans/the media/some other cheap target. But in contrast I thought that DLU depicted everyone failing i...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Consider trying the ELK contest (I am), published by Holden Karnofsky on January 5, 2022 on The Effective Altruism Forum.

Starting this week, there's a contest (with cash prizes) for playing what I'll call the "ELK [eliciting latent knowledge] game": trying to think of ways to get an AI to "tell the truth" in a contrived hypothetical situation. To put the hypothetical problem very briefly (more detail in an appendix, and still more at the the full writeup on ELK):

A diamond sits in a vault, and there's an AI operating a complex set of controls that can spring traps and otherwise try to stop a robber from taking the diamond. We want to train the AI, via rewarding1 the right behavior during a trial-and-error process,2 to tell the truth about whether the robber took the diamond.

A challenge here is that sometimes the robber (or the AI) might tamper with the cameras we're using to check what actually happened in the vault, leaving us unable to know whether the AI is telling the truth about what happened. We're looking for a process - and a procedure for rewarding the AI - that will nonetheless tend (across many trials) to reward the AI more for (a) actually telling the truth about what happened than for (b) making us think (including via deception) that it has told the truth.

It could be hard to distinguish (a) and (b) empirically, or to give rewards that are clearly for (a) and not (b), because successful deception looks like truth-telling to us - an issue that could pose challenges for training an AI in many possible real-world scenarios. (Unlike today's AIs, advanced AIs could become capable of reliably, successfully deceiving humans.) Much of the writeup is about potential differences in the structure of an AI's calculation for "telling the truth about what happened" vs. "inferring how a particular observer perceives what happened" that could be used to design a reward system that favors the former.

(I give a bit more of a detailed description of the problem in the appendix below. The writeup on ELK has the most detail.)

The contest exists mostly to find potential researcher hires for ARC, a ~1-year-old AI alignment organization currently consisting of Paul Christiano and Mark Xu.3 But here, I'm going to make the case that even if you aren't going to end up working at ARC, there's a lot to be gained from trying to play the game / enter the contest. I personally have put a fair amount of time into this, and I'd like to pitch readers to do so as well.

Before I continue, I want to avoid overselling this:

This isn't a fun game the way Minecraft is, and I expect most people who play to spend more than half their time trying to understand the rules. Playing the ELK game means trying to understand an extremely dense, somewhat technical writeup that hasn't been optimized (or tested) for clarity, and coming up with ideas to address a pretty vaguely defined problem.

Many people will just lack the background to have any idea what's going on. (If you read the writeup, I think you'll pretty quickly know whether it's "sort of understandable, though I have questions and confusions" or "totally incomprehensible").

I imagine that just about anyone who tries this (regardless of background!) will find it time-consuming, difficult, confusing and possibly frustrating.

I think it will be easier for people to get value out of this the more "technical" they are (comfortable with math, etc.), though I don't think you need to be super technical to get some learning value out of it.

That said, here's my case for putting some serious time into playing the "ELK game." I'll start with a bullet-point summary, then expand below.

I think the general type of research ARC is doing is extremely important to the strategic landscape for AI risk today. In particular, I see progress on research like this a...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: A guided cause prioritisation flowchart, published by JackM on January 3, 2022 on The Effective Altruism Forum.

Overview

I have previously written about the importance of making it as easy as possible for EAs to make a fully-informed decision on preferred cause area, given the potentially astronomical differences in value between cause areas. Whilst one piece of feedback was sceptical about the claim that these vast expected value differences exist, generally feedback agreed that the idea of making cause prioritisation easier, for example by highlighting key considerations that have the biggest effect on choice of preferred cause area, could be high impact.

In light of this I have decided to progress this idea by putting together a first draft cause prioritisation flowchart designed to guide people through the process of cause prioritisation. The flowchart would ideally be accompanied by guidance assisting making informed decisions throughout the flowchart. I haven’t finalised this guidance, although present a sample for one particular decision in the flowchart. At this point I am attempting a proof of concept rather than delivering a final product, and so would welcome feedback on both the idea and the preliminary attempt.

Introducing the flowchart

In the following section you can see my draft flowchart. The flowchart asks individuals ethical and empirical questions that I view as most important in determining which cause area they should focus on. Only cause areas that are accepted as important by a non-negligible proportion of the EA community are included in the flowchart. In addition, some foundational assumptions common to EA are made, including a consequentialist view of ethics in which wellbeing is what has intrinsic value.

A key component of a final flowchart would be accompanying guidance to help individuals make informed decisions as they progress through the flowchart. I have not put together all of this guidance at this stage, however as part of the proof of concept I have attempted to illustrate what this might look like for the “Can we influence the future?” question. My vision for a final flowchart would be being able to click on each box to be guided to easily-digestible reading enabling an informed choice on how to proceed.

In my view, some strengths of this flowchart compared to the main previous attempt include:

It includes an up-to-date set of cause areas: the previous attempt that I have come across is, in my opinion, slightly out of date in that it excludes options that have fairly recently entered the mainstream of EA, such as investing for the future. The previous attempt also includes some cause areas that aren’t typically considered high impact by EAs today, such as education or human rights.

It includes more nuances relevant to longtermist cause areas: my flowchart considers nuances such as the consideration of whether or not one thinks that the future is likely to be good. It has been suggested/hinted that not believing that the future is likely to be good may lead one away from longtermism as one would not want to prolong a bad future. However, there is a class of cause areas that could be said to broadly sit under “Improving quality of the future” that would remain robust to this view, and indeed other views that some people think invalidate longtermism such as a person-affecting view of population ethics.

It includes (very incomplete) guidance that assists with making decisions along the flowchart: my vision for a final flowchart is one that is accompanied by guidance helping one make an informed decision throughout the flowchart. This feels like an essential part of any flowchart - without it one is just making gut judgements on very complex questions for which much discussion has already taken place. This is my main bone to pick with...