The Nonlinear Library

The Nonlinear Library

By The Nonlinear FundEducation
Download on the App Store

All The Nonlinear Library episodes

  • AF - Is Deontological AI Safe? [Feedback Draft] by Dan H
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Is Deontological AI Safe? [Feedback Draft], published by Dan H on May 27, 2023 on The AI Alignment Forum.
    [Note: This post is an excerpt from a longer paper, written during the first half of the Philosophy Fellowship at the Center for AI Safety. I (William D'Alessandro) am a Postdoctoral Fellow at the Munich Center for Mathematical Philosophy. Along with the other Philosophy Fellowship midterm projects, this draft is posted here for feedback.The full version of the paper includes a discussion of the conceptual relationship between safety and moral alignment, and an argument that we should choose a reliably safe powerful AGI over one that's (apparently) successfully morally aligned. I've omitted this material for length but can share it on request.The deontology literature is big, and lots of angles here could be developed further. Questions and suggestions much appreciated!]
    1 Introduction
    Value misalignment arguments for AI risk observe that artificial agents needn’t share human ideas about what sorts of ends are intrinsically good and what sorts of means are morally permissible. Without such values for guidance, a powerful AI might turn its capabilities toward human-unfriendly goals. Or it might pursue the objectives we’ve given it in dangerous and unforeseen ways. Thus, as Bostrom writes, “Unless the plan is to keep superintelligence bottled up forever, it will be necessary to master motivation selection” (Bostrom 2014, 185). Indeed, since more intelligent, autonomous AIs will be favored by competitive pressures over their less capable kin (Hendrycks 2023), the hope of keeping AI weak indefinitely is probably no plan at all.
    Considerations about value misalignment plausibly show that equipping AIs with something like human morality is a necessary step toward AI safety. It’s natural to wonder whether moral alignment might also be sufficient for safety, or nearly so. Would an AI guided by an appropriate set of ethical principles be unlikely to cause disastrous harm by default?
    This is a tempting thought. By the lights of common sense, morality is strongly linked with trustworthiness and beneficence; we think of morally exemplary agents as promoting human flourishing while doing little harm. And many moral systems include injunctions along these lines in their core principles. It would be convenient if this apparent harmony turned out to be a robust regularity.
    Deontological morality looks like an especially promising candidate for an alignment target in several respects. It’s perhaps the most popular moral theory among both professional ethicists and the general public. It looks to present a relatively tractable technical challenge in some respects, as well-developed formal logics of deontic inference exist already, and large language models have shown promise at classifying acts into deontologically relevant categories (Hendrycks et al. 2021). Correspondingly, research has begun on equipping AIs with deontic constraints via a combination of top-down and bottom-up methods (Kim et al. 2021). Finally, deontology appears more inherently safety-friendly than its rivals, since many deontological theories posit strong harm-avoidance principles. (By contrast, standard forms of consequentialism recommend taking unsafe actions when such acts maximize expected utility.
    Adding features like risk-aversion and future discounting may mitigate some of these safety issues, but it’s not clear they solve them entirely.)
    I’ll argue that, unfortunately, deontological morality is no royal road to safe AI. The problem isn’t just the trickiness of achieving complete alignment, and the chance that partially aligned AIs will exhibit risky behavior. Rather, there’s reason to think that deontological AI might pose distinctive safety risks of its own. This suggests that existential catastrophe...
    48 min
  • EA - By failing to take serious AI action, the US could be in violation of its international law obligations by Cecil Abungu
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: By failing to take serious AI action, the US could be in violation of its international law obligations, published by Cecil Abungu on May 27, 2023 on The Effective Altruism Forum.
    “Long-term risks remain, including the existential risk associated with the development of artificial general intelligence through self-modifying AI or other means”.
    2023 Update to the US National Artificial Intelligence Research and Development Strategic Plan.
    Introduction
    The United States is yet to take serious steps to govern the licensing, setting up, operation, security and supervision of AI. In this piece I suggest that this could be in violation of its obligations under Article 6(1) of the International Covenant on Civil and Political Rights (ICCPR). By most accounts, the US is the key country in control of how quickly we have artificial general intelligence (AGI), a goal that companies like OpenAI have been very open about pursuing. The fact that AGI could carry risk to human life has been detailed in various fora and I won’t belabor that point. I present this legal argument so that those trying to get the US government to take action have additional armor to call on.
    A. Some important premises
    The US signed and ratified the ICCPR on June 8 1992.[1] While it has not ratified the Optional Protocol allowing for individual complaints against it, it did submit to the competence of the Human Rights Committee (the body charged with interpreting the ICCPR) where the party suing is another state. This means that although individuals cannot bring action against the US for ICCPR violations, other states can. As is the case for domestic law, provisions of treaties are given real meaning when they’re interpreted by courts or other bodies with the specific legal mandate to do so. Most of this usually happens in a pretty siloed manner, but international human rights law is famously non-siloed. The interpretive bodies determining international human rights law cases regularly borrow from each other when trying to make meaning of the different provisions before them.
    This piece is focused on what the ICCPR demands, but I will also discuss some decisions from other regional human rights courts because of the cross fertilization that I’ve just described. Before understanding my argument, they’re a few crucial premises you have to appreciate. I will discuss them next.
    (i) All major human rights treaties, including the ICCPR, impose on states a duty to protect life
    In addition to the ICCPR, the African Charter, European Convention and American Convention have all give states a duty to protect life.[2] As you might imagine, the existence of the actual duty is generally undisputed. It is when we get to the specific content of the duty where things become murky.
    (ii) A state’s duty to protect life under the ICCPR can extend to citizens of other countries
    The Human Rights Committee (quick reminder: this is the body with the legal mandate to interpret the ICCPR) has made it clear that this duty to protect under the ICCPR extends not only to activities which are conducted within the territory of the state being challenged but also to those conducted in other places – so long as the activities could have a direct and reasonably foreseeable impact on persons outside the state’s territory. The fact that the US has vehemently disputed this understanding[3] does not mean it is excused from abiding by it.
    (iii) States’ duties to protect life under the ICCPR require attention to the activities of corporate entities headquartered in their countries
    Even though the US protested the move,[4] the Human Rights Committee has been clear that the duty to protect extends to protecting individuals from violations by private persons or entities,[5] including activities by corporate entities based in their territory or subjec...
    17 min
  • EA - Co-found an incubator for independent AI Safety researchers! by Alexandra Bos
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Co-found an incubator for independent AI Safety researchers!, published by Alexandra Bos on May 26, 2023 on The Effective Altruism Forum.
    Full-time, remote
    APPLY HERE
    Deadline: Thursday, June 8th (in your timezone)
    If your ideal job would be leading an impact-driven organization, being your own boss and pushing for a safer future with AI, you might be a great fit for co-founding Catalyze Impact!
    Below, you will find out more about Catalyze’s mission and focus, why co-founding this org would be high-impact, how to tell if you’re a good fit, and how to apply.
    In short, Catalyze will 1) help people become independent technical AI Safety researchers, and 2) deliver key support to independent AI Safety researchers so they can do their best work.
    If you think this non-profit’s work could be important, please like/upvote and share this message so that the right people get to see it.
    You can ask questions, register interest to potentially fund us, work with us, make use of our services in the future and share information here.
    Why support independent AI Safety researchers?
    Lots of people want to do AI Safety (AIS) research and are trying to get in a position where they can, yet only around 100-300 people worldwide are actually doing research in this crucial area. Why? Because there are almost no AIS researcher jobs available due to AIS research organizations facing difficult constraints to scaling up. Luckily there is another way to grow the research field: having more people do independent research (where a self-employed individual gets a grant, usually from a fund).
    There is, however, a key problem: becoming and being a good independent AIS researcher is currently very difficult. It requires a lot of qualities which have nothing to do with being able to do good research: you have to be proactive, pragmatic, social, good enough at fundraising, very good at self-management and willing to take major career risks. Catalyze Impact will take away a large part of the difficulties that come with being an independent researcher, thereby making it a suitable option for more people so they are empowered to do good AIS research.
    How will we help?
    This is the current design of the pilot - but you will help shape this further!
    1. Fundraising support
    > help promising individuals get funded to do research
    2. Peer support networks & mentor-matching
    > get feedback, receive mentorship, find collaborators, brainstorm and stay motivated rather than falling into isolation
    3. Accountability and coaching
    > have structure, stay motivated and productive
    4. Fiscal sponsorship: hiring funded independent researchers as ‘employees’
    > take away operational tasks which distract from research & help them build better career capital through institutional affiliation
    In what ways would this be impactful?
    Alleviating a bottleneck for scaling the AIS research field by making independent research suitable for more people: it seems that we need a lot more people to be working on solving alignment. However, talented individuals who have invested in upskilling themselves to go do AIS research (e.g. SERI MATS graduates) are largely unable to secure research positions. This is oftentimes not because they are not capable enough of doing the research, but because there are simply too few positions available (see footnote). Because of this, many of these talented individuals are left with a few sub-optimal options. 1) try to do research/a PhD in a different academic field in hopes that it will make them a better AIS researcher in the future, 2) take a job working on AI capabilities (!), or 3) try to become an independent AIS researcher.
    For many people, independent research (i.e. without this incubator) is not a good & viable option because being an independent researcher brings a lot of difficulties with it and arran...
    11 min
  • EA - [Linkpost] Longtermists Are Pushing a New Cold War With China by Mohammad Ismam Huda
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: [Linkpost] Longtermists Are Pushing a New Cold War With China, published by Mohammad Ismam Huda on May 27, 2023 on The Effective Altruism Forum.
    Jacob Davis, a writer for the socialist political magazine Jacobin, raises an interesting concern about how current longermist initiatives in AI Safety are in his assessment escalating tensions between the US and China. This highlights a conundrum for the Effective Altruism movement which seeks to advance both AI Safety and avoid a great power conflict between the US and China.
    This is not the first time this conundrum has been raised which has been explored on the forum previously by Stephen Clare.
    The key points Davis asserts are that:
    Longtermists have been key players in President Biden’s choice last October to place heavy controls on semiconductor exports.
    Key longtermist figures advancing export controls and hawkish policies against China include former Google CEO Eric Schmidt (through Schmidt Futures and the longtermist political fund Future Forward PAC), former congressional candidate and FHI researcher Carrick Flynn, as well as other longtermists in key positions at Gerogetown Center for Security and Emerging Technology and the RAND Corporation.
    Export controls have failed to limit China's AI research, but have wrought havoc on global supply chains and seen as protectionist in some circles.
    I hope this linkpost opens up a debate about the merits and weaknesses of current strategies and views in longtermist circles.
    Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.
    2 min
  • EA - How to evaluate relative impact in high-uncertainty contexts? An update on research methodology & grantmaking of FP Climate by jackva
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: How to evaluate relative impact in high-uncertainty contexts? An update on research methodology & grantmaking of FP Climate, published by jackva on May 26, 2023 on The Effective Altruism Forum.
    1/ Introduction
    We recently doubled our full-time climate team (hi Megan!), and we are just going through another doubling (hiring a third researcher, as well as a climate communications manager, job ad for the latter coming soon, for now reach out to [email protected]).
    Apart from getting a bulk rate for wedding cake, we thought this would be a good moment to update on our progress and what we have in the pipeline for the next months, both in terms of research to be released as well as grantmaking with the FP Climate Fund and beyond.
    As discussed in the next section, If you are not interested in climate, but in EA grantmaking research in general, we think it still might be interesting reading. Being part of Founders Pledge and the effective altruist endeavor at large, we continually try to build tools that are useful for applications outside the narrow cause area work – for example, some of the methodology work on impact multipliers has also been helpful for work in other areas, such as global catastrophic risks (here, as well as FP's Christian Ruhl's upcoming report on the nuclear risk landscape) and air pollution. Another way to put this is that we think of our climate work as one example of an effective altruist research and grantmaking program in a “high-but-not-maximal-uncertainty” environment, facing and attacking similar epistemic and methodological problems as, say, work on great power war, or risk-neutral current generations work. We will come back to this throughout the piece.
    In what follows, this update is organized as follows: We first describe the fundamental value proposition and mission of FP Climate (Section 2). We then discuss, at a high level, the methodological principles that flow from this mission (Section 3), before making this much more concrete with the discussion of three of the furthest developed research projects putting this into action (Section 4). This is the bulk of this methodology-focused-update. We then briefly discuss grantmaking plans (Section 5) and backlog (Section 6) before concluding (Section 7).
    2/ The value proposition and mission of FP Climate
    As part of Founders Pledge’s research team, the fundamental goal of FP Climate is to provide donors interested in maximizing the impact of their climate giving with a convenient vehicle to do so – the Founders Pledge Climate Fund. Crucially, and this is often misunderstood, our goal is not to serve arbitrary donor preferences but rather to guide donors to the most impactful opportunities available.. Taking caring about climate as given, we seek to answer the effective altruist question of what to prioritize.We are conceiving of FP Climate as a research-based grantmaking program to find and fund the best opportunities to reduce climate damage.
    We believe that at the heart of this effort has to be a credible comparative methodology to estimate relative expected impact, fit for purpose to the field of climate where a layer of uncertainties about society, economy, techno-economic factors, and the climate system, as well as a century-spanning global decarbonization effort. This is so because we are in a situation where causal effects and theories of change are often indirect and uncertainty is often irreducible on relevant time-frames (we discuss this more in our recent 80K Podcast (throughout links to 80K link to specific sections of the transcript), as well as Volts, and in our Changing Landscape report).
    While we have been building towards such a methodology since 2021 our recent increase in resourcing is quickly narrowing the gap between aspiration and reality. Before describing some exe...
    31 min
  • EA - Bandgaps, Brains, and Bioweapons: The limitations of computational science and what it means for AGI by titotal
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Bandgaps, Brains, and Bioweapons: The limitations of computational science and what it means for AGI, published by titotal on May 26, 2023 on The Effective Altruism Forum.
    [In this post I discuss some of my field of expertise in computational physics. Although I do my best to make it layman friendly, I can't guarantee as such. In later parts I speculate about other fields such as brain simulation and bioweapons, note that I am not an expert in these subjects.]
    In a previous post, I argued that a superintelligence that only saw three frames of a webcam would not be able to deduce all the laws of physics, specifically general relativity and Newtonian gravity. But this specific scenario would only apply to certain forms of boxed AI.
    Any AI that can read the internet has a very easy way to deduce general relativity and all our other known laws of physics: look it up on wikipedia. All of the fundamental laws of physics relevant to day to day life are on there. An AGI will probably need additional experiments to deduce a fundamental theory of everything, but you don’t need that to take over the world. The AI in this case will know all the laws of physics that are practically useful.
    Does this mean that an AGI can figure out anything?
    There is a world of difference between knowing the laws of physics, and actually using the laws of physics in a practical manner. The problem is one that talk of “solomonoff induction” sweeps under the rug: Computational time is finite. And not just that. Compared to some of the algorithms we’d like to pull off, computational time is miniscule.
    Efficiency or death
    The concept of computational efficiency is at the core of computer science. The running of computers costs time and money. If we are faced with a problem, we want an algorithm to find the right answer. But just as important is figuring out how to find the right answer in the least amount of time.
    If your challenge is “calculate pi”, getting the exact “right answer” is impossible, because there are an infinite number of digits. At this point, we are instead trying to find the most accurate answer we can get for a given amount of computational resources.
    This is also applicable to NP-hard problems. Finding the exact answer to the travelling salesman problem for large networks is impossible within practical resource limits (assuming P not equal NP). What is possible is finding a pretty good answer. There’s no efficient algorithm for getting the exact right route, but there is one for guaranteeing you are within 50% of the right answer.
    When discussing AI capabilities, the computational resources available to the AI are finite and bounded. Balancing accuracy with computational cost will be fundamental to a successful AI system. Imagine an AI that, when asked a simple question, starts calculating an exact solution that would take a decade to finish. We’re gonna toss this AI in favor of one that gives a pretty good answer in practical time.
    This principle goes double for secret takeover plots. If computer model A spends half it’s computational resources modelling proteins, while computer model B doesn’t, computer model A is getting deleted. Worse, the engineers might start digging in to why model A is so slow, and get tipped off to the plot. All this is just to say: computational cost matters. A lot.
    A taste of computational physics
    In this section, I want to give you a taste of what it actually means to do computational physics. I will include some equations for demonstration, but you do not need to know much math to follow along. The subject will be a very highly studied problem in my field called the “band gap problem”.
    “band gap” is one of the most important material properties in semiconductor physics. It describes whether there is a slice of possible energy values that are forbidden ...
    29 min
  • LW - Why I'm Not (Yet) A Full-Time Technical Alignment Researcher by NicholasKross
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Why I'm Not (Yet) A Full-Time Technical Alignment Researcher, published by NicholasKross on May 25, 2023 on LessWrong.
    I have bills to pay (rent and food). For those, I need money. Most full-time technical alignment researchers solve this problem by looking for outside funding. (I.e., most researchers are not independently wealthy; they go to someone else to afford to work on alignment full-time.)
    To get funding in alignment, you generally apply for a grant or a job. In both cases, anyone who'd give you those things will want to see evidence, beforehand, that you know what you're doing. To a near-tautological degree, this evidence must be "legible".
    How do I make my skills/ideas legible? The recommended route is to write my ideas on a blog/LessWrong, read and interact with other alignment materials, and then... eventually it's enough, I assume. For reasons I may/not write about in the near future, many ideas about alignment (especially anything that could be done with today's systems) could very well accelerate capabilities work. There are at least some types of alignment research that are also easy to use for increasing capabilities. Since I'm especially interested in John Wentworth's "abstraction" ideas, anything good I come up might also be like that. In other words legibility and security conflict at least some of the time, especially on the sorts of ideas I'm personally likely to have.
    OK, fine, maybe it's hard to publish legible non-exfohazardous original/smart thoughts about AI alignment. Luckily, funders and hire-ers don't expect everyrone coming in to have already published papers! Perhaps I can simply demonstrate my skills, instead?
    Firstly, many technical skills relevant to AI alignment are hard to demonstrate efficiently. Say you develop a cool new ML algorithm. Did you just speed up capabilities? Okay, just take public notes on a large amount of technical reading... but that mostly signals conscientiousness, not research Talent:tm:! Well, how about you do another technical project... well, now you're wasting precious time that maybe should've been spent on original research. How long are your timelines? (This also applies to the "become independently wealthy to fund yourself" strategy, only more so. I spent an embarrassingly long time on that route in my spare time...)
    My skills, themselves, are not always legible!
    Some of my skills are legible enough for my resume: I've engineered some software at some companies, I know how to debug things, and I list more in a section below. I'm pretty okay at these things.
    However, I think the stuff I'm best at is currently under-measured. These skills include (but aren't limited to): fast learning (assuming I have the energy and sleep and hopefully a bit of prior exposure to the topic), some technical intuition, absurdly-general knowledge, a good bit of security mindset, having-read-and-understood-most-of-The-Sequences, thinking clearly (again modulo sleep/energy), noticing some things, curiosity, and the oft-mocked-but-probably-underrated "creativity" or "idea generation". I wish there were something like Human Benchmark but for the kinds of "mental motions" needed in AI alignment research.
    Even when my skills are legible, they don't seem to be "world-class" in the way that MIRI or OpenAI seem to select for. I got a Bachelor's degree in Computer Science (with a minor in Mathematics) from RIT, in upstate New York. Is that impressive? I got a math-SAT score of over 700, IIRC, and the paper I got back said I was in the 98th or 99th percentile. Is that interesting? I worked with Tensorflow at an internship, and have learned (and often forgotten) the basics of ML coding in classes and online courses. Is that enough for more theoretical/mathematical/conceptual work in alignment? I list more of these in a section below, bu...
    7 min
  • LW - Request: stop advancing AI capabilities by So8res
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Request: stop advancing AI capabilities, published by So8res on May 26, 2023 on LessWrong.
    Note: this has been in my draft queue since well before the FLI letter and TIME article. It was sequenced behind the interpretability post, and delayed by travel, and I'm just plowing ahead and posting it without any edits to acknowledge ongoing discussion aside from this note.
    This is an occasional reminder that I think pushing the frontier of AI capabilities in the current paradigm is highly anti-social, and contributes significantly in expectation to the destruction of everything I know and love. To all doing that who read this: I request you stop.
    (There's plenty of other similarly fun things you can do instead! Like trying to figure out how the heck modern AI systems work as well as they do, preferably with a cross-organization network of people who commit not to using their insights to push the capabilities frontier before they understand what the hell they're doing!)
    (I reiterate that this is not a request to stop indefinitely; I think building AGI eventually is imperative; I just think literally every human will be killed at once if we build AGI before we understand what the hell we're doing.)
    Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.
    2 min
  • EA - Ingroup Deference by trammell
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Ingroup Deference, published by trammell on May 26, 2023 on The Effective Altruism Forum.
    Epistemic status: yes. All about epistemics
    Introduction
    In principle, all that motivates the existence of the EA community is collaboration around a common goal. As the shared goal of preserving the environment characterizes the environmentalist community, say, EA is supposed to be characterized by the shared goal of doing the most good. But in practice, the EA community shares more than just this abstract goal (let’s grant that it does at least share the stated goal) and the collaborations that result. It also exhibits an unusual distribution of beliefs about various things, like the probability that AI will kill everyone or the externalities of polyamory.My attitude has long been that, to a first approximation, it doesn’t make sense for EAs to defer to each other’s judgment any more than to anyone else’s on questions lacking consensus.
    When we do, we land in the kind of echo chamber which convinced environmentalists that nuclear power is more dangerous than most experts think, and which at least to some extent seems to have trapped practically every other social movement, political party, religious community, patriotic country, academic discipline, and school of thought within an academic discipline on record. This attitude suggests the following template for an EA-motivated line of strategy reasoning, e.g. an EA-motivated econ theory paper:
    Look around at what most people are doing. Assume you and your EA-engaged readers are no more capable or better informed than others are, on the whole; take others’ behavior as a best guess on how to achieve their own goals.
    Work out [what, say, economic theory says about] how to act if you believe what others believe, but replace the goal of “what people typically want” with some conception of “the good”.
    And so a lot of my own research has fit this mold, including the core of my work on “patient philanthropy”[1, 2] (if we act like typical funders except that we replace the rate of pure time preference with zero, here’s the formula for how much higher our saving rate should be). The template is hardly my invention, of course. Another example would be Roth Tran’s (2019) paper on “mission hedging” (if a philanthropic investor acts like a typical investor except that they’ll be spending the money on some cause, instead of their own consumption, here’s the formula for how they should tweak how they invest). Or this post on inferring AI timelines from interest rates and setting one’s philanthropic strategy accordingly.But treating EA thought as generic may not be a good first approximation.
    Seeing the “EA consensus” be arguably ahead of the curve on some big issues—Covid a few years ago, AI progress more recently—raises the question of whether there’s a better heuristic: one which doesn’t treat these cases as coincidences, but which is still principled enough that we don’t have to worry too much about turning the EA community into [more of] an echo chamber all around. This post argues that there is. The gist is simple. If you’ve been putting in the effort to follow the evolution of EA thought, you have some “inside knowledge” of how it came to be what it is on some question. (I mean this not in the sense that the evolution of EA thinking is secret, just in the sense that it’s somewhat costly to learn.)
    If this costly knowledge informs you that EA beliefs on some question are unusual because they started out typical and then updated in light of some idiosyncratic learning, e.g. an EA-motivated research effort, then it’s reasonable for you to update toward them to some extent. On the other hand, if it informs you that EA beliefs on some question have been unusual from the get-go, it makes sense to update the other way, toward the distribution o...
    34 min
  • EA - EA cause areas are likely power-law distributed too by Stian
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: EA cause areas are likely power-law distributed too, published by Stian on May 25, 2023 on The Effective Altruism Forum.
    So there are two pieces of common effective altruist thinking that I think are in tension with each other, but for which a more sophisticated version of a similar view makes sense and dissolves that tension. This means that in my experience, people can see others holding the more sophisticated view, and adopting the simple one without really examining them (and discovering this tension).
    This proposed tension is between two statements/beliefs. The first is the common (and core!) community belief that the impact of different interventions is power-law distributed. This means that the very best intervention is several times more impactful than the almost-best ones. The second is a statement or belief along the lines of "I am so glad someone has done so much work thinking about which areas/interventions would have the most impact, as that means my task of choosing among them is easier", or the extreme one which continues "as that means I don't have to think hard about choosing among them." I will refer to this as the uniform belief.
    Now, there is on the face of it many things to take issue with in how I phrased the uniform belief[1], but I want to deal with two things. 1) I think the uniform belief is a ~fairly common thing to "casually" believe - it is a belief that is easy to automatically form after cursorily engaging with EA topics - and 2) it goes strictly against the belief regarding the power-law distribution of impact.
    On a psychological level, I think people can come to hold the uniform belief when they fail to adequately reflect on and internalise that interventions are power-law distributed. Because once they do, the tension between the power-law belief and the uniform belief becomes clear. If the power-law (or simply a right-skewed distribution) holds, then even among the interventions and cause areas already identified, their true impact might be very different from each other. We just don’t know which ones have the highest impact.
    The holding of the uniform belief is a trap that I think people who don't reflect too heavily can fall into, and which I know because I was in it myself for a while - making statements like "Can't go wrong with choosing among the EA-recommended topics". Now I think you can go wrong in choosing among them, and in many different ways. To be clear, I don't think too many people stay in this trap for too long - EA has good social mechanisms for correcting others' beliefs [2] and I would think that it is often caught early. But it is the kind of thing that I am afraid new or cursory EAs might come away permanently believing: that someone else has already done all of the work of figuring out which interventions are the best.
    The more sophisticated view, and which I think is correct, is that because no one knows ex ante the "true" impact of an intervention, or the total positive consequences of work in an area, you personally cannot, before you start doing the difficult work of figuring out what you think, know which of the interventions you will end up thinking is the most important one. So at first blush - at first encounter with the 80k problem profiles, or whatever - it is fine to think that all the areas have equal expected impact [3]. You probably won't come in thinking this - because you have some prior knowledge - but it would be fine to think. What is not fine would be to (automatically, unconsciously) go on to directly choose a career path among them without figuring out what you think is important, what the evidence for each problem area is, and which area you would be a good personal fit for.
    So newcomers see that EA has several problem areas, and are looking at a wide selection of possible interventions, ...
    5 min