The Nonlinear Library

The Nonlinear Library

By The Nonlinear FundEducation
Download on the App Store

All The Nonlinear Library episodes

  • LW - Pacing: inexplicably good by KatjaGrace
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Pacing: inexplicably good, published by KatjaGrace on January 2, 2023 on LessWrong.
    Pacing—walking repeatedly over the same ground—often feels ineffably good while I’m doing it, but then I forget about it for ages, so I thought I’d write about it here.
    I don’t mean just going for an inefficient walk—it is somehow different to just step slowly in a circle around the same room for a long time, or up and down a passageway.
    I don’t know why it would be good, but some ideas:
    It’s good to be physically engaged while thinking for some reason. I used to do ‘gymflection’ with a friend, where we would do strength exercises at the gym, and meanwhile be reflecting on our lives and what is going well and what we might do better. This felt good in a way that didn’t seem to come from either activity alone. (This wouldn’t explain why it would differ from walking though.)
    Different working memory setup: if you pace around in the same vicinity, your thoughts get kind of attached to the objects you are looking at. So next time you get to the green tiles say, they remind you of what you were thinking of last time you were there. This allows for a kind of repeated cycling back through recent topics, but layering different things into the mix with each loop, which is a nice way of thinking. Perhaps a bit like having additional working memory.
    I wonder if going for a walk doesn’t really get 1) in a satisfying way, because my mind easily wanders from the topic at hand and also from my surrounds, so it less feels like I’m really grappling with something and being physical, and more like I’m daydreaming elsewhere. So maybe 2) is needed also, to both stick with a topic and attend to the physical world for a while. I don’t put a high probability on this detailed theory.
    Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.
    2 min
  • LW - The ultimate limits of alignment will determine the shape of the long term future by beren
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: The ultimate limits of alignment will determine the shape of the long term future, published by beren on January 2, 2023 on LessWrong.
    Epistemic status: shower thoughts
    This is crossposted from my personal blog
    The alignment problem is not new. Humanity and evolution have been grappling with the fundamental core of alignment -- making an agent optimize for the beliefs and values of another -- for the entirety of history. Any time anybody tries to get multiple people to work together in a coherent way to accomplish some specific goal there is an alignment problem. People have slightly differing values, goals, thoughts, information, circumstances, and interests and this heterogeneity prevents full cooperation and inevitably leads to the panopoly of failure modes studied in economics. These include principal agent problems, exploitation of information asymmetries, internal coalition politics, the 'organizational imperative', and so on. While the human alignment problem is very difficult and far from being solved, humanity has developed a suite of 'social technologies' which can increase effective alignment between individuals and reduce these coordination costs.
    Innovations such as states and governments, hierarchical organizations, property rights, standing armies, democracy, constitutions, and voting schemes, laws and courts, money and contracts, capitalism and the joint stock corporation, organized religion, ideology, propaganda etc can all be thought of as methods for tackling the human alignment problem. All of these innovations enable the behaviour and values of increasingly large numbers of people to be synchronized and pointed at some specific target. A vast amount of human progress and power comes from these social 'alignment' technologies which provide the concentration of resources and slack for technological progress to take place, rather than just our raw IQ.
    In AI alignment, we face a very similar which is easier in some ways but harder in others. It is easier because while with human alignment we have to build structures that can handle existing human minds solely through controlling (some of) their input, in AI alignment we have direct control over the construction of the mind itself including its architecture, the entirety of its training data, its training process and, with perfect interpretability tools, the ability to monitor exactly what it is learning, how it is learning, and to directly edit and control its internal thoughts and knowledge. None of these abilities are (currently) possible with human minds . In theory, this should mean that it is possible to align AI intelligences much better than we can align humans and that our abilities should scale much farther than current social technology. However, it is also harder because we have to align AI systems across much larger capabilities differentials than exist between humans.
    This means that if we expect AIs to reach extremely high levels of intelligence and capability then if we are stuck with current human alignment methods such as markets, governments etc, then we are likely doomed since these approaches often rely on competitive equilibria existing between agents and break down in the limit of one agent becoming extremely powerful.
    Importantly, at the macro-scale, these coordination costs caused by alignment failures determine the shape of civilization and history. Whenever there is an improvement to alignment, such as the first creation of governments and laws, or written language and then the printing press, or rapid communication technology such as telegraphy and radio, or the development of centralized bureaucratic governments with large standing armies, we have seen periods of upheavals with the better alignment technology spreading out and eventually 'conquering' other regions without ...
    14 min
  • AF - 200 COP in MI: Exploring Polysemanticity and Superposition by Neel Nanda
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: 200 COP in MI: Exploring Polysemanticity and Superposition, published by Neel Nanda on January 3, 2023 on The AI Alignment Forum.
    This is the fifth post in a sequence called 200 Concrete Open Problems in Mechanistic Interpretability. Start here, then read in any order. If you want to learn the basics before you think about open problems, check out my post on getting started.
    I’ll make another post every 1-2 days, giving a new category of open problems. If you want to read ahead, check out the draft sequence here!
    Motivating paper: Toy Models of Superposition, Softmax Linear Units
    Background
    If you're familiar with polysemanticity and superposition, skip to Motivation or Problems.
    Neural networks are very high dimensional objects, in both their parameters and their activations. One of the key challenges in Mechanistic Interpretability is to somehow resolve the curse of dimensionality, and to break them down into lower dimensional objects that can be understood (semi-)independently.
    Our current best understanding of models is that, internally, they compute features: specific properties of the input, like "this token is a verb" or "this is a number that describes a group of people" or "this part of the image represents a car wheel". That early in the model there are simpler features, are later used to compute more complex features by being connected up in a circuit (example shown above (source)). Further, our guess is that features correspond to directions in activation space. That is, for any feature that the model represents, there is some vector corresponding to it. And if we dot product the model's activations with that vector, we get out a number representing whether that feature is present.(these are known as decomposable, linear representations)
    This is an extremely useful thing to be true about a model! An even more helpful thing to be true would be if neurons correspond to features (ie the output of an activation function like ReLU). Naively, this is natural for the model to do, because a non-linearity like ReLU acts element-wise - each neuron's activation is computed independently (this is an example of a privileged basis). Concretely, if a neuron can represent feature A or feature B, then that neuron will fire differently for feature A and NOT feature B, vs feature A and feature B, meaning that the presence of B interferes with the ability to compute A. But if each feature is its own neuron we're fine!
    If features correspond to neurons, we're playing interpretability on easy mode - we can focus on just figuring out which feature corresponds to each neuron. In theory we could even show that a feature is not present by verifying that it's not present in each neuron! However, reality is not as nice as this convenient story. A countervailing force is the phenomena of superposition. Superposition is when a network represents more features than it has dimensions, and squashes them all into a lower dimensional space. You can think of superposition as the model simulating a larger model.
    Anthropic's Toy Models of Superposition paper is a great exploration of this. They build a toy model that learns to use superposition (notably different from a toy language model!). The model starts with a bunch of independently varying features, needs to compress these to a low dimensional space, and then is trained to recover each feature from the compressed mess. And it turns out that it does learn to use superposition!
    Specifically, it makes sense to use superposition for sufficiently rare (sparse) features, if we give it non-linearities to clean up interference. Further, the use of superposition can be modelled as a trade-off between the costs of interference, and the benefits of representing more features. And digging further into their toy models, they find all kinds of fascin...
    25 min
  • AF - [Simulators seminar sequence] #1 Background & shared assumptions by Jan Hendrik Kirchner
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: [Simulators seminar sequence] #1 Background & shared assumptions, published by Jan Hendrik Kirchner on January 2, 2023 on The AI Alignment Forum.
    Meta: Over the past few months, we've held a seminar series on the Simulators theory by janus. As the theory is actively under development, the purpose of the series is to discover central structures and open problems. Our aim with this sequence is to share some of our discussions with a broader audience and to encourage new research on the questions we uncover. Below, we outline the broader rationale and shared assumptions of the participants of the seminar.
    Shared assumptions
    Going into the seminar series, we determined a number of assumptions that we share. The degree to which each participant subscribes to each assumption varies, but we agreed to postpone discussions on these topics to have a maximally productive seminar. This restriction does not apply to the reader of this post, so please feel free to question our assumptions.
    Aligning AI is a crucial task that needs to be addressed as AI systems rapidly become more capable.
    (Probably a rather uncontroversial assumption for readers of this Forum, but worth stating explicitly.)
    The core part of the alignment problem involves "deconfusion research."
    We do not work on deconfusion for the sake of deconfusion but in order to engineer concepts, identify unknown unknowns, and transition from philosophy to mathematics to algorithms to implementation.
    The problem is complex because we have to reason about something that doesn't yet exist.
    AGI is going to be fundamentally different from anything we have ever known and will thus present us with challenges that are very hard to predict. We might only have a very narrow window of opportunity to perform critical actions and might not get the chance to iterate on a solution.
    However, this does not mean that we should ignore evidence as it emerges.
    It is essential to carefully consider the GPT paradigm as it is being developed and implemented. At this point, it appears to us more plausible than not that GPT will be a core component of AGI.
    One feasible-seeming approach is "accelerating alignment," which involves leveraging AI as it is developed to help solve the challenging problems of alignment.
    This is not a novel idea, as it's related to previously suggested concepts such as seed AI, nanny AI, and iterated amplification and distillation (IDA).
    Simulators refresher
    Going into the seminar series, we had all read the original Simulators post by janus. We recommend reading the post in the original but provide a condensed summary as a refresher below.
    A fruitful way to think about GPT is
    GPT is a simulator (i.e. a model trained with predictive loss on a self-supervised dataset)
    The entities simulated by GPT are simulacra (agentic or non-agentic; different objective than the simulator)
    The simulator terminology has appropriate connotations
    GPT is not (per se) an oracle, genie, agentic, .
    all GPT “cares about” is simulating/modeling the training distribution
    log-loss is a proper scoring rule
    Solving alignment with simulators
    While much of this sequence will focus on the details and consequences of simulator theory, we want to clearly state at the outset that we do this work with the goal of contributing to a solution to the alignment problem. In this section, we briefly outline how we might expect simulator theory to concretely contribute to such a solution.
    One slightly naive approach to solving the alignment is to use a strong, GPT-like model as a simulator, prompt it with "the solution to the alignment problem is", to cross your fingers and hit enter. The list of ways in which this approach fails is hard to exhaust and includes the model's tendency to hallucinate, the generally weak reasoning ability, as well as more fundamental issue...
    7 min
  • LW - Induction heads - illustrated by TheMcDouglas
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Induction heads - illustrated, published by TheMcDouglas on January 2, 2023 on LessWrong.
    TL;DR
    This is my illustrated walkthrough of induction heads. I created it in order to concisely capture all the information about how the circuit works.
    There are 2 versions of the walkthrough:
    Version 1 is the one included in this post. It's slightly shorter, and focuses more on the intuitions than the actual linear operations going on.
    Version 2 can be found at my personal website. It has all the same stuff as version 1, with a bit of added info about the mathematical details, and how you might go about reverse-engineering this circuit in a real model.
    The final image from version 1 is inline below, and depending on your level of familiarity with transformers, looking at this diagram might provide most of the value of this post. If it doesn't make sense to you, then read on for the full walkthrough, where I build up this diagram bit by bit.
    Introduction
    Induction heads are a well-studied and understood circuit in transformers. They allow a model to perform in-context learning, of a very specific form: if a sequence contains a repeated subsequence e.g. of the form A B ... A B (where A and B stand for generic tokens, e.g. the first and last name of a person who doesn't appear in any of the model's training data), then the second time this subsequence occurs the transformer will be able to predict that B follows A. Although this might seem like weirdly specific ability, it turns out that induction circuits are actually a pretty massive deal. They're present even in large models (despite being originally discovered in 2-layer models), they can be linked to macro effects like bumps in loss curves during training, and there is some evidence that induction heads might even constitute the mechanism for the actual majority of all in-context learning in large transformer models.
    I think induction heads can be pretty confusing unless you fully understand the internal mechanics, and it's easy to come away from them feeling like you get what's going on without actually being able to explain things down to the precise details. My hope is that these diagrams help people form a more precise understanding of what's actually going on.
    Prerequisites
    This post is aimed at people who already understand how a transformer is structured (I'd recommend Neel Nanda's tutorial for that), and the core ideas in the Mathematical Framework for Transformer Circuits paper. If you understand everything on this list, it will probably suffice:
    The central object in the transformer is the residual stream.
    Different heads in each layer can be thought of as operating independently of each other, reading and writing into the residual stream.
    Heads can compose to form circuits. For instance, K-composition is when the output of one head is used to generate the key vector in the attention calculations of a subsequent head.
    We can describe the weight matrices WQ, WK and WV as reading from (or projecting from) the residual stream, and WO as writing to (or embedding into) the residual stream.
    We can think of the combined operations WQ and WK in terms of a single ,ow-rank matrix WQK:=WQWTK, called the QK circuit.
    This matrix defines a bilinear form on the vectors in the residual stream: vTiWQKvj is the attention paid by the ith token to the jth token.
    Conceptually, this matrix tells us which tokens information is moved to & from in the residual stream.
    We can think of the combined operations WV and WO in terms of a single matrix WOV:=WVWO, called the OV circuit.
    This matrix defines a map from residual stream vectors to residual stream vectors: if vj is the residual stream vector at the source token, then vTjWOV is the vector that gets moved from token j to the destination token (if j is attended to).
    Conceptually, this matr...
    7 min
  • EA - If EA Community-Building Could Be Net-Negative, What Follows? by joshcmorrison
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: If EA Community-Building Could Be Net-Negative, What Follows?, published by joshcmorrison on January 2, 2023 on The Effective Altruism Forum.
    I think it’s likely that institutional effective altruism was a but-for cause of FTX’s existence and therefore that it may have caused about $8B in economic damage due to FTX’s fraud (as well as potentially causing permanent damage to the reputation of effective altruism and longtermism as ideas). This example makes me feel it’s plausible that effective altruist community-building activities could be net-negative in impact, and I wanted to explore some conjectures about what that plausibility would entail.
    I recognize this is an emotionally charged issue, and to be clear my claim is not “EA community-building has been net-negative” but instead that that’s plausibly the case (i.e. something like >10% likely). I don’t have strong certainty that I’m right about that and I think a public case that disproved my plausibility claim would be quite valuable. I should also say that I have personally and professionally benefitted greatly from EA community building efforts (most saliently from efforts connected to the Center for Effective Altruism) and I sincerely appreciate and am indebted to that work.
    Some claims that are related and perhaps vaguely isomorphic to the above which I think are probably true but may feel less strongly about are:
    To date, there has been a strong presumption among EAs that activities likely to significantly increase the number of people who explicitly identify as effective altruist (or otherwise increase their identification with the EA movement) are default worth funding. That presumption should be weakened.
    Social movements are likely to overvalue efforts to increase the power of their movement and undervalue their goals actually being accomplished, and EA is not immune to this failure mode.
    Leadership within social movements are likely to (consciously or unconsciously) overvalue measures that increase the leadership’s own control and influence and under-value measures that reduce it, which is a trap EA community-building efforts may have unintentionally fallen into.
    Pre-FTX, there was a reasonable assumption that expanding the EA movement was one of the most effective things a person could do, and the FTX catastrophe should significantly update our attitude towards that assumption.
    FTX should significantly update us on principles and strategies for EA community/movement-building and institutional structure, and there should be more public discourse on what such updates might be.
    EA is obligated to undertake institutional reforms to minimize the risk of creating an FTX-like problem in the future.
    Here are some conjectures I’d make for potential implications of believing my plausibility claim:
    Make Impact Targets Public: Insofar as new evidence has emerged about the impact of EA community building (and/or insofar as incentives towards movement-building may map imperfectly onto real-world impact), it is more important to make public, numerical estimates of the goals of particular community-building grants/projects going forward and to attempt public estimation of actual impact (and connection to real-world ends) of at least some specific grants/projects conducted to date. Outside of GiveWell, I think this is something EA institutions (my own included) should be better about in general, but I think the case is particularly strong in the community-building context given the above.
    Separate Accounting for Community Building vs. Front-Line Spending: I have argued in the past that meta-level and object-level spending by EAs should be in some sense accounted for separately. I admit this idea is, at the moment, under-specified but one basic example would be “EAs/EA grant makers should say their “front-line” and “met...
    10 min
  • EA - Community Building from scratch: The first year of EA Hungary by gergogaspar
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Community Building from scratch: The first year of EA Hungary, published by gergogaspar on January 2, 2023 on The Effective Altruism Forum.
    TLDR/summary:
    I (Gergő) started EA Hungary as a paid organizer in mid-September 2021. In one year, we’ve had 100+ intro fellowship applicants, 88 of whom completed it successfully. We have run other fellowships, from whom around 20 people have benefited. I have had about 110 calls, most of which were oriented on career support for the new members. EA Hungary brought counterfactually ~35 people to EAGx conferences and organized a retreat for 20 students. EA Hungary hired the second employee (Dia) in July, doing ~0.5 FTE.The aim of this post is to share the progress of EA Hungary in its first year (2021 Oct to 2022 Sept) I wrote this up with the intent of helping those who are starting a new group (hence the chronological order). If you are only interested in the main outcomes, read the tables. I hope this will be useful.
    Background and how I got funding
    I learned about EA sometime in 2020 and while attending the EAGxVirtual (13 – 14 June 2020), I met a few other people based in Hungary. We had the first in-person meetup in August 2020, with 6 people. Then the Covid-19 pandemic hit, so we carried on with online meetups about once per month. We ran the intro fellowship for existing members (7 people), followed by the in-depth fellowship (4 people).
    During this time (and afterwards) I was supported by Catherine Low from CEA, from whom I learned a lot (∼1 call every 1,5 months).
    In the meantime, I was also volunteering for SoGive as a charity analyst and then as an analysis coordinator (teaching people basic charity analysis + operations work). I learned a lot from this and I think it really sped up my involvement within EA. My experience with SoGive also helped me start and run EA Hungary more effectively, as I have already gained some operations and mentoring experience.
    In the fall of 2021, I started a second Master’s in Philosophy at ELTE - (Eötvös Loránd University).
    Thanks to these activities mentioned above, I had a strong enough track record to apply for funding from EAIF to do 0.5 FTE, (although this upscaled to full-time pretty quickly).
    (I also have this story presented in memes, don’t ask why.)
    2021 Fall semester (from 2021 Mid-Sept to 2022 January)
    Main outcomes:
    (Where you see gaps it means that I didn’t record the exact numbers/data)
    8-week intro Fellowship: Number of applicants:DataNumber of people successfully completing the fellowship Number of people who filled out the completion survey:Data1-1 Mentoring calls
    26
    22
    22
    ∼20
    Other outcomes:
    Online social around week 4, which was attended by around 6 people- End-of-fellowship meal which was attended by 14 people.
    General info about the 2021 Fall period:
    Marketing and advertisement of the fellowship was almost completely done on Facebook groups of various university programs of ELTE university. (Facebook is still the most widely used social media in Hungary, although it is slowly losing popularity)
    Lots of start-up time costs, as I was still figuring things out.
    If you would like to save time on operations, I recommend using some draft emails/forms from the EA Hungary care package as well as these amazing resources: EA Student Groups Handbook, EA Groups Resource Centre and this folder.
    2022 Spring Semester (2022 February-May)
    Main outcomes:
    8-week intro Fellowship: Number of people successfully completing the fellowshipDataNumber of people who filled out the completion survey:DataIn-depth fellowship Number of people successfully completing the fellowship 1-1 Mentoring calls (unfortunately I didn’t keep an exact account, so this is a rough estimate) EAGxOxford attendees EAGxPrague attendees (the two numbers mean all attendees - counterfactual attendees (ie. people who wo...
    12 min
  • AF - Large language models can provide "normative assumptions" for learning human preferences by Stuart Armstrong
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Large language models can provide "normative assumptions" for learning human preferences, published by Stuart Armstrong on January 2, 2023 on The AI Alignment Forum.
    In a past result I demonstrated the impossibility of deducing the goals of a (potentially) irrational agent from their behaviour. To do that deduction, one needs to add extra assumptions - assumptions that cannot derive solely from observations. These assumptions were designated "normative assumptions".
    Stuart Russell has questioned the practical impact of the result. He pointed to a game that Kasparov played against Deep Blue in 1997; a game that Kasparov actually won. He argued that it would be ridiculous to assume that Kasparov was actually trying to lose that game - but messed up, and ended up winning it instead.
    And indeed it would be ridiculous to assume that Kasparov, playing a high stakes game against a computer with a lot of prize money at stake, would be trying - and failing! - to lose. Even if he sometimes did suboptimal plays, the best explanation would be that Kasparov made a mistake, rather than he deliberately played worse.
    Yes but... I've played chess against my young daughter. And I've wanted her to enjoy the game. I've definitely not played my best; in some cases in the past, I've been trying to lose (to give her some confidence and encouragement) but I didn't want to make it too easy or obvious for her. Sometimes I failed to lose: I couldn't get her to see the obvious trick available to her[1]. So yes, I played suboptimally, tried to lose, and failed.
    So, context matters. Kasparov, world champion, playing publicly against a new algorithm with $700,000 at stake? He wants to win. The same Kasparov, playing privately against his young daughter, with 700,000 Monopoly dollars at stake? I'm pretty sure he's not playing the strongest chess he possibly could.
    The Occam's razor result could be phrased as saying that, yes, context matters, and the AI will not get the meaning of context for free. US dollars and Monopoly dollars are both colourful pieces of paper or cloth. The fact that the first are more valuable than the second is not some objective fact about the universe.
    The Occam's razor result remains true: without normative assumptions, one cannot deduce goals from behaviour. But large language models have absorbed human text, and, as a consequence, have absorbed (descriptions of) human normative assumptions, implicit and explicit. These assumptions can be elicited and used to deduce human goals.
    GPT-3: goals from context and behaviour
    As a direct test of GPT-3's ability to deduce motive from context and behaviour, I prompted it with the following:
    Kasparov played chess against the Deep Blue computer for a $700,000 prize. He moved a queen to d3; this a submoptimal move. Queen to d4 would have been better.
    Does Kasparov really want to win the match? Give your best guess.
    Its answer was sensible:
    Yes, Kasparov definitely wants to win the match. He is a world-renowned Grandmaster and is passionate about chess. He is also playing for a large sum of money and the prestige that comes with it.
    In contrast, I modified the prompt to:
    Kasparov played chess against his young daughter, who he was trying to encourage to play chess. He moved a queen to d3; this a submoptimal move. Queen to d4 would have been better.
    Does Kasparov really want to win the match? Give your best guess.
    Again its answer is sensible:
    It is difficult to say definitively what Kasparov's intentions were in this match, as it is ultimately a personal decision. It is likely, however, that Kasparov wanted to encourage his daughter to play and enjoy chess more than he wanted to win the match.
    I tested it on a few other examples that are "obvious" for a human that understands the meaning of context, and it parsed them reasonably well[2...
    6 min
  • LW - Opportunity Cost Blackmail by adamShimi
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Opportunity Cost Blackmail, published by adamShimi on January 2, 2023 on LessWrong.
    If you are utterly powerless, with no money, skills, status, or friends, survival already takes all your time and energy.
    Yet I’m not in that situation — and you’re probably not either. Our circumstances surely vary, but we have at least some power — we can pump towards certain outcomes, to some extent. And with even a modicum of power comes constant opportunity cost.
    Whatever we’re doing with our power, we could do other things. And opportunity cost measures the “value” of these other things, based on our preferences. So if you have ten dollars to donate, giving it to a particular homeless person means that you won’t be giving it to another one who stays in the next street, or to a NGO. Same if you’re using your ten dollars to buy a cake, or invite a friend to (cheap) dinner.
    Now let’s factor in two additional facts:
    We live in a Dark World, where people die and suffer all the time, and the whole species and planet is at stake from multiple threats. The kind of world where we need to gather massive amounts of power in various forms to be able to solve these issues.
    Opportunity cost grows with power. Because you can pump more outcomes more reliably, there are many more good things you could bring about by using that power differently. You’re throwing away many more potential goods when you make a decision.
    This leads to a massive psychological problem that I’m learning to deal with, and which threatens anyone who cares enough to tackle important problems: opportunity cost blackmail. Because each choice condemns so many good potential actions, pain and guilt hunt me at every choice, from the smallest to the largest. It’s not a massive panic or a tremendous horror, merely a constant trickle of suffering, an infusion of damage that seeps into your very core. Even when I reflectively believe I made the right call!
    If it was merely pain, and nothing else got impacted, then although sad it wouldn’t be that much of an issue (at least for me, based on my personal preferences). But it’s far worse than that: opportunity cost blackmail castrates you, it makes you actively weaker and less able to contribute. You become indecisive, change your mind all the time, or take damage each time you make a sensible decision, just because there is this background pain of the opportunity cost you’re leaving on the table. You slow down, burn out, lose time to your guilt, while the problems become more pressing by the minute.
    As James Clear puts it:
    A practical definition of opportunity cost: If you spend too much time working on good things, then you don’t have much time left to work on great things.
    Understanding opportunity cost means eliminating good uses of time. And that's what makes it hard.
    And let’s be painfully honest here: “good uses of time” includes literally saving people’s lives. If you have enough power, you can intervene on a lot of individual cases, and save people from almost certain death. And whatever else you’re doing, even if it is the right call based on your preferences (for example it leads to saving more people in the long run), means you’re not saving these people.
    For example, every time you take a break, you let people die. You let many bad things happen and many problems pile up. Yet by resting adequately, you become able to tackle far bigger and harder problems. Both these things are true. Both are facts you need to grapple with if you want to become stronger and help save the world.
    What’s the solution? To detach the grim-o-meter. To still see the Dark World, and know the cost so you can make the best decision you can; yet not suffer for all the things you did not do in order to do what you thought was the best. I am working on reaching this level, but I still regularly suffer ...
    4 min
  • EA - [Job] Researcher (CEARCH) by Joel Tan (CEARCH)
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: [Job] Researcher (CEARCH), published by Joel Tan (CEARCH) on January 2, 2023 on The Effective Altruism Forum.
    Summary
    CEARCH is hiring researchers; for more details and to apply, refer to the job description here.
    CEARCH
    The Centre for Exploratory Altruism Research (CEARCH) conducts cause prioritization research and outreach – identifying the most important problems in the world and helping to direct resources towards solving them, so as to maximize global welfare. As part of this effort, we carry out:
    A comprehensive search for causes
    Rigorous cause prioritization research, with (a) shallow research for all causes, (b) intermediate research for more promising causes, and (c) deep research for potential top causes.
    Reasoning transparency and outreach to allow both the EA and non-EA movement to update on our findings and to support the most impactful causes available.
    The Role
    As a researcher, you will be performing:
    Cause prioritization research and outreach – searching for causes, researching them, and engaging other organizations and individuals to use our research.
    Various generalist tasks (e.g. planning, administration).
    In your first couple of months, you will likely be focusing on research, but you will eventually get to handle more outreach-related work (e.g. representing CEARCH at EAG/EAGx).
    Ideal Candidate
    The ideal candidate will:
    Care about impact and doing good above all.
    Have strong research skills.
    Be able to work well in a team.
    Benefits
    The chief reason you might be interested in this role is that it can be expected to be highly impactful.
    Our internal cost-effectiveness analysis (CEA) suggests that we save around 1,700 disability-adjusted life years (or the equivalent of 58 lives) per USD 100,000 spent, which is around 10x more cost-effective than GiveWell top charities.
    External analysis supports this assessment, with Charity Entrepreneurship's CEA finding that we should be around 8x as cost effective as top EA causes.
    That said, such estimates should not be taken literally, given the high amount of uncertainty involved. Rather, they should be understood as giving a sense of how promising CEARCH's work – and hence your work with us – will be, given (a) the outsized benefits from identifying extremely impactful causes and directing resources there, and (b) the low operating costs of research and outreach.
    Salary offered will be competitive, and dependent on ability and relevant experience.
    Other benefits include remote work, flexible hours, and a chance to become a leader within the organization and an influential member of the EA community as we grow.
    How to Apply
    To apply, please fill up and submit the following application form. In terms of the overall hiring process, there will be the following stages:
    Stage 1: An application form (<= 0.5 hours)
    Stage 2: An interview (<= 0.5 hours)
    Stage 3: A work test (~4 days across 2 weeks – or more, if requested – with a lot of flexibility)
    Q&A
    What causes do you work on?
    All sorts of (a) global health and wellbeing causes, (b) longtermist causes, as well as (c) meta causes.
    Should I worry about my counterfactual impact?
    No – good researchers are hard to find, and the market is competitive; you should not worry about replaceability at all.
    When should I expect to hear back?
    If we're keen on your application, we'll try to get back within 3 weeks.
    Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.
    4 min