The Nonlinear Library

The Nonlinear Library

By The Nonlinear FundEducation
Download on the App Store

All The Nonlinear Library episodes

  • EA - Save the date: EAGxVirtual 2023 by Sasha Berezhnoi
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Save the date: EAGxVirtual 2023, published by Sasha Berezhnoi on May 23, 2023 on The Effective Altruism Forum.
    EAGxVirtual 2023 will take place on November 17-19
    Imagine interacting with EAs from over 70 countries and learning from their unique perspectives. Imagine walking across a virtual venue and making valuable connections, all from the comfort of your own home. Imagine no visa requirements and no airports. It's about to come true this November.
    Vision for the conference
    Our main goal is to help attendees identify the next steps to act based on EA principles wherever they are in the world and build stronger bonds within the community.
    Many people living outside of major EA hubs have uncertainties about how to take action. They don't have a good understanding of the EA landscape or who to ask. There are many types of constraints: language barriers, travel restrictions, or lack of knowledge about relevant opportunities.
    We want to address that by facilitating valuable connections, highlighting relevant opportunities and resources, and inviting speakers who are working on concrete projects. There will be a range of talks, workshops, live Q&A sessions, office hours with experts, and facilitated networking.
    What to expect
    Last year's EAGxVirtual featured 900 participants from 75 countries and facilitated lots of connections and progress. We want to build on this success, experiment, and improve.
    You can expect:
    Action-oriented content that will be relevant to people from different contexts and locations
    Always-available virtual venue (Gathertown) for unstructured conversations, socials, and private meetings
    Schedule tailored for participants from different time zones
    Application process
    Applications will be open in September. Sign-up here to get notified when it’s open.
    Admissions will not be based on prior EA engagement or EA background knowledge. We welcome all who have a genuine interest in learning more or connecting!
    If you are completely new to EA, we recommend signing up for the Introductory EA Program to familiarise yourself with the core ideas before applying.
    EAGxVirtual 2023 will be hosted by EA Anywhere with the support of the CEA Events team.
    We are looking forward to an inspiring conference with you!
    Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.
    3 min
  • LW - Yoshua Bengio: How Rogue AIs may Arise by harfe
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Yoshua Bengio: How Rogue AIs may Arise, published by harfe on May 23, 2023 on LessWrong.
    [I think this is noteworthy, as Yoshua Bengio is a famous ML expert, and he presents his arguments for AI xrisk here. I copied the whole blogpost below.]The rise of powerful AI dialogue systems in recent months has precipitated debates about AI risks of all kinds, which hopefully will yield an acceleration of governance and regulatory frameworks. Although there is a general consensus around the need to regulate AI to protect the public from harm due to discrimination and biases as well as disinformation, there are profound disagreements among AI scientists regarding the potential for dangerous loss of control of powerful AI systems, also known as existential risk from AI, that may arise when an AI system can autonomously act in the world (without humans in the loop to check that these actions are acceptable) in ways that could potentially be catastrophically harmful.
    Some view these risks as a distraction for the more concrete risks and harms that are already occurring or are on the horizon. Indeed, there is a lot of uncertainty and lack of clarity as to how such catastrophes could happen. In this blog post we start a set of formal definitions, hypotheses and resulting claims about AI systems which could harm humanity and then discuss the possible conditions under which such catastrophes could arise, with an eye towards helping us imagine more concretely what could happen and the global policies that might be aimed at minimizing such risks.
    Definition 1: A potentially rogue AI is an autonomous AI system that could behave in ways that would be catastrophically harmful to a large fraction of humans, potentially endangering our societies and even our species or the biosphere.
    Executive Summary
    Although highly dangerous AI systems from which we would lose control do not currently exist, recent advances in the capabilities of generative AI such as large language models (LLMs) have raised concerns: human brains are biological machines and we have made great progress in understanding and demonstrating principles that can give rise to several aspects of human intelligence, such as learning intuitive knowledge from examples and manipulating language skillfully. Although I also believe that we could design AI systems that are useful and safe, specific guidelines would have to be respected, for example limiting their agency. On the other hand the recent advances suggest that even the future where we know how to build superintelligent AIs (smarter than humans across the board) is closer than most people expected just a year ago. Even if we knew how to build safe superintelligent AIs, it is not clear how to prevent potentially rogue AIs to also be built.
    Rogue AIs are goal-driven, i.e., they act towards achieving given goals. Current LLMs have little or no agency but could be transformed into goal-driven AI systems, as shown with Auto-GPT. Better understanding of how rogue AIs may arise could help us in preventing catastrophic outcomes, with advances both at a technical level (in the design of AI systems) and at a policy level (to minimize the chances of humans giving rise to potentially rogue AIs). For this purpose, we lay down different scenarios and hypotheses that could yield potentially rogue AIs. The simplest scenario to understand is simply that if a recipe to obtain a rogue AI is discovered and generally accessible, it is enough that one or a few genocidal humans do what it takes to build one.
    This is very concrete and dangerous, but the set of dangerous scenarios is enlarged by the possibility of unwittingly designing potentially rogue AIs, because of the problem of AI alignment (the mismatch between the true intentions of humans and the AI’s understanding and behavior) and the compe...
    30 min
  • LW - Self-leadership and self-love dissolve anger and trauma by Richard Ngo
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Self-leadership and self-love dissolve anger and trauma, published by Richard Ngo on May 22, 2023 on LessWrong.
    I find the “bids and boundaries” framework from my previous post to be very useful, but also a little. sterile, perhaps. Imagine prefixing every request you ever make with “it’s okay to say no, but.”; or always refusing bids which you think you might come to resent; or always asking people if it’s okay to flirt with them before doing it. That makes relationships less risky, but also shallower and less spontaneous. In the shell-shield-staff terminology from this blog post: carefulness in making bids and setting boundaries is a shield which protects you from risk, and is better than staying in your shell, but still weighs you down—as opposed to a staff, which helps you move lithely, but which requires that you can absorb and recover from the consequences of making mistakes. You can replace a shield with a staff using the two skills I discuss in this post: self-leadership and self-love.
    These terms have straightforward meanings, but also more specific connotations in the IFS framework. In IFS, the “Self” is the aspect of you which mediates between other parts, and which is present when those parts aren’t active. Self-leadership is the ability to consistently engage with the world in a way that all your parts endorse, rather than letting some of them seize control in ways that others will later resent. It’s a difficult skill because when somebody hurts you, the hurt part often feels a strong drive to express itself, and will only let you intervene if it trusts that you’ll treat it fairly, and defend its interests, even when it’s less emotionally activated.
    Having said that, self-leadership doesn’t mean never getting angry—it just means never fully giving in to that anger or wielding it with the goal of hurting another person (or another part of yourself). Self-leadership might involve telling the other person that you feel angry at them, but without launching into a tirade; or telling them that you need to go on a walk to calm down, but giving them a reassuring gesture before you leave. In other words, self-leadership means that while your angry parts can’t seize full control, neither can the parts of you that want to suppress your anger. And on the receiving end, it involves being able to face someone who’s angry at you without either retreating into yourself or lashing out in response.
    For some readers, expressing anger at all might be scary. But suppressing anger doesn’t make it go away; once something starts to bug you, even small examples can trigger the underlying feeling of hurt. Eventually you end up with a conversation like:
    A: [finally hitting their breaking point] “I can’t stand how you never treat me with respect.”
    B: “What the hell? I was just doing [X]; you’re massively overreacting.”
    A: “I hate it when you [X], but you don’t care and just keep doing it anyway.”
    B: “I had no idea; why didn’t you say anything? I would have stopped ages ago!”
    A: “I kept giving you signs, but you never paid any attention to them.”
    B: “If you’re so indirect, how on earth am I meant to figure out what you mean?”
    Even this argument is better than just letting the disagreement fester (like the long-married couples who constantly snipe at each without ever expressing their needs openly). But ideally A would be able to express a bit of annoyance much earlier, in a less accusatory way; and B would be able to listen to it open-mindedly, without feeling defensive. That not only makes more progress on the object-level issue, but also helps you handle future issues more productively: anger is a mechanism for ensuring that you enforce your boundaries, and the security of knowing that you’ll do so even when you’re not very angry means that your angry parts need to be less active ...
    8 min
  • AF - 'Fundamental' vs 'applied' mechanistic interpretability research by Lee Sharkey
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: 'Fundamental' vs 'applied' mechanistic interpretability research, published by Lee Sharkey on May 23, 2023 on The AI Alignment Forum.
    When justifying my mechanistic interpretability research interests to others, I've occasionally found it useful to borrow a distinction from physics and distinguish between 'fundamental' versus 'applied' interpretability research.
    Fundamental interpretability research is the kind that investigates better ways to think about the structure of the function learned by neural networks. It lets us make new categories of hypotheses about neural networks. In the ideal case, it suggests novel interpretability methods based on new insights, but is not the methods themselves.
    Examples include:
    A Mathematical Framework for Transformer Circuits (Elhage et al., 2021)
    Toy Models of Superposition (Elhage et al., 2022)
    Polysemanticity and Capacity in Neural Networks (Scherlis et al., 2022)
    Interpreting Neural Networks through the Polytope Lens (Black et al., 2022)
    Causal Abstraction for Faithful Model Interpretation (Geiger et al., 2023)
    Research agenda: Formalizing abstractions of computations (Jenner, 2023)
    Work that looks for ways to identify modules in neural networks (see LessWrong 'Modularity' tag).
    Applied interpretability research is the kind that uses existing methods to find the representations or circuits that particular neural networks have learned. It generally involves finding facts or testing hypotheses about a given network (or set of networks) based on assumptions provided by theory.
    Examples include
    Steering GPT-2-XL by adding an activation vector (Turner et al., 2023)
    Discovering Latent Knowledge in Language Models (Burns et al., 2022)
    The Singular Value Decompositions of Transformer Weight Matrices are Highly Interpretable (Millidge et al., 2022)
    In-context Learning and Induction Heads (Olsson et al., 2022)
    We Found An Neuron in GPT-2 (Miller et al., 2023)
    Language models can explain neurons in language models (Bills et al., 2023)
    Acquisition of Chess Knowledge in AlphaZero (McGrath et al., 2021)
    Although I've found the distinction between fundamental and applied interpretability useful, it's not always clear cut:
    Sometimes articles are part fundamental, part applied (e.g. arguably 'A Mathematical Framework for Transformer Circuits' is mostly theoretical, but also studies particular language models using new theory).
    Sometimes articles take generally accepted 'fundamental' -- but underutilized -- assumptions and develop methods based on them (e.g. Causal Scrubbing, where the key underutilized fundamental assumption was that the structure of neural networks can be well studied using causal interventions).
    Other times the distinction is unclear because applied interpretability feeds back into fundamental interpretability, leading to fundamental insights about the structure of computation in networks (e.g. the Logit Lens lends weight to the theory that transformer language models do iterative inference).
    Why I currently prioritize fundamental interpretability
    Clearly both fundamental and applied interpretability research are essential. We need both in order to progress scientifically and to ensure future models are safe.
    But given our current position on the tech tree, I find that I care more about fundamental interpretability.
    The reason is that current interpretability methods are unsuitable for comprehensively interpreting networks on a mechanistic level. So far, our methods only seem to be able to identify particular representations that we look for or describe how particular behaviors are carried out. But they don't let us identify all representations or circuits in a network or summarize the full computational graph of a neural network (whatever that might mean). Let's call the ability to do these things 'comprehensive interpreta...
    6 min
  • LW - Papers, Please #1: Various Papers on Employment, Wages and Productivity by Zvi
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Papers, Please #1: Various Papers on Employment, Wages and Productivity, published by Zvi on May 22, 2023 on LessWrong.
    For a while, I’ve been keeping a bookmark folder called ‘Papers, Please’ of all the papers I’d like to check out in the future. For those I do get to look at, I’ve compiled my observations, with the intent of making this another kind of roundup. I noticed a bunch of them were focused on questions of employment, wages and productivity, so it made sense to pull those out into a post, and stay on the lookout for similar groupings in the future as the section expands.
    If there is a central theme here, it is the outsized role of norms, expectations and comparisons in determining outcomes, as well as underlying differences, and how such considerations seem to be neglected in many of the papers.
    What interventions are implied, if any, although that is very much not the question I’m looking to ask here? Highly unfashionable ones.
    US labor market can be thought of as two and a half mostly distinct labor markets.
    Aggregate U.S. labor market dynamics are well approximated by a dual labor market supplemented with a third, predominantly, home-production segment. We uncover this structure by estimating a Hidden Markov Model, a machine-learning method. The different market segments are identified through (in-)equality constraints on labor market transition probabilities. This method yields time series of stocks and flows for the three segments for 1980-2021.
    Workers in the primary sector, who make up around 55 percent of the population, are almost always employed and rarely experience unemployment.
    The secondary sector, which constitutes 14 percent of the population, absorbs most of the short-run fluctuations, both at seasonal and business cycle frequencies. Workers in this segment experience six times higher turnover rates than those in the primary tier and are ten times more likely to be unemployed than their primary counterparts.
    The tertiary segment consists of workers who infrequently participate in the labor market but nevertheless experience unemployment when they try to enter the labor force. Our individual-level analysis shows that observable demographic characteristics only explain a small part of the cross-individual variation in segment membership. The combination of the aggregate and individual-level evidence we provide points to dualism in the U.S. labor market being an equilibrium division of labor, under labor market imperfections, that minimizes adjustment costs in response to predictable seasonal as well as unpredictable business cycle fluctuations.
    This seems right to me. If you can check off a standard set of boxes that make you capable of ‘normal work’ and you are willing to accept what the market has on offer, you will be highly employable. When you lose one job, you will be able to find another. It would take quite a lot of AI automation or economic decline to change this.
    Whereas if you can’t check enough of the boxes, and you have some incompatibility with the jobs on offer and can’t find a niche that fixes this that you will accept, you will struggle. And it makes sense that there’s mostly a sharp line between these two groups, with a third group that doesn’t always want to work and is mostly in category two when they try.
    Note that a large majority of workers can check off the necessary boxes. This rules out anything too onerous, or the possession of an especially valuable skill, as a barrier to joining the first group.
    Is Pay Transparency Good? In some ways yes, in some ways no. Here’s the abstract.
    Countries around the world are enacting pay transparency policies to combat pay discrimination. 71% of OECD countries have done so since 2000.
    Most are enacting transparency horizontally, revealing pay between co-workers of similar seniority...
    14 min
  • LW - How I learned to stop worrying and love skill trees by junk heap homotopy
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: How I learned to stop worrying and love skill trees, published by junk heap homotopy on May 23, 2023 on LessWrong.
    There seems to be a stupid, embarrassingly simple solution to the following seemingly unrelated problems:
    Upskilling is hard: the available paths are often lonely and uncertain, workshops aren't mass-producing Paul Christianos, and it's hard for people to stay motivated over long periods of time unless they uproot their entire lives and move to London/Berkeley[1].
    It takes up to five years for entrants in alignment research to build up their portfolio and do good work–too slow for short timelines.
    Alignment researchers don't seem to stack.
    LessWrong–and by extension greenfield alignment–is currently teetering on the edge of an Eternal September: most new people are several hundred thousand words of reading away from automatically avoiding bad ideas, let alone being able to discuss them with good truth-seeking norms.
    We don't have a reliable way to gauge the potential of someone we've never met to do great work[2].
    This is not a new idea. It’s a side project of mine that could be built by your average first-year CS undergrad and that I have shelved multiple times. It's just that, for some reason, like moths to a flame or a dog to its vomit I just keep coming back to it. So I figured, third time’s the charm, right?
    The proposal (which I call 'Blackbelt' for obscure reasons) is really simple: a dependency graph of tests of skill.
    Note that last bit: 'tests of skill'. If my intention was merely to add to the growing pile of Intro to AI Safety (Please Don't Betray Us and Research Capabilities Afterward)[3] courses out there then we can all just pack up and go home and forget this poorly-worded post ever existed. But alas, my internal model says we will not go from doomed to saved with the nth attempt at prettifying the proof of the rank-nullity theorem. The real problem is not finding better presentations or a better Chatty McTextbook explanation, but can be found by observing what does not change.
    That is, let's invert the question of how to produce experts and instead ask: "What things should I be able to do, to be considered a minimum viable expert in X?"
    So for instance, since we’re all trying to get more dignity points in before 2028, let’s consider the case of the empirical alignment researcher.
    The minimum viable empirical researcher (and by 'minimum', I mean it) should probably know:
    How to multiply two matrices together
    How to train a handwriting classifier on the MNIST dataset
    How to implement backprop from scratch
    How to specify a reward function as Python code
    etc.
    Sure, there's nothing groundbreaking here, but that's precisely the point. What happens in the wild, in contrast, looks something like grocery shopping: "Oh, you need vector calculus, and set theory, and–textbooks? Read Axler, then Jaynes for probability 'cause you don't want to learn from those dirty, dirty frequentists...yeah sprinkle in some category theory as well from Lawvere, maybe basic game theory, then go through MLAB's course..."
    Maybe it's just me, but I get dizzy when every other word of someone's sentence packs months' worth of implied thankless work. Never mind how much it sounds like a wide-eyed Victorian-era gentleman rattling off classics one supposedly has read: reading a whole textbook is not an atomic action, let alone going through entire courses and assuming infinite motivation on the part of the victim[4].
    There's no accounting for tests
    What is a test, really?
    Related: the most accurate map of the territory is the territory itself, but what happens when the territory is slippery[5]?
    An apocryphal story goes that, when Pope Benedict XI was in search of a fresco artist he sent a messenger to a man named Giotto. The messenger asked him to provide a demonstration of ...
    24 min
  • EA - Give feedback on the new 80,000 Hours career guide by Benjamin Hilton
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Give feedback on the new 80,000 Hours career guide, published by Benjamin Hilton on May 23, 2023 on The Effective Altruism Forum.
    We’ve spent the last few months updating 80,000 Hours’ career guide (which we previously released in 2017 and which you've been able to get as a physical book). Today, we’ve put our new career guide live on our website. Before we formally launch and promote the guide - and republish the book - we’d like to gather feedback from you!
    How can you help?
    Take a look at the new career guide, which you can find at 80000hours.org/career-guide/.
    Please bear in mind that the vast majority of people who read the 80,000 Hours website are not EAs. Rather, our target audience for this career guide is approximately the ~100k young adults most likely to have high-impact careers, in the English speaking world. In particular, many of them are not yet familiar with many of the ideas that are widely discussed in the EA community. Also, this guide is primarily aimed at people aged 18-24.
    When you’re ready there’s a simple form to fill in:
    Click here to give feedback.
    Thank you so much!
    Extra context: why are we making this change?
    In 2018, we deprioritised 80,000 Hours’ career guide in favour of our key ideas series.
    Our key ideas series had a more serious tone, and was more focused on impact. It represented our best and most up-to-date advice. We expected that this switch would reduce engagement time on our site, but that the key ideas series would better appeal to people more likely to change their careers to do good.
    However, the drop in engagement time which we could attribute to this change was larger than we’d expected. In addition, data from our user survey suggested that people who changed their careers were more, not less, likely to have found and used the older, more informal career guide (which we kept up on our site).
    As a result, we decided to bring the advice in our career guide in line with our latest views, while attempting to retain its structure, tone and engagingness.
    We’re retaining the content in our key ideas series: it’s been re-released as our advanced series.
    Thank you for your help! You can find the new career guide here, and the feedback form here.
    Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.
    3 min
  • LW - Distillation of Neurotech and Alignment Workshop January 2023 by lisathiergart
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Distillation of Neurotech and Alignment Workshop January 2023, published by lisathiergart on May 22, 2023 on LessWrong.
    Disclaimer: This post is preliminary and doesn't yet fully align with the rigorous standards we typically aim for in LessWrong publications. It recaps the dynamic discussions from the neurotech for AI alignment workshop and therefore does not necessarily represent any single participant’s or author’s viewpoint. Shared ahead of the Foresight WBE workshop, our intent is to foster enriching discussion among participants. We welcome your input as we continue refining these insights.
    Introduction
    This document reviews key insights from a January workshop exploring the potential of Neurotech to contribute to the AI alignment problem. Neurotech, or neurotechnology, refers to the set of tools and methods designed to enhance understanding of the brain, manipulate its function, or interface directly with neural circuits, often for therapeutic or augmentation purposes. The AI alignment problem is the challenge of ensuring that the behavior of an artificial (general) intelligence system is in line with human values and intentions.
    The workshop, attended by 16 technical researchers and domain specialists, aimed to jointly map out whether and how neurotech can be useful towards alignment. Our core goal was to cast the net wide for all possible solutions, with an emphasis on those relevant to shortened alignment timelines. This initiative was a follow-up to the Paris Foresight Vision Weekend in November 2022, where it became evident that Neurotech-based approaches to alignment warranted further investigation. The two-hour interactive workshop aimed to compile a comprehensive list of potential approaches and prioritize them based on technical feasibility and timeline relevance.
    The workshop kicked off with a keynote presentation from David Dalrymple that laid the groundwork for participant discussion and defined key working definitions. This was followed by a brainstorming session, break-out room discussions, cluster identification and idea sorting, preliminary cluster prioritization (based on alignment impact, feasibility, timeline, and cost), and analysis of the prioritization results. David's keynote included a figure illustrating a potential breakdown of research intersecting Brain-Computer-Interface and Alignment.
    Several key insights from the workshop include
    Workshop participants brainstormed together, identifying six clusters of neuro-based alignment approaches:
    BCIs to extract human knowledge
    neurotech to enhance humans
    understanding human value formation
    cyborgism
    whole brain emulation
    BCIs creating a reward signal.
    Each cluster aims to align AI with human values but with highly separable/unique approaches.
    These categories likely don’t cover all potential approaches. In this post, we map these approaches with the hope that future workshops can explore and extrapolate to other less-common approaches that could hold potential.
    The workshop identified trends that agreed with commonly held assumptions. For example, participants found that complex technologies take longer to develop but have a greater potential impact on AI alignment. The relationship between cost and the other variables remains unclear. Future workshops should address cost and expected value to better guide investment in the space.
    Neurotech hardware development faces potentially long timelines that may fall behind the advent of AGI, which presents a major risk for all neurotech-based tradeoffs.
    Neurotech for AI safety is in an early phase with high uncertainty. During this phase, exploring various potential solutions, even unlikely ones, is valuable. This can lead to crucial insights and guide more focused investments, increasing the chances of finding effective AI alignment strategies.
    The...
    27 min
  • LW - Worrying less about acausal extortion by Raemon
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Worrying less about acausal extortion, published by Raemon on May 23, 2023 on LessWrong.
    I've wanted a good version of this post to exist for awhile. I'm not sure how to write that post. But here's a okay-ish first draft, which seemed hopefully better than nothing. I've made it possible for anyone to write comments on the LessWrong doc editor version of the post, so if you have ideas for how to improve it you can leave line-notes here.
    Once a month or so, the Lesswrong mods get a new user who's worried about Roko's basilisk, or other forms of acausal extortion. They write a post discussing their worries. In the past, they'd often they would get comments explaining "Here's why you don't need to be worried about acausal exortion", but then respond with "but, what about this edge case you didn't specifically address?".
    And this is a fairly sad situation – acausal trade/exortion is a bit confusing, and there are few places on the internet where people are knowledgeable enough to really explain it. But, my experience from talking to people worried about this is that no amount of explanation is really satisfying. They're left with a lingering "but, what if I didn't understand this properly and I'm still vulnerable?" and they keep looking to understand things better.
    My impression is that often, they're caught in an anxiety loop that wants to reinforce itself, and the act of worrying about it mostly encourages that anxiety loop to keep going rather than reassure it. I don't think the winning move here looks like "thoroughly understand the situation until you are no longer anxious". I think the winning move is more like "find something else to think about that is interesting so that the loop can wind itself down" or "I dunno, go outside and get some fresh air and take some deep breaths."
    I don't actually know what empirically works. (I'm interested in comments from anyone who was worried about acausal extortion and who eventually stopped being worried, who can talk about their experience)
    It seemed good to have an FAQ that at least covered some basics.
    So:
    Why shouldn't I worry about Roko's Basilisk or other acausal extortion stuff?
    Here's two main reasons as I understand them:
    Acausal extortion really just doesn't work on humans, for a variety of reasons.
    Even if there were edge cases that somehow were risky... thinking about them would presumably be harmful rather than helpful.
    Either it's true that acausal trade is risky (and so thinking about it is dangerous, and you shouldn't do it), or it's not, so worrying about it is mostly a waste of time.
    I realize it's generally hard to follow advice of the form "try not to think about X." I've often been frustrated by people telling me "just stop worrying". It's especially frustrating when they don't even get why I'm worried, and don't seem to have any cruxes for why it might make sense to worry.
    But sometimes "just try not to think about X" is really the best option. Meanwhile, I am pretty confident that Roko's basilisk or similar acausal threats don't actually work on humans.
    So my recommendation is to re-read a few posts on why it's in fact not that risky, so your brain doesn't have a giant scary blackbox next to the "is this dangerous?" question. And then, if your brain keeps generating "but, what about [edge case nobody has explicitly addressed?]", the problem isn't that you need to understand more weird decision theory. The problem is that you have an anxiety spiral, and you should solve that using the usual set of tools for resolving anxiety spirals. Talk to a therapist or some friends.
    Different anxiety-management-tools work for different people, so I can't give a great one-size-fits-all solution here, but the general strategy you should be following is "figure out how how to chill out about something you're being irratio...
    20 min
  • EA - Announcing a new organization: Epistea by Epistea
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Announcing a new organization: Epistea, published by Epistea on May 22, 2023 on The Effective Altruism Forum.
    Summary
    We are announcing a new organization called Epistea. Epistea supports projects in the space of existential security, epistemics, rationality, and effective altruism. Some projects we initiate and run ourselves, and some projects we support by providing infrastructure, know-how, staff, operations, or fiscal sponsorship.
    Our current projects are FIXED POINT, Prague Fall Season, and the Epistea Residency Program. We support ACS (Alignment of Complex Systems Research Group), PIBBSS (Principles of Intelligent Behavior in Biological and Social Systems), and HAAISS (Human Aligned AI Summer School).
    History and context
    Epistea was initially founded in 2019 as a rationality and epistemics research and education organization by Jan Kulveit and a small group of collaborators. They ran an experimental workshop on group rationality, the Epistea Summer Experiment in the summer of 2019 and were planning on organizing a series of rationality workshops in 2020. The pandemic paused plans to run workshops and most of the original staff have moved on to other projects.
    In 2022, Irena Kotíková was looking for an organization to fiscally sponsor her upcoming projects. Together with Jan, they decided to revamp Epistea as an umbrella organization for a wide range of projects related to epistemics and existential security, under Irena’s leadership.
    What?
    Epistea is a service organization that creates, runs, and supports projects that help with clear thinking and scale-sensitive caring. We believe that actions in sensitive areas such as existential risk mitigation often follow from good epistemics, and we are particularly interested in supporting efforts in this direction.
    The core Epistea team is based in Prague, Czech Republic, and works primarily in person there, although we support projects worldwide. As we are based in continental Europe and in the EU, we are a good fit for projects located in the EU.
    We provide the following services:
    Fiscal sponsorship (managing payments, accounting, and overall finances)
    Administrative and operations support (booking travel, accommodation, reimbursements, applications, visas)
    Events organization and support (conferences, retreats, workshops)
    Ad hoc operations support
    We currently run the following projects:
    FIXED POINT
    Fixed Point is a community and coworking space situated in Prague. The space is optimized for intellectual work and interesting conversations but also prioritizes work-life balance. You can read more about FIXED POINT here.
    Prague Fall Season
    PFS is a new model for global movement building which we piloted in 2022. The goal of the Season is to have a high concentration of people and events, in a limited time, in one space, and working on a specific set of problems. This allows for better coordination and efficiency and creates more opportunities for people to collaborate, co-create and co-work on important projects together, possibly in a new location - different from their usual space. Part of PFS is a residency program. You can read more about the Prague Fall Season here.
    Additionally, we support:
    ACS - Alignment of Complex Systems Research Group
    PIBBSS - Principles of Intelligent Behavior in Biological and Social Systems
    HAAISS - Human Aligned AI Summer School
    Who?
    Irena Kotíková leads a team of 4 full-time staff and 4 contractors:
    Jana Meixnerová - Head of Programs, focus on the Prague Fall Season
    Viktorie Havlíčková - Head of Operations
    Martin Hrádela - Facilities Manager, focus on Fixed Point
    Jan Šrajhans - User Experience Specialist
    Karin Neumanová - Interior Designer
    Linh Dan Leová - Operations Associate
    Jiří Nádvorník - Special Projects
    František Drahota - Special Projects
    The team has a wide range of experience...
    5 min