The Nonlinear Library

The Nonlinear Library

By The Nonlinear FundEducation
Download on the App Store

All The Nonlinear Library episodes

  • AF - Seven Strategies for Tackling the Hard Part of the Alignment Problem by Stephen Casper
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Seven Strategies for Tackling the Hard Part of the Alignment Problem, published by Stephen Casper on July 8, 2023 on The AI Alignment Forum.
    Thanks to Michael Ripa for feedback.
    TL;DR
    There are two types of problems that AI systems can have: problems that we can observe during development and problems that we can’t. The hardest part of the alignment problem involves problems we can’t observe.
    I outline seven types of solutions for unobservable problems and taxonomize them based on whether they require that we know what failure looks like and how it happens mechanistically.
    Solutions that require both knowing what failure looks like and how it happens
    Ambitious mechanistic interpretability
    Formal verification
    Solutions that require knowing what failure looks like
    Latent adversarial training
    Evals
    Solutions that require knowing how failure happens
    Mechanistic interpretability + heuristic model edits
    Solutions that require neither
    Anomaly detection
    Scoping models down
    I argue that latent adversarial training, mechanistic interpretability + heuristic model edits, and scoping models down all may be highly important, tractable, and neglected and loft out a few ideas for new work.
    Two ways that AI systems can fail
    Consider a simple way of dividing AI failures into two groups.
    Observable failures: Failures that we can encounter in training and development. These include failures on the train set, test set, some types of adversaries, or anything else that we might think to test our AI system on. These can be very bad, but they are problems that we can get feedback on - we can spot them and then work to fix them.
    Unobservable failures: Failures that we will not encounter in training and development. These failures are somewhat scarier and harder to address because we will not have feedback to help us solve them. These can include trojans, some adversarial examples, misgeneralization, and deceptive alignment.
    Because one cannot get feedback on them, unobservable failures are the harder part of AI alignment, and this is why the AI safety community is especially interested in them. It is important not to ignore the importance and difficulty of fixing observable problems, and there might exist a very valid critique of the AI safety community’s (over)emphasis on unobservable ones. However, this post will focus on unobservable failures.
    What are the ways that we can try to tackle unobservable failures? One good way may be to use models and data that are designed to avoid some of these problems in the first place (e.g. use better training data), but this will be outside the scope of this post. Instead, I will focus on ways to remove problems given a model and training set.
    A taxonomy of solutions
    Consider two things that might make the process of addressing an unobservable problem in a model very difficult.
    Knowing what failure looks like. Can we recognize the bad behaviors when we see them? Lots of the time in ML, bad behavior is thought of as something that can be detected from a single output/action. But in reality, badness may be a property of a trajectory and may not be recognizable in simple ways.
    Knowing how the failure happens. Can we figure out the internal mechanisms behind failures? This a difficult challenge because neural networks are hard to understand in terms of faithful mechanistic interpretations.
    Now consider which approaches to tackling unobservable failures depend on solving each of these two challenges.
    (Hard) Approaches that require both knowing what failure looks like and how it happens:
    Ambitious mechanistic interpretability: This refers to anything that involves using mechanistic interpretability to actually figure out how the model will do something bad. Once this is done, then the model that will do bad things can be deleted or modified.
    Formal verificatio...
    11 min
  • AF - Continuous Adversarial Quality Assurance: Extending RLHF and Constitutional AI by Benaya Koren
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Continuous Adversarial Quality Assurance: Extending RLHF and Constitutional AI, published by Benaya Koren on July 8, 2023 on The AI Alignment Forum.
    Introduction
    Lately, the problem of aligning artificial intelligence with human values rapidly changed its status from hypothetical to most concrete, with the rise of more general and more powerful models. The existing methods (Constitutional AI, RLHF and the like) are mostly good enough for common usage with the current models, but are probably not robust enough to scale much beyond human level, or to stand against smart attempts at malicious usage. My goal in this post is not to replace those methods with a complete solution to the AGI Alignment Problem, but to try and make the existing methods more robust - to buy us some more time before those break, and to maybe make our chances slightly better if an ASI suddenly emerges.
    Broadly speaking, my approach here is aimed at the outer alignment problem - i.e. making the model train on a signal of human values, as clean and as diverse as possible. The approach is based on explicit modeling of how Human Values are supposed to flow from humanity into the model, and then using this model to improve the flow. To improve the flow I will present two concrete directions.The first - and the one that I will develop in more detail - is about making the flow more robust by putting continuous adversarial pressure on every part of the chain. I will call it Continuous Adversarial Quality Assurance. The second direction for improving the flow of values is more holistic - there, the idea is to use explicit modeling of the relations between different sources of information about human values, in order to develop more principled ways of aggregating them. Both directions may be applied to improve RLHF, Constitutional AI, or any similar method.
    I also try to relate my suggestions with other well-known agendas - namely Debate and CIRL.
    Disclaimer: Against Benevolent Sovereigns
    In the introduction, I intentionally used “Human Values” with capital letters. The reason was to highlight the simplification. I am not a moral realist, and subscribe to some version of shard-theory of human values. I view human values as a messy negotiation between local preferences and long-term goals of individuals and coalitions, popular memes in the moral and aesthetic discourse, laws and institutions, societal norms and coordination mechanisms, etc. I use “Human Values” as a placeholder for something like “what humanity would have agreed to value, if it could cohere into something that has values”. Basically “Coherent Extrapolated Volition”, but without trusting the AI to do the extrapolation. Instead of a fixed target that a system finds and then maximizes, it is a moving target, always partially and locally defined, that should not be optimized faster than it may be actually negotiated by humanity.
    This text should therefore not be read as an attempted recipe for the ASI that would bring Utopia, but as a modest step for creating an AI that is compatible with open society and liberal democracy - an AI that does what its user asks it to do, but warn from unintended consequences and refuse illegal requests or those with substantial consequences for other people. Trying to build an ASI as a global optimizer of Human Values, and for that purpose to negotiate Human Values on a deadline, may only result in a Paperclip Maximizer or in a World War around which kind of paperclips each country wants to maximize.
    However, as long as we don’t push in that direction too hard, the concept of “Aligning with Human Values” is a modestly good approximation of what we try to do.
    Continuous Adversarial Quality Assurance
    The basic steps of Constitutional AI as described in Anthropic’s paper are:
    How is that process supposed to bring Hu...
    15 min
  • AF - "Concepts of Agency in Biology" (Okasha, 2023) - Brief Paper Summary by Nora Ammann
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: "Concepts of Agency in Biology" (Okasha, 2023) - Brief Paper Summary, published by Nora Ammann on July 8, 2023 on The AI Alignment Forum.
    This is a brief summary post of a recent paper titled "The Concept of Agent in Biology: Motivations and Meanings" by Samir Okasha (2023).
    In both Biology and AI Alignment, the concept of agency comes up a lot and appears relevant to properly understanding the respecting "territory" - while also being used in confusing, and sometimes confused, ways.
    Many more person-hours have been invested in progress in Biology compared to AI alignment, so it seems apt to keep an eyes out for any insights (as well as mistakes) from the discourse in Biology that might help improve the quality of discourse around the notion of 'agency' in Alignment.
    I decided to write this summary post in order to make the information in the paper more accessible to the Alignment discourse. The paper itself is already a succinct (5 page!) summary of decades of discussions about the appropriate use of the notion of 'agency' in Philosophy of Biology. I can recommend to read it fully, though I have tried to capture all relevant ideas/concepts in this post.
    Okasha himself is a well-respected Philosopher of Biology who has a range of substantive prior work on this and related topics (e.g. Agents and Goals in Evolution (2018), Evolution and the Levels of Selection (2006)).
    0. Overview
    "Biological agency" is a topic that has recently been gaining attention within Philosophy of Biology. In this paper, Okasha sets out to discuss "what exactly we mean by the term 'agent'?".
    To bring some clarity to the discourse, he:
    Distinguishes two motivations for why scholars typically invoke notions of 'biological agency', and then goes on to
    Differentiate three distinct concepts of agency.
    1. Two motivations
    Motivation 1: 'Agency' as a way to point at what is distinctive about living organisms relative to other biological entities at different hierarchical levels (e.g. organs, tissues, species or ecosystems)
    Biological organisms seems to possess certain 'agent-like attributes' (e.g. decision-making, learning about the environment, performing actions) that other biological entities, at different hierarchical scales, do not.
    This is sometimes motivated by scholars wanting to counterbalance or oppose "gene-centric or reductionistic views of evolution and development".
    "The worry that the gene-centric paradigm either 'leaves out the organism' altogether, or wrongly depicts all organismic activity as subservient to gene action, is of course familiar his was a well-known critique of Richard Dawkins’s early views and still resonates today." (p.2)
    "The 'organism-as-agent' idea is seen by some authors as a potential corrective to gene-centrism, emphasizing as it does the power of organisms to make choices, overcome challenges, modify their environment, and shape their own fate. " (p.2)
    The 'organism-as-agent' notion finds some empirical and theoretical support. E.g. "Sultan et al. (2021) point to a number of empirical phenomena, including phenotypic plasticity, the evolution of novelties, and niche construction, which, they claim, are easily accommodated by an agential perspective but less easily by rival perspectives on evolution and development." (p.2)
    Motivation 2: Treating biological entities as if they were agents is useful when it comes to understanding how evolution has shaped their phenotype or behavior
    'As-if Agency' is invoked as a "heuristically valuable" way of treating biological entities for the "purpose of scientific understanding".
    This 'as if' treatment remains silent about whether or not "the organisms in question really do have the attributes of agency" (p. 2)
    "Biological entities" here refers typically to individual organism, but can also be other entities like genes or group...
    13 min
  • LW - What Does LessWrong/EA Think of Human Intelligence Augmentation as of mid-2023? by marc/er
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: What Does LessWrong/EA Think of Human Intelligence Augmentation as of mid-2023?, published by marc/er on July 8, 2023 on LessWrong.
    Zvi Recently asked on Twitter:
    If someone was founding a new AI notkilleveryoneism research organization, what is the best research agenda they should look into pursuing right now?
    To which Eliezer replied:
    Human intelligence augmentation.
    And then elaborated:
    No time for GM kids to grow up, so:
    collect a database of genius genomes and try to interpret for variations that could have effect on adult state rather than child development
    try to disable a human brain's built-in disablers like rationalization circuitry, though unfortunately a lot of this seems liable to have mixed functional and dysfunctional use, but maybe you can snip an upstream trigger circuit
    upload and mod the upload
    neuralink shit but aim for 64-node clustered humans
    This post contains the most in-depth analysis of HIA I have seen recently, and provides the following taxonomy for applications of neurotechnology to alignment:
    BCIs to extract human knowledge
    neurotech to enhance humans
    understanding human value formation
    cyborgism
    whole brain emulation
    BCIs creating a reward signal.
    It also includes the opinions of attendees (stated to be 16 technical researchers and domain specialists) who provide the following analysis of these options:
    Outside of cyborgism, I have seen very little recent discussion regarding human intelligence augmentation (HIA) with the exclusion of the above post. This could be because I am simply looking in the wrong places, or it could be because the topic is not discussed much in the context of being a legitimate AI safety agenda. The following is a list of questions I have about the topic:
    Does anyone have a comprehensive list of organizations working on HIA or related technologies?
    Perhaps producing something like this map for HIA might be valuable.
    Does independent HIA research exist outside of cyborgism?
    My intuition is that HIA research probably has a much higher barrier to entry than say mechanistic interpretability (both in cost and background education). Does this make it unfit for independent research?
    (If you think HIA is a good agenda: ) What are some concrete steps that we (members of the EA and LW community) can take to push forward HIA for the sake of AI safety?
    EDIT: "We have to Upgrade" is another recent piece on HIA which has some useful discussion in the comments and in which some people give their individual thoughts, see: Carl Shulman's response and Nathan Helm-Burger's response.
    Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.
    3 min
  • EA - Some Observations on Alcoholism by Devin Kalish
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Some Observations on Alcoholism, published by Devin Kalish on July 8, 2023 on The Effective Altruism Forum.
    This is a tough one to post, it’s also a little off topic for the forum. I went back and forth a great deal about whether to crosspost it anyway, and ultimately decided to, since I have in some ways posted on this topic here before, and since there are several parts that are directly relevant to Effective Altruism (the last three sections all have substantial relevance of some sort). Doing this also makes it easier for this to be relatively public, and so to get some difficult conversations over with. The short version of it is that I’ve been an alcoholic, mostly in secret, for about three years now. This blogpost is a lengthy dive into different observations about it, and ways it has changed my mind on various issues. I don’t want to post the whole thing below because, well, frankly it’s huge and only occasionally relevant, so instead I’m going to post some relevant quotes as people often do with linkposts of things they didn’t write. There’s a good deal more in the link.
    First, here’s a quick summary of how things got started:
    “I started drinking during early 2020, when as far as I can tell there was no special drama going on with Effective Altruism, and I had already been involved with it in a similar capacity for a couple years. Most of the alcoholics I’ve met at this point either got started or got significantly worse during the pandemic, I was no different.
    But the truth is my drinking even then wasn’t terribly dramatic a coping mechanism. There was never anything that meaningfully ‘drove me to drink’. The idea that drinking at this point could land me here wasn’t part of my decision at all, I was just kind of bored and lonely and decided it would be a fun treat to drink a beer or two at night - something I had very rarely done before.
    As the pandemic wore on, it became something I looked forward to more and more, and eventually I discovered the appeal of hard liquor, which I never switched back from, and eventually I started working on my thesis for my first MA. The combination of my thesis and hard liquor turned a casual habit and minor coping mechanism into something more obviously hard for me to let go of. Over the course of the next three years things got slowly worse from there, and I came to realize more and more how little control I had.
    It wasn’t some meaningful part of the larger story of my life, replete with a buried darkness in my soul coming to the forefront, or a unique challenge driven by terrible circumstances. I have had to push back in therapy repeatedly on these subtler and more interesting attempts to make something of the event. The truth is sober reflection makes it all look like little more than a meaningless tragedy.”
    Here are some reflections on what it feels like:
    “One thing that I think people on the outside of this get wrong is the way conscious and unconscious feelings work in this context. I have been asked many times, especially in therapeutic contexts, what it feels like to get urges to drink. Truthfully, it doesn’t feel like much of anything as far as I can tell. It isn’t like thirst or hunger where there is some identifiable physical sensation. The easiest description of how the ‘urges’ start, is with intrusive thoughts. Not necessarily something as consciously obvious as ‘hey, you should drink’, but often more abstract things about drinking. ‘Do you think you’re going to drink tonight?’ ‘what happens if you drink tonight?’ ‘what happens if you don’t drink tonight?’ ‘if you do this, then you can drink tonight’. Argue with these, and they keep going all day until you drink. Drinking, in fact, is largely how you settle the argument, how you release the building tension.
    Do not argue with these thoughts, it does not help, o...
    10 min
  • EA - Even an obligate omnivore can try to eat less meat by Joe Rogero
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Even an obligate omnivore can try to eat less meat, published by Joe Rogero on July 8, 2023 on The Effective Altruism Forum.
    I've seen some good posts lately about veganism and health. Here's my two cents.
    I'm convinced by the arguments that many animals might have conscious experiences. I'd prefer not to torture the ones that do. Unfortunately, if I adopted a fully vegan lifestyle I might well be hospitalized for malnutrition within the year. In this post I'll give an overview of how I balance nutrition, ethics, and allergy constraints, and solicit ideas for improvement.
    My problem
    I am demonstrably allergic to a startling variety of foods, including, but not limited to:
    Peas
    Celery
    Uncooked egg yolks
    Chickpeas (and derivatives like hummus)
    Beans (but not peanuts, which are also legumes)
    Not deathly allergic, but definitely my-afternoon-is-ruined allergic. A tablespoon of hummus can give me 2-4 hours of intensely distracting pain. I have also experienced an identical, though sometimes inconsistent, reaction to:
    Fish
    Soylent
    Shellfish
    Eggplant
    Sweet potato
    Cranberry sauce
    Butternut squash
    Every protein shake mix I have ever tried
    Several mixtures containing none of the above but some hard-to-isolate combination of ingredients
    You will notice that several of the items on the above lists are widely considered key sources of vital nutrients in a vegetarian or vegan diet.
    Coupled with my strong dislike of spicy food, a typical restaurant menu typically contains at most 1-5 entrees I can safely eat. Catered meals often hedge me out entirely. At one (non-EA) event, after listing my allergies in the RSVP, I was served a dinner plate consisting of fish, lima beans, and rice pilaf with peas and celery.
    Even with these honestly ridiculous constraints, I've found some cheap ways to reduce my dependence on harmfully-produced meat.
    Reducing harm in an omnivorous diet
    It's easier to cut the first half of my meat intake than the second half. Diminishing returns apply. By the same logic, two people eating half as much meat is just as good as one person eating none.
    Here are some things I've learned while trying to minimize the suffering my diet imposes:
    Tofu is a fine supplement in many dishes, and rice is a cheap and flexible staple.
    Green vegetables like spinach and broccoli make good additions to any diet.
    Nuts and many dried fruits are nutrient-rich. Also, peanuts and cashews add a nice crunchy texture to homemade rice dishes.
    I can also eat cooked eggs and yogurt, and I've heard some convincing arguments in favor of eggs being less harmful than chicken and dairy less harmful than beef.
    I'm distrustful of "cruelty-free" branding because so many standards for that kind of thing are false or misleading, but with further research I expect I could find more harm-minimizing options there too.
    Being allergic to fish really hurts, because I think fish probably suffer less than birds or mammals, if at all. If I could eat more fish instead of meat, I would.
    I could probably bring myself to eat insect-based protein if it were a) actually available where I live and b) not recognizably still a whole bug at the time. Still working on that angle.
    I'm tentatively excited about lab-grown meat.
    My wife and I did some math and determined we aren't getting enough protein in our diets, so we're stepping up meat intake overall, but it's still a significant improvement over my past eating habits. I used to cook lots of chicken.
    My wife can eat beans and likes them fine. We don't have to eat the same things, especially when leftovers are available.
    We do a lot of home cooking, buy foods that keep, and try not to waste anything.
    If you have other ideas for reducing diet-induced harm, please share!
    Conclusion
    Even if you can't or don't want to fully expunge meat from your diet, it's possible to signific...
    5 min
  • AF - Views on when AGI comes and on strategy to reduce existential risk by Tsvi Benson-Tilsen
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Views on when AGI comes and on strategy to reduce existential risk, published by Tsvi Benson-Tilsen on July 8, 2023 on The AI Alignment Forum.
    Summary: AGI isn't super likely to come super soon. People should be working on stuff that saves humanity in worlds where AGI comes in 20 or 50 years, in addition to stuff that saves humanity in worlds where AGI comes in the next 10 years.
    Thanks to Alexander Gietelink Oldenziel, Abram Demski, Daniel Kokotajlo, Cleo Nardo, Alex Zhu, and Sam Eisenstat for related conversations.
    My views on when AGI comes
    AGI
    By "AGI" I mean the thing that has very large effects on the world (e.g., it kills everyone) via the same sort of route that humanity has large effects on the world. The route is where you figure out how to figure stuff out, and you figure a lot of stuff out using your figure-outers, and then the stuff you figured out says how to make powerful artifacts that move many atoms into very specific arrangements.
    This isn't the only thing to worry about. There could be transformative AI that isn't AGI in this sense. E.g. a fairly-narrow AI that just searches configurations of atoms and finds ways to do atomically precise manufacturing would also be an existential threat and a possibility for an existential win.
    Conceptual capabilities progress
    The "conceptual AGI" view:
    The first way humanity makes AGI is by combining some set of significant ideas about intelligence. Significant ideas are things like (the ideas of) gradient descent, recombination, probability distributions, universal computation, search, world-optimization. Significant ideas are to a significant extent bottlenecked on great natural philosophers doing great natural philosophy about intelligence, with sequential bottlenecks between many insights.
    The conceptual AGI doesn't claim that humanity doesn't already have enough ideas to make AGI. I claim that - - though not super strongly.
    Timelines
    Giving probabilities here doesn't feel great. For one thing, it seems to contribute to information cascades and to shallow coalition-forming. For another, it hides the useful models. For yet another thing: A probability bundles together a bunch of stuff I have models about, with a bunch of stuff I don't have models about. For example, how many people will be doing original AGI-relevant research in 15 years? I have no idea, and it seems like largely a social question. The answer to that question does affect when AGI comes, though, so a probability about when AGI comes would have to depend on that answer.
    But ok. Here's some butt-numbers:
    3%-10% probability of AGI in the next 10-15ish years. This would be lower, but I'm putting a bit of model uncertainty here.
    40%-45% probability of AGI in the subsequent 45ish years. This is denser than the above because, eyeballing the current state of the art, it seems like we currently lack some ideas we'd need - - but I don't know how many insights would be needed, so the remainder could be only a couple decades around the corner. It also seems like people are distracted now.
    Median 2075ish. IDK. This would be further out if an AI winter seemed more likely, but LLMs seem like they should already be able to make a lot of money.
    A long tail. It's long because of stuff like civilizational collapse, and because AGI might be really really hard to make. There's also a sliver of a possibility of coordinating for a long time to not make AGI.
    If I were trying to make a model with parts, I might try starting with a mixture of Erlang distributions of different shapes, and then stretching that according to some distribution about the number of people doing original AI research over time.
    Again, this is all butt-numbers. I have almost no idea about how much more understanding is needed to make AGI, except that it doesn't seem like we're there yet.
    Respon...
    23 min
  • EA - Announcing the Existential InfoSec Forum by calebp
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Announcing the Existential InfoSec Forum, published by calebp on July 8, 2023 on The Effective Altruism Forum.
    We're pleased to announce the first Existential InfoSec Forum, a half-day event aimed at strengthening the infosec community pursuing important ways to reduce the risk of an existential catastrophe.
    The forum will feature talks from infosec leaders across AI labs, policy and research, small group discussions and interactive workshops on real-world infosec challenges.
    This event will significantly emphasise AI as it coincides with a big push at Defcon for AI model red-teaming. We think this is an exciting opportunity to share insights and updates from a rapidly developing space - both for those working in infosec in priority areas for existential risk reduction and those newer to the field.
    Event Details
    Date: 10 August 2023
    Time: 2 pm - 8 pm PDT, including dinner
    Location: Las Vegas
    The conference will take place just before DEFCON 31 and will be a short walk from the main DEFCON venue (though the event is not affiliated with DEFCON). Wim van der Schoot is running the event with support from EA Funds and the Partner Events team at CEA.
    Why Attend?
    The forum is designed to:
    Facilitate knowledge transfer among attendees, improve understanding of the existential infosec landscape, and develop professional skills.
    Connect attendees to active infosec practitioners working on existential risk for feedback, identify potential job opportunities, and foster mentorship relationships.
    Enhance situational awareness of attendees to help them make more informed decisions and identify ecosystem gaps and opportunities.
    Mitigate risks associated with transitioning to direct work in infosec through a supportive network, which can help identify job opportunities.
    Who Should Attend?
    People already engaged in infosec within high-priority areas for existential risk reduction.
    Infosec professionals interested in transitioning into priority areas for existential risk reduction.
    Policy professionals, members of the national security community, and others working on regulating AI who would benefit from connecting with x-risk-conscious infosec practitioners.
    Software engineers (or others with a strong background for skilling up in infosec) who are interested in working in a high-priority infosec role for existential risk reduction in the next two years.
    Tentative Event Schedule
    2 pm - 3:30 pm: Keynote talks (speakers include Jason Clinton, Anthropic CISO, and others from frontier labs, government agencies, think tanks and researchers)
    3:30 pm - 4:30 pm: Networking break
    4:30 pm - 6 pm: Breakout sessions/workshops, e.g. AI infosec policy, AI cybersecurity capabilities and evaluations, how to defend against nation-states, and building a career in infosec.
    6 pm - 7 pm: Drinks reception
    7 pm - 8:30 pm: Closing dinner
    Please complete this form before 20 July to register your interest in attending. Note that we can only invite some people who register due to capacity limitations, but early registration will increase your chances of receiving an invitation.
    Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org
    4 min
  • LW - Passing the ideological Turing test? Arguments against existential risk from AI. by NinaR
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Passing the ideological Turing test? Arguments against existential risk from AI., published by NinaR on July 7, 2023 on LessWrong.
    I think there is a non-negligible risk of powerful AI systems being an existential or catastrophic threat to humanity. I will refer to this as “AI X-Risk.”
    However, it is important to understand the arguments of those you disagree with. In this post, I aim to provide a broad summary of arguments suggesting that the probability of AI X-Risk over the next few decades is low if we continue current approaches to training AI systems.
    Before describing counterarguments, here is a brief overview of the AI X-Risk position:
    Continuing the current trajectory of AI research and development could result in an extremely capable system that:
    Doesn’t care sufficiently about humans
    Wants to affect the world
    The more powerful a system, the more dangerous minor differences in goals and values are. If a powerful system doesn’t care about something, it will make arbitrary sacrifices to pursue an objective or take a particular action. Encoding everything we care about into an AI poses an unsolved challenge.
    As written by Professor Stuart Russell, author of Artificial Intelligence: A Modern Approach:
    A system that is optimizing a function of n variables, where the objective depends on a subset of size k
    In some sense, training a powerful AI is like bringing a superintelligent alien species into existence. If you would be scared of aliens orders of magnitude more intelligent than us visiting Earth, you should be scared of very powerful AI.
    The following arguments will question one or more aspects of the case above.
    Superintelligent AI won’t pursue a goal that results in harm to humans
    Proponents of this view argue against the idea that a highly optimized, powerful AI system will likely take actions that disempower or drastically harm humanity. They claim that either the system will not behave as a strong goal-oriented agent or that the goal will be fully compatible with not harming humans.
    For example, Yann LeCun, a pioneering figure in the realm of deep learning, has written:
    We tend to conflate intelligence with the drive to achieve dominance. This confusion is understandable: During our evolutionary history as (often violent) primates, intelligence was key to social dominance and enabled our reproductive success. And indeed, intelligence is a powerful adaptation, like horns, sharp claws or the ability to fly, which can facilitate survival in many ways. But intelligence per se does not generate the drive for domination, any more than horns do.
    It is just the ability to acquire and apply knowledge and skills in pursuit of a goal. Intelligence does not provide the goal itself, merely the means to achieve it. “Natural intelligence” - the intelligence of biological organisms - is an evolutionary adaptation, and like other such adaptations, it emerged under natural selection because it improved survival and propagation of the species. These goals are hardwired as instincts deep in the nervous systems of even the simplest organisms.
    But because AI systems did not pass through the crucible of natural selection, they did not need to evolve a survival instinct. In AI, intelligence and survival are decoupled, and so intelligence can serve whatever goals we set for it.
    LeCun’s argument implies that an AI is unlikely to execute perilous actions unless it possesses the drive to achieve dominance. Still, undermining or harming humanity could be an unintended side-effect or instrumental goal while the AI pursues another unrelated objective. Achieving most goals becomes easier when one has power and resources; taking power and resources from humans is one way to accomplish this. However, it’s unclear that all goals incentivize disempowering humanity. Furthermore, even if tak...
    12 min
  • EA - Why we're funding clubfoot treatment through MiracleFeet by GiveWell
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Why we're funding clubfoot treatment through MiracleFeet, published by GiveWell on July 7, 2023 on The Effective Altruism Forum.
    Author: Miranda Kaplan, GiveWell Communications Associate
    For many people, GiveWell is practically synonymous with our short list of top charities. But the amount of money we've sent to other organizations, doing other important work, has been increasing. In 2021, we made or recommended about $190 million in grants to non-top charity programs, like water treatment and malnutrition treatment, and in 2022, we set up the All Grants Fund specifically so donors could contribute to programs in this category.
    Source: GiveWell, GiveWell Metrics Report - 2021 Annual Review, p. 9
    We want to use this blog to give you more frequent, brief insights into these newer areas of our grantmaking before we publish our formal grant write-ups. Below we'll discuss, in light detail, a program that's well outside of our traditional wheelhouse, but that we think significantly improves children's lives - treatment for clubfoot with an organization called MiracleFeet.
    The grant
    Clubfoot is a congenital (i.e., present from birth) abnormality that causes one or both feet to twist inward and upward. Children born with clubfoot must walk on the sides or backs of their feet, which leads to pain, severely limited mobility, and, reportedly, social stigma. If not corrected, clubfoot is a lifelong condition.[1] In January 2023, we recommended a $5.2 million grant to MiracleFeet to expand its existing clubfoot treatment program in the Philippines and launch two new programs in Chad and Côte d'Ivoire.[2]
    In the countries where it works, MiracleFeet and its local NGO partners help health facilities diagnose and treat clubfoot, using a process called the Ponseti method. This generally requires placing the affected foot in a series of casts, performing a minor surgical procedure to improve the foot's flexibility, and bracing the foot during sleep for up to five years.[3] MiracleFeet and its partners provide supplies for casting and bracing, train government health care workers in the above procedures, build awareness of clubfoot, and help health systems collect data on treatment.[4] This makes it comparable to a "technical assistance" program: MiracleFeet doesn't perform clubfoot treatment itself; instead, along with its partners, it helps set health facilities up to successfully find and treat clubfoot cases themselves.
    The brace and custom shoes supplied by MiracleFeet for clubfoot treatment. Photograph courtesy of MiracleFeet.
    We were excited to recommend this grant because we think it will probably result in a lot more kids being treated for a serious, lifelong condition that nevertheless appears neglected. Clubfoot is debilitating but not life-threatening, and affects only about 1 in 800 babies born.[5] In resource-strapped countries, a relatively new and involved treatment like the Ponseti method may not be prioritized unless an NGO like MiracleFeet is there to advocate for and assist with it.[6] We estimate that MiracleFeet will support treatment of about 10,000 children with this grant, and that only about 10% of those children would get treated absent MiracleFeet,[7] though we don't feel very certain about this (more below, under "What we're still learning").
    All in all, after adjustments, we think that this grant will lead to about 3,700 cases of clubfoot successfully treated that otherwise wouldn't have been, and that will result in lifelong mobility gains and pain relief for the children treated.[8]
    Why this grant is different
    MiracleFeet's program is different from our top charities for a few reasons:[9]
    The program is expensive compared with our top charities. The Ponseti method requires specialized equipment, training for medical staff, and a multi-step execution wit...
    8 min