The Nonlinear Library

The Nonlinear Library

By The Nonlinear FundEducation
Download on the App Store

All The Nonlinear Library episodes

  • EA - Announcing the AIPolicyIdeas.com Database by abiolvera
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Announcing the AIPolicyIdeas.com Database, published by abiolvera on June 23, 2023 on The Effective Altruism Forum.
    Executive Summary
    AIPolicyIdeas.com is a new database that compiles AI policy ideas from various sources. It’s intended to help AI policy practitioners and researchers quickly review high-impact, high-feasibility AI policy ideas, inform decisions on what to push for or research, and identify gaps in knowledge for future work (alternate link if any issues AIPolicyIdeas.com URL).
    This was created relatively quickly. Users are encouraged to conduct their own research and analysis before making any decisions.
    Submit ideas through this form.
    If you’re working on existential-risk-relevant AI policy or related research, request access to the database via this form.
    Other people can also use GCR Policy’s related public database.
    Approval for accessing the AI policy ideas database is not guaranteed. We appreciate your understanding if your application is not approved.
    We are excited to announce the launch of AIPolicyIdeas.com, a database compiling AI policy ideas from various sources across the longtermist AI governance community and beyond. The database prioritizes inclusion of policy ideas that may help reduce catastrophic risk from AI and may be implementable in the US in the near- or medium-term (in the next ~5-10 years). The database includes policy ideas of varying levels of expected impact, clarity about how impactful they’d be, and feasibility.
    The ideas were curated by Abi Olvera from various sources such as Google Docs, the GCR Policy database, individual submissions, and public reports. For most ideas, we have included information on its source, relevant topic area, relevant U.S. agency, as well as loose ratings estimating expected levels of impact, feasibility, and specificity, and degree of confidence/certainty.
    Collection Process: Abi started off with a collection of lists of AI policy ideas from personal Google Docs, contacts, conversations, and public reports. To avoid redundancy, ideas were only added if they contained unique ideas not already on the database. The two largest sources of AI ideas were RP’s Survey on intermediate goals in AI governance and ideas shared by the GCR Policy team. Additional ideas will be gradually added from similar sources and a form for idea submission.
    Loose Ratings: To help sort the ideas, we used a loose five-point scale for impact, confidence in our impact assessment, feasibility, and specificity. These ratings were assigned by the original author, the GCR Policy evaluation team, or Abi. However, the ratings were not rigorously assessed and come from various sources, including different assessors with their biases.
    Note that most choices about what to include in the database and what ratings to give were made by Abi alone, without someone else reviewing that.Negative Impacts Not Well Accounted For: We want to make it clear that while we have included a range of policy ideas in this database, some may have lower confidence and unclear levels of expected impact. Therefore, potential negative impacts are not well represented in this database. We encourage users to exercise caution when considering ideas, particularly those with uncertain impacts, and to conduct their research and analysis before making any decisions.
    Flag if You’re Researching or Available for Expertise on an Idea: We hope this database will serve as a useful resource for effective policymaking and research that can help make a positive impact on society. Researchers and policy practitioners can engage with the database by reviewing ideas, filtering them by relevant agency, and adding their names to the "Person Researching or Familiar With" column to collaborate with others. Users can also help keep the database up-to-date by sharing relevant ideas...
    5 min
  • EA - Open Board recruitment should be a norm by Jack Lewars
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Open Board recruitment should be a norm, published by Jack Lewars on June 23, 2023 on The Effective Altruism Forum.
    Better governance would not necessarily have prevented prominent scandals in EA. However, stronger accountability and governance mechanisms would almost certainly have mitigated them, and possibly even prevented them entirely.
    Governance standards are generally quite poor in EA. If you audited EA organisations using a checklist of things like whistleblowing protections, managing conflicts of interest, Board function and Board composition, I think most Boards would be 'amber' or 'red' against wider governance norms. This unnecessarily increases the risk of future scandals. Unless this is reformed, we should expect more scandals that hurt people, organisations and the community, and which prevent us achieving the goals of the EA community.
    One of the easiest fixes here is Board composition. You want a Board that has:
    enough members that it can continue to function well if ~2 people are unavailable for some reason (therefore 6-8 people, given that some members may be conflicted on some issues)
    at least some people with relevant expertise, particularly the ability to interpret internal financial reports; to manage, assess and hold accountable the CEO; and to create strong governance processes
    a mixture of inside and outside views of the organisation (and the EA community) to reduce blind spots
    These things are much less likely to be achieved if the recruitment process for Boards is entirely done via personal networks and direct, unconditional invitations to join the Board. This approach is especially likely to fail tests two and three.
    Accordingly, I think it's really important that there have been at least three open calls for trustee recruitment this year. Rethink Priorities led the way in January, and One for the World (my organisation) and EVF are conducting open searches at the moment. This is a significant positive step and should become a norm.
    It can also be really inspiring. In our latest round, we had more than 20 people who met our bar for 'would be a values-aligned, valuable Board member'. Also, some of them were so impressive that we wouldn't have approached them directly, believing that they would be unlikely to accept.
    Epistemic status: reasonably high. There are well-established governance norms based on more than a century of real-world experience; it seems highly unlikely that EA is exempt from these.
    Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org
    3 min
  • EA - How CEA’s communications team is thinking about EA communications at the moment by Shakeel Hashim
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: How CEA’s communications team is thinking about EA communications at the moment, published by Shakeel Hashim on June 23, 2023 on The Effective Altruism Forum.
    TL;DR: For now, we're going to be promoting EA as a place for intellectual exploration, incredible research, and real-world impact and innovation.
    These are my thoughts, but Emma Richter has been closely involved with developing them.
    This post is intended as a very overdue introduction to CEA’s communications team, our goals, and what we’re currently working on/planning to work on.
    I started at CEA as head of communications in September 2022. My position was a new one: as I understand it, various EA stakeholders were concerned that EA communications had fallen into a diffusion of responsibility. Though everyone in this ecosystem wanted it to go well, no one explicitly managed it. I was therefore hired with the remit of trying to fix this. Emma Richter joined the team as a contractor in December and became a permanent member of the team in March. We’ve also worked with a variety of external advisors, most notably Mike Levine at TSD Communications.
    Our team has two main goals. The first is to help look after the EA brand. That means, broadly, that we want the outside world to have an accurate, and positive impression of effective altruism and the value created by this ecosystem. The second, more nebulous goal, is to help the EA ecosystem better use communications to achieve various object-level goals. This means things like “helping to publicise a report on effective giving”, or “advocating for AI safety in the press”. As communications capacity grows across the EA ecosystem, I expect this goal to become less of a priority for us — but for now I think we have expertise that can be used to make a big difference in this way.
    With that in mind, here’s how we’re thinking about things at the moment.
    I’ll start with what’s going on in the world. There are a few particularly salient things I’m tracking:
    On the EA brand:
    Negative attention on EA has significantly died down.
    We expect it to flare back up somewhat this autumn, around SBF’s trial and various book releases, though probably not to the level that it was in late 2022.
    On net, there doesn’t appear to be a hit to the EA brand from FTX (see here for various data). Among those who have heard of both, though, there may have been a hit — and I suspect that group of people would include important subgroups like journalists and politicians.
    There is uncertainty about what people want EA (the brand, the ecosystem and/or the community) to be.
    Within CEA, our new executive director might make fairly radical changes (though they may also keep things quite similar).
    From the job announcement: “One thing to highlight is that we are both open to and enthusiastic about candidates who want to pursue significant changes to CEA. This might include: Spinning off or shutting down programs, or starting new programs; Focusing on specific cause areas, or on promoting general EA principles; Trying to build something more like a mass movement or trying to be more selective and focused; Significant staffing changes; Changing CEA’s name.
    There is increased interest in cause-specific field building (e.g. see here).
    In general, there are lots of conversations and uncertainties about what direction to take EA in (“should we frame EA as a community or a philosophical movement?" or "should we devote most of our resources to AI safety right now?")
    I expect this uncertainty to clear up a little as conversations continue in the next few months (e.g. in things like EA Strategy Fortnight), and CEA getting a new ED might help too. But I don’t expect it to resolve altogether.
    That said, EA community building (groups, conferences, online discussion spaces) has a strong track record and it seems likel...
    8 min
  • EA - Four claims about the role of effective giving in the EA community by Sjir Hoeijmakers
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Four claims about the role of effective giving in the EA community, published by Sjir Hoeijmakers on June 23, 2023 on The Effective Altruism Forum.
    I’m sharing the below as part of the EA Strategy Fortnight. I think there’s value in discussing what the role of effective giving in the EA community should be, as (1) I expect people have quite different views on this, and (2) I think there are concrete things we should do differently based on our views here (I share some suggestions at the bottom of this post).
    These claims or similar ones have been made by others in various places (e.g. here, here, and here), but I thought it'd be useful to put them together in one place so people can critique them not only one-by-one but also as a set. This post doesn’t make a well-supported argument for all these claims and suggestions: many are hypotheses on which I’d love to see more data and/or pushback.
    Full disclosure: I work at Giving What We Can (though these are my personal views).
    Claim 1: Giving effectively and significantly should be normal in the EA community
    More concretely, I think it would be desirable and feasible for most people who currently self-associate with EA to give at least 10% of their income to high-impact funding opportunities (e.g. by taking the GWWC Pledge) or to be on their way there (e.g. by taking the Trial Pledge).
    I think this is desirable for three reasons: (1) effective giving is — in absolute terms — an incredibly efficient way for us to convert resources into impact, (2) even for individuals who may have more impact directly through their careers, giving effectively is often highly cost-effective on the margin and is not mutually exclusive with their direct impact (so worth doing!), and (3) there are many positive effects for the EA community as a whole from having effective giving as a norm.
    I also think this is feasible. There are good reasons for some people to not give at some points in their lives — for instance, if it leaves someone with insufficient resources to live a comfortable life, or if it would interfere strongly with the impact someone could have in their career. However, I expect these situations will be the exception rather than the rule within the current EA community, and even where they do apply there are often ways around them (e.g. exceptions to the Pledge for students and people who are unemployed).
    Claim 2: Giving effectively and significantly should not be required in the EA community
    I think we should positively encourage everyone in the EA community who can give effectively and significantly to do so, and celebrate people when they do — but I don’t think that this should be an (implied) requirement for people in order to “feel at home” in the community, for a couple of reasons:
    EA is about using one's resources to try to do the most good, and its community should be accessible to people who want to use different types of resources to do this (e.g. money, time, network, expertise).
    Moreover, we don’t want the EA community to intentionally or unintentionally select only for people who have significant financial resources: we would be missing out on many (if not most) of the people we need to achieve our ambitious goals, including a large part of the global population that isn’t in a position (yet) to give significantly.
    Claim 3: Giving effectively and significantly should be sufficient to be part of the EA community
    I think giving at least 10% to high-impact funding opportunities (or being on the path there) should be "enough" for someone to fully feel part of the EA project and community, regardless of their career. For example, people who give effectively should feel respected and included by other people in the community, feel represented by community leaders, and feel welcome at general EA-themed events. I believe th...
    12 min
  • EA - The flow of funding in EA movement building by Vaidehi Agarwalla
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: The flow of funding in EA movement building, published by Vaidehi Agarwalla on June 23, 2023 on The Effective Altruism Forum.
    This post is part of EA Strategy Fortnight. You can see other Strategy Fortnight posts here.
    I’ve been reflecting on the role of funding in the EA movement & community over time. Specifically I wanted to improve common knowledge around funding flows in the EA movement building space. It seems that many people may not be aware of it.
    Funders (and the main organizations they have supported) have shaped the EA community in many ways - the rate & speed at which EA has grown (example), the people that are attracted and given access to opportunities, and the culture and norms the community embodies and the overall ecosystem.
    I share some preliminary results from research I’ve conducted looking at the historical flow of data to movement building sources. I wanted to share what I have so far for the strategy fortnight to get conversation started. I think there is enough information here to understand the general pattern of funding flows. If you want to play around with the data, here is my (raw, messy) spreadsheet.
    Key observations
    Overall picture
    Total funding 2012-2023 by known sources
    According to known funding sources, approximately $245M have been granted to EA movement building organizations and projects since 2012. I’d estimate the real number is something like $250-280M. The Open Philanthropy EA Community Growth (Longtermism) team (OP LT) has directed ~64% ($159M) of known movement building funding (incl. ~5% or $12M to the EAIF) since 2016. Note that OP launched an EACG program for Global Health and Wellbeing in 2022, which started making grants in 2023. Their budget is significantly smaller (currently ~$10M per year) and they currently prioritize effective giving organizations.
    The unlabeled dark blue segment is “other donors”
    Funders of EA Groups from 2015-2022
    See discussion below for description of the "CEA - imputed" category. Note that I’ve primarily estimated paid organizer time, not general groups expenses.
    EA groups are an important movement building project. The Centre for Effective Altruism (CEA) has had an outsized influence on EA groups for much of the history of the EA movement. Until May 2021, CEA was the primary funder of part- and full-time work on EA groups. In May 2021, CEA narrowed its scope to certain university & city/national groups, and the EA Infrastructure Fund (EAIF) started making grants to non-target groups. In 2022, OP LT took over most university groups funding from both CEA (in April) and EAIF (in August). Until 2021 most of CEA’s funding has come from OP LT, so its EA groups funding can be seen as an OP LT regrant.
    Breakdown of funding by source and time (known sources)
    2012-2016
    Before 2016, there was very limited funding available for meta projects and almost no support from institutional funders. Most organizations active during this period were funded by individual earning-to-givers and major donors or volunteer-run. Here’s a view of funding from 2012-2016:
    No donations from Jaan Tallinn during this period were via SFF as it didn’t exist yet. There is a $10K donation from OP to a UC Berkeley group in 2015 that is not visible in the main chart. “Other donors” includes mostly individual donors and some small foundations
    Quick details on active funders during this period:
    Individual Donors: A number of (U)HNW & earning-to-give donors, many of whom are still active today, such as Jaan Tallinn, Luke Ding, Matt Wage and Jeff Kaufman & Julia Wise. I expect I’m missing somewhere between ~$100,000 to $1,000,000 of donations from individuals in this chart per year from 2012 to 2016.
    EA Giving Group: In 2013, Nick Beckstead and a large anonymous donor started a fund (the EA Giving Group) to which multiple individual ...
    16 min
  • LW - Catastrophic Risks from AI #1: Summary by Dan H
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Catastrophic Risks from AI #1: Summary, published by Dan H on June 22, 2023 on LessWrong.
    This is the first post in a sequence of posts giving an overview of catastrophic AI risks.
    Abstract
    Rapid advancements in artificial intelligence (AI) have sparked growing concerns among experts, policymakers, and world leaders regarding the potential for increasingly advanced AI systems to pose catastrophic risks. Although numerous risks have been detailed separately, there is a pressing need for a systematic discussion and illustration of the potential dangers to better inform efforts to mitigate them. This paper provides an overview of the main sources of catastrophic AI risks, which we organize into four categories: malicious use, in which individuals or groups intentionally use AIs to cause harm; AI race, in which competitive environments compel actors to deploy unsafe AIs or cede control to AIs; organizational risks, highlighting how human factors and complex systems can increase the chances of catastrophic accidents; and rogue AIs, describing the inherent difficulty in controlling agents far more intelligent than humans.
    For each category of risk, we describe specific hazards, present illustrative stories, envision ideal scenarios, and propose practical suggestions for mitigating these dangers. Our goal is to foster a comprehensive understanding of these risks and inspire collective and proactive efforts to ensure that AIs are developed and deployed in a safe manner. Ultimately, we hope this will allow us to realize the benefits of this powerful technology while minimizing the potential for catastrophic outcomes.
    Executive Summary
    Artificial intelligence (AI) has seen rapid advancements in recent years, raising concerns among AI experts, policymakers, and world leaders about the potential risks posed by advanced AIs. As with all powerful technologies, AI must be handled with great responsibility to manage the risks and harness its potential for the betterment of society. However, there is limited accessible information on how catastrophic or existential AI risks might transpire or be addressed. While numerous sources on this subject exist, they tend to be spread across various papers, often targeted toward a narrow audience or focused on specific risks. In this paper, we provide an overview of the main sources of catastrophic AI risk, which we organize into four categories:
    Malicious use. Actors could intentionally harness powerful AIs to cause widespread harm. Specific risks include bioterrorism enabled by AIs that can help humans create deadly pathogens; the deliberate dissemination of uncontrolled AI agents; and the use of AI capabilities for propaganda, censorship, and surveillance. To reduce these risks, we suggest improving biosecurity, restricting access to the most dangerous AI models, and holding AI developers legally liable for damages caused by their AI systems.
    AI race. Competition could pressure nations and corporations to rush the development of AIs and cede control to AI systems. Militaries might face pressure to develop autonomous weapons and use AIs for cyberwarfare, enabling a new kind of automated warfare where accidents can spiral out of control before humans have the chance to intervene. Corporations will face similar incentives to automate human labor and prioritize profits over safety, potentially leading to mass unemployment and dependence on AI systems. We also discuss how evolutionary dynamics might shape AIs in the long run. Natural selection among AIs may lead to selfish traits, and the advantages AIs have over humans could eventually lead to the displacement of humanity. To reduce risks from an AI race, we suggest implementing safety regulations, international coordination, and public control of general-purpose AIs.
    Organizational risks. Organ...
    13 min
  • LW - Catastrophic Risks from AI #2: Malicious Use by Dan H
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Catastrophic Risks from AI #2: Malicious Use, published by Dan H on June 22, 2023 on LessWrong.
    This is the second post in a sequence of posts giving an overview of catastrophic AI risks.
    2 Malicious Use
    On the morning of March 20, 1995, five men entered the Tokyo subway system. After boarding separate subway lines, they continued for several stops before dropping the bags they were carrying and exiting. An odorless, colorless liquid inside the bags began to vaporize. Within minutes, commuters began choking and vomiting. The trains continued on toward the heart of Tokyo, with sickened passengers leaving the cars at each station. The fumes were spread at each stop, either by emanating from the tainted cars or through contact with people's clothing and shoes. By the end of the day, 13 people lay dead and 5,800 seriously injured. The group responsible for the attack was the religious cult Aum Shinrikyo [5]. Its motive for murdering innocent people? To bring about the end of the world.
    Powerful new technologies offer tremendous potential benefits, but they also carry the risk of empowering malicious actors to cause widespread harm. There will always be those with the worst of intentions, and AIs could provide them with a formidable tool to achieve their objectives. Moreover, as AI technology advances, severe malicious use could potentially destabilize society, increasing the likelihood of other risks.
    In this section, we will explore the various ways in which the malicious use of advanced AIs could pose catastrophic risks. These include engineering biochemical weapons, unleashing rogue AIs, using persuasive AIs to spread propaganda and erode consensus reality, and leveraging censorship and mass surveillance to irreversibly concentrate power. We will conclude by discussing possible strategies for mitigating the risks associated with the malicious use of AIs.
    Unilateral actors considerably increase the risks of malicious use. In instances where numerous actors have access to a powerful technology or dangerous information that could be used for harmful purposes, it only takes one individual to cause significant devastation. Malicious actors themselves are the clearest example of this, but recklessness can be equally dangerous. For example, a single research team might be excited to open source an AI system with biological research capabilities, which would speed up research and potentially save lives, but this could also increase the risk of malicious use if the AI system could be repurposed to develop bioweapons. In situations like this, the outcome may be determined by the least risk-averse research group. If only one research group thinks the benefits outweigh the risks, it could act unilaterally, deciding the outcome even if most others don't agree.
    And if they are wrong and someone does decide to develop a bioweapon, it would be too late to reverse course.
    By default, advanced AIs may increase the destructive capacity of both the most powerful and the general population. Thus, the growing potential for AIs to empower malicious actors is one of the most severe threats humanity will face in the coming decades. The examples we give in this section are only those we can foresee. It is possible that AIs could aid in the creation of dangerous new technology we cannot presently imagine, which would further increase risks from malicious use.
    2.1 Bioterrorism
    The rapid advancement of AI technology increases the risk of bioterrorism. AIs with knowledge of bioengineering could facilitate the creation of novel bioweapons and lower barriers to obtaining such agents. Engineered pandemics from AI-assisted bioweapons pose a unique challenge, as attackers have an advantage over defenders and could constitute an existential threat to humanity. We will now examine these risks and how AIs...
    33 min
  • AF - Why Not Subagents? by johnswentworth
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Why Not Subagents?, published by johnswentworth on June 22, 2023 on The AI Alignment Forum.
    Alternative title for economists: Complete Markets Have Complete Preferences
    The justification for modeling real-world systems as “agents” - i.e. choosing actions to maximize some utility function - usually rests on various coherence theorems. They say things like “either the system’s behavior maximizes some utility function, or it is throwing away resources” or “either the system’s behavior maximizes some utility function, or it can be exploited” or things like that. [...]
    Now imagine an agent which prefers anchovy over mushroom pizza when it has anchovy, but mushroom over anchovy when it has mushroom; it’s simply never willing to trade in either direction. There’s nothing inherently “wrong” with this; the agent is not necessarily executing a dominated strategy, cannot necessarily be exploited, or any of the other bad things we associate with inconsistent preferences. But the preferences can’t be described by a utility function over pizza toppings.
    . that’s what I (John) wrote four years ago, as the opening of Why Subagents?. Nate convinced me that it’s wrong: incomplete preferences do imply a dominated strategy. A system with incomplete preferences, which can’t be described by a utility function, can contract/precommit/self-modify to a more-complete set of preferences which perform strictly better even according to the original preferences.
    This post will document that argument.
    Epistemic Status: This post is intended to present the core idea in an intuitive and simple way. It is not intended to present a fully rigorous or maximally general argument; our hope is that someone else will come along and more properly rigorize and generalize things. In particular, we’re unsure of the best setting to use for the problem setup; we’ll emphasize some degrees of freedom in the High-Level Potential Problems (And Potential Solutions) section.
    Context
    The question which originally motivated my interest was: what’s the utility function of the world’s financial markets? Just based on a loose qualitative understanding of coherence arguments, one might think that the inexploitability (i.e. efficiency) of markets implies that they maximize a utility function. In which case, figuring out what that utility function is seems pretty central to understanding world markets.
    In Why Subagents?, I argued that a market of utility-maximizing traders is inexploitable, but not itself a utility maximizer. The relevant loophole is that the market has incomplete implied preferences (due to path dependence). Then, I argued that any inexploitable system with incomplete preferences could be viewed as a market/committee of utility-maximizing subagents, making utility-maximizing subagents a natural alternative to outright utility maximizers for modeling agentic systems.
    More recently, Nate counterargued that a market of utility maximizers will become a utility maximizer. (My interpretation of) his argument is that the subagents will contract with each other in a way which completes the market’s implied preferences. The model I was previously using got it wrong because it didn’t account for contracts, just direct trade of goods.
    More generally, Nate’s counterargument implies that agents with incomplete preferences will tend to precommit/self-modify in ways which complete their preferences.
    . but that discussion was over a year ago. Neither of us developed it further, because it just didn’t seem like a core priority. Then the AI Alignment Awards contest came along, and an excellent entry by Elliot Thornley proposed that incomplete preferences could be used as a loophole to make the shutdown problem tractable. Suddenly I was a lot more interested in fleshing out Nate’s argument.
    To that end, this post will argue tha...
    23 min
  • EA - Yip Fai Tse on animal welfare in AI ethics and long termism by Karthik Palakodeti
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Yip Fai Tse on animal welfare in AI ethics and long termism, published by Karthik Palakodeti on June 22, 2023 on The Effective Altruism Forum.
    We talk about Fai's work in the field of AI ethics and non-human animals with Peter Singer, his thoughts on whether animal welfare is an issue concerning long termism, and some of the big debates in the philosophy of animal welfare interventions!
    Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org
    1 min
  • EA - US public perception of CAIS statement and the risk of extinction by Jamie Elsey
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: US public perception of CAIS statement and the risk of extinction, published by Jamie Elsey on June 22, 2023 on The Effective Altruism Forum.
    Summary
    On June 2nd-June 3rd 2023, Rethink Priorities conducted an online poll of US adults, to assess their views regarding a recent open statement from the Center for AI Safety (CAIS). The statement, which has been signed by a number of prominent figures in the AI industry and AI research communities, as well as other public figures, was:
    “Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war.”
    The goal of this poll was to determine the degree to which the American public’s views of AI risk align with this statement.
    The poll covered opinions regarding:
    Agreement/disagreement with the CAIS open statement
    Support for/opposition to the CAIS open statement
    Worry about negative effects of AI
    Perceived likelihood of human extinction from AI by the year 2100
    Our population estimates reflect the responses of 2407 US adults, poststratified to be representative of the US population. See the Methodology section of the Appendix for more information on sampling and estimation procedures.
    Key findings
    Attitudes towards the CAIS statement were largely positive. A majority of the population supports (58%) and agrees with (59%) the CAIS statement, relative to 22% opposition and 26% disagreement.
    Worry about AI remains low. We estimate that most (68%) US adults would say that, at most, they only worry a little bit in their daily lives about the possible negative effects of AI on their lives or society more broadly. This is similar to our estimate in April (where 71% were estimated to have this level of worry).
    The public estimates of the chance of extinction from AI are highly skewed, with the most common estimate around 1%, but substantially higher medians and means. We estimate that half the population would give a probability below 15%, and half would give a probability above 15%. The most common response is expected to be around 1%, with 13% of people saying there is no chance. However, the mean estimate for the chance of extinction from AI by 2100 is quite high, at 26%, owing to a long tail of people giving higher ratings. It should be noted that just because respondents provided ratings in the form of probabilities, it does not mean they have a full grasp of the exact likelihoods their ratings imply.
    Attitudes towards the CAIS statement
    Respondents were presented with the CAIS statement on AI risk, and asked to indicate both the extent to which they agreed/disagreed with it, and the extent to which they supported/opposed it. We estimate that the US population broadly agrees with (59%) and supports (58%) the statement. Disagreement (26%) and opposition (22%) were relatively low, and sizable proportions of people remained neutral (12% and 18% for agreement and support formats, respectively).
    It is important to note that agreement with or support of this statement may not translate to agreement with or support of more specific policies geared towards actually making AI risk a comparable priority to pandemics or nuclear weapons. People may also support certain concrete actions that serve to mitigate AI risk despite not agreeing that it is of comparable concern to pandemics or nuclear security.
    The level of agreement/support appears to vary with age: the youngest age brackets of 18-24 are expected to show the most disagreement with/opposition to the statement. However, all ages were still expected to have majority support for the statement.
    Perceived likelihood of human extinction from AI
    We were interested to understand how likely the public believed the risk of extinction from AI to be. In our previous survey of AI-related attitudes and beliefs, we asked...
    19 min