The Nonlinear Library

The Nonlinear Library

By The Nonlinear FundEducation
Download on the App Store

All The Nonlinear Library episodes

  • EA - Does Sentience Legislation help animals? by Animal Ask
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Does Sentience Legislation help animals?, published by Animal Ask on December 9, 2022 on The Effective Altruism Forum.
    EXECUTIVE SUMMARY
    The sentience of animals has long been recognised and has continued to be demonstrated on ever firmer scientific grounds. From the Brambell report (1965), which emphasised the importance of sentience in the understanding of animal welfare to the Cambridge Declaration of Consciousness (2012), which suggested widespread scientific acceptance of the idea (Rowan et al. 2021). In recent years, this widespread and scientific belief has been explicitly recognised in legislation in a growing number of countries and jurisdictions.
    It is certainly crucial that animal sentience be recognised in this way, as it is the most widely accepted basis for the inclusion of animals as moral patients. However, there is another question as to the value of the kind of legislation recognising the sentience of animals that we see in many different countries. This legislation explicitly recognises their sentience, though many other pieces of legislation could be thought to already implicitly do so.
    In our research helping organisations prioritise among different potential asks, we have considered the value of animal sentience legislation in many contexts. This report analyses the value of this legislation in terms of its current and future impact on animals.
    However, despite the apparently high-minded language recognising animal sentience in legislation, it is often accompanied by quite little direct and immediate change for animals. In some cases the legislation is accompanied by some specific statements about what the recognition of animal sentience is taken to directly imply, but there is typically little of this, leaving the legislation to largely be a symbolic statement of pledged values, leading to some concerns that it may be ineffectual legislation that leads to complacency. This type of humane washing remains our biggest concern with the legislation, but we think it is unlikely that it makes the ask net negative.
    In the case of the EU and New Zealand legislation, the intention behind the legislation as purely symbolic has been publicly stated, though this intention does not foreclose the possibility that animal advocates are able to draw some future victories from the legislation. In other cases such as Oregon and Québec we have seen some court cases that have successfully leveraged the legislation to push against the treatment of animals as property, though any significant improvements to animal welfare have yet to be seen.
    The most successful case so far has been that of the UK, because it promises to establish a committee to make sure that government decisions give due consideration to animal sentience. Further, it includes cephalopods and decapod crustaceans, and there is some chance this will lead to further protection for these animals. However, the head of the sentience committee does not seem like the appropriate choice to ensure its independence because of his conflict of interest as a farmer.
    Despite the absence of direct effects so far, sentience legislation so far has some plausibility as being instrumental in the long-term strategy of the movement. This makes assessing the value of this ask quite difficult, since this potential long-term importance is much more difficult to evaluate.
    Overall, our best estimate is that this ask has modest strength. In other words, with significant uncertainty, we think that the impact is fairly small compared to our top asks. We do not think that there is a risk of this ask being net negative, though we strongly recommend that organisations try to push to get sentience legislation to have concrete protections, so that it is more than symbolic and this risk is minimised. The strength of the ask will of cou...
    50 min
  • LW - The Story Of VaccinateCA by hath
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: The Story Of VaccinateCA, published by hath on December 9, 2022 on LessWrong.
    Linked is Patrick MacKenzie's writeup of VaccinateCA, the nonprofit that succeeded at creating the best source of vaccine availability data when the government completely dropped the ball. It's long, but one of the best things I've read in a while. Some highlights:
    We found it surprising that The System did not know where the vaccine was and that this fact persisted. The System allocates and ships the vaccine, after all. Devolution to various layers of government, community groups, and healthcare providers; pervasive inability to connect IT systems; and unwillingness to correct this problem meant that people in positions of authority considered the ground truth beyond the abilities of their institutions to discern.
    It is not harder to track a shipment of vaccine than it is to track a package from Amazon. Full stop. We are a nation that is extremely skilled at logistics, including healthcare logistics. A pharmacy chain can calculate, within a matter of minutes, the number of bottles of aspirin it owns, broken down by address. That count will be shockingly close to physical reality. Capitalism, ho!
    We chose, as a nation, that knowing the location of the vaccine was . . . just not a top priority.
    As an example of places where the data chain of custody broke down, consider the (true) case where a government actor directs some vials that it controls into the University of California at XYZ hospital system. (I will elide naming the specific hospital system, but for people not familiar with California, note that there are many different academic institutions called the University of California and their names are distinguished by the city they are primarily located in.) That hospital system has one address, according to a shipping spreadsheet.
    That hospital system routinely centrally receives, records, allocates, and reships all the medical supplies needed to keep a hospital system running, from saline to radiomedicine to scalpels. Then it parcels them out to the locations it provides healthcare at. Which it has more than one of and which are not a short walk from one another. I invite you to take a look at the locations list for the University of California at San Diego hospital system.
    We surprised the government by telling them that the vaccine was present where they believed it was absent. After delivery was taken at the central receiving facility, the vaccine was moved to individual locations where healthcare was conducted within the area of interest.
    What if the State of California had an alternative to engaging consultancies to deliver information systems months late and half functional? What if it, for example, had the world’s leading tech industry, which is abundantly capable of shipping and operating websites? What if that industry was also extremely experienced at solving scaled logistical challenges, including ‘atoms, not bits’ logistical challenges, including via the employment of tens of thousands of call center workers?
    Throughout the pandemic, as part of the ongoing estrangement between the tech industry and other corridors of power, there was unwillingness in the political class to work directly with the tech industry. You can write the tweet yourself, right? ‘Government tells Big Tech to ask millions of vulnerable Americans about their medical conditions. So they can do what, sell their data to advertisers?’ It certainly did not help matters that various people in positions of power assumed tech was complicit in serial prevarication being heard from elsewhere in the public sector, including about, e.g., Covid testing.
    This rift deepened sharply in the immediate wake of 6th January, which many important people laid at the door of tech. Damn techies trying to Ctrl-Alt-Delete consti...
    16 min
  • EA - Rethink Priorities is hiring: help support and communicate our work by Rethink Priorities
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Rethink Priorities is hiring: help support and communicate our work, published by Rethink Priorities on December 9, 2022 on The Effective Altruism Forum.
    TL;DR
    The Operations Department is hiring a Chief of Staff, Development Professional, and Communications Strategy Professional (applications close January 8, 2023).
    The Development and Communications team will hold a Q&A webinar on its job openings on December 15 at 11 am EST.
    We’re also accepting applications to join our Board of Directors until January 13.
    Visit our Careers Page for more information and to apply.
    Background
    Rethink Priorities (RP) has grown significantly, hiring 32 people in 2022 and completing ~60 research projects. In addition to our ongoing animal welfare research, we scaled our teams addressing global health and development, AI governance and strategy, and general longtermism. We also worked on pioneering initiatives like the Moral Weight Project, ran message testing, coordinated forums, and incubated new projects. In 2023, we intend to continue driving progress on global priorities by accelerating priority projects and further increasing the effectiveness of others’ work. We also intend to launch a Worldview Investigations team.
    More on RP’s ambitious plans can be found in this post on our impact, strategy, and funding gaps.
    Open positions
    Strong operations and good governance are integral to RP’s success as an organization. To help us scale and have an impact, we are opening the below new roles.
    All positions are remote and may require collaborating with staff in multiple time zones using Google Workspace, Asana, Slack, and other technologies.
    Chief of Staff
    Salary: $117,000 to $122,000 USD annually (pre-tax)
    Summary: Work closely with the COO, operations and HR leads, and the Directors of the Special Projects and the Development & Communications teams, overseeing high-level initiatives and ensuring projects stay on track across the organization
    A good fit for someone:
    Who understands nonprofit operations (finance, HR, project management, event planning, fundraising, communications, and legal compliance)
    With excellent organization and project management skills and attention to detail
    Who is comfortable working with confidential information and on multiple projects
    Based in the US or UK who is able to attend meetings during working hours between UTC-8 and UTC+3 time zones and travel 5-7 weeks per year
    Deadline: January 8, 2023
    Development Professional
    Salary: $80,155 to $115,235 USD annually (pre-tax)
    Summary: Help RP to grow sustainably by strengthening relationships with existing donors as well as prospecting and cultivating relationships with new donors with the capacity to give at least $100,000/year toward our annual budget of ~$10 million
    A good fit for someone:
    With experience in a development role or similar position
    With existing networks in EA and adjacent communities, especially with funders
    Who is intellectually curious, open-minded, resourceful, creative, and good at communicating (especially verbally/interpersonally)
    Who is able to think strategically and exercise good judgment in identifying new sources of funding
    Deadline: January 8, 2023
    Communications Strategy Professional
    Salary: $84,540 to $115,235 USD annually (pre-tax)
    Summary: Help RP to have an impact by mapping and identifying the most effective ways to target and engage our external audiences (e.g. researchers, nonprofit organizations, funders, and policymakers)
    A good fit for someone:
    With experience developing a communications plan
    Who understands EA and longtermism and is able to convey complex topics to different audiences in accessible yet nuanced ways
    Who is intellectually curious, open-minded, resourceful, creative, and good at communicating (especially in writing and with a sense of visual aesthetics)
    Who is...
    7 min
  • EA - Some observations from an EA-adjacent (?) charitable effort by patio11
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Some observations from an EA-adjacent (?) charitable effort, published by patio11 on December 9, 2022 on The Effective Altruism Forum.
    Hiya folks! I'm Patrick McKenzie, better known on the Internets as patio11. (Proof.) Long-time-listener, first-time-caller; I don't think I would consider myself an EA but I've been reading y'all, and adjacent intellectual spaces, for some time now.
    Epistemic status: Arbitrarily high confidence with regards to facts of the VaccinateCA experience (though speaking only for myself), moderately high confidence with respect to inferences made about vaccine policy and mechanisms for impact last year, one geek's opinion with respect to implicit advice to you all going forward.
    A Thing That Happened Last Year
    As some of the California-based EAs may remember, the rollout of the covid-19 vaccines in California and across the U.S. was... not optimal. I accidentally ended up founding a charity, VaccinateCA, which ran the national shadow vaccine location information infrastructure for 6 months.
    The core product at the start of the sprint, which some of you may be familiar with, was a site which listed places to get the vaccine in California, sourced by a volunteer-driven operation to conduct an ongoing census of medical providers by calling them. Importantly, that was not our primary vector for impact, though it was very important to our trajectory.
    I recently wrote an oral history of VaccinateCA. It may be worth your time. Obligatory disclaimer: I'm speaking, there and here, in entirely my personal capacity, not on behalf of the organization (now wound-down) or others.
    A brief summary of impact: I think this effort likely saved many thousands of lives at the margin, at a cost of approximately $1.2 million. This feels remarkable relative to my priors for cost of charitably saving lives at scale in the US, and hence this post.
    Some themes of the experience I think you may find useful:
    Enabling trade as a mechanism for impact
    To a first approximation, Google, the White House, the California governor's office, the Alameda County health department, the pharmacist at CVS, and several hundred thousand other actors have unified values and expectations with regards to the desirability of vaccinating residents of America against covid-19.
    They are also bad at trading with each other. Pathologically so, in many cases.
    One of the reasons we had such leveraged impact is that we didn't have to build Google, or recruit a few hundred million Americans to use it every day. We just had to find a very small number of people within Google and convince them that Google users would benefit from seeing our data on their surfaces as quickly as possible.
    Google and large national pharmacy chains cannot quickly negotiate an API, even given substantial mutual desire to do so. As it turns out, pharmacists already have a data store—pharmacists—and a transport layer—the English language spoken over a telephone call—and if you add a for loop, a cron job, and an SFTP upload to that then Google basically doesn't care about pharmacy chain IT anymore.
    Repeat this by many other pairwise interactions between actors within an ecosystem, and we got leveraged impact through their ongoing operations, with a surprising amount of insight into (and perhaps some level of influence upon) policy decisions which your prior (and my prior) would have probably suggested "arbitrarily high confidence that that is substantially above your pay grade."
    We didn't have to be chosen by e.g. the White House as the officially blessed initiative. We just had to find that initiative and be useful to it. (Though, if—God forbid—I ever have to do this again, I would give serious consideration to becoming the national initiative prior to asking for permission to do so and then asking the White House whether the ...
    12 min
  • LW - Setting the Zero Point by Duncan Sabien
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Setting the Zero Point, published by Duncan Sabien on December 9, 2022 on LessWrong.
    Context: I have hopes of assembling a full "defense against the dark arts" sequence. This essay will not necessarily be the very first one in the sequence, but it's One Of The Basic Spells To Defend Against, and it's the one I happen to have done the most data gathering on so it's getting written up first.
    Preamble I: Defense Against the Dark Arts
    By "Dark Arts," what I mean is taking actions which cause someone else to get lost inside an inaccurate map—making their map unmatch the territory in a way that is advantageous for you or disadvantageous for them.
    i.e. doing things which cause them to think that reality is a certain way, when it is not—when, if they had all of the information and context and unpressured time to think and process, they would reach a very different conclusion from the one they've reached under the influence of the spell.
    As a very simple, silly example, the actions of the king in The Parable of the Dagger are dark-arts-y in the sense that I intend. (If you haven't read it: it's short and very good.)
    I'm not planning to make a fine distinction between spells cast maliciously (i.e. people who are deceiving with deliberate intent) and spells cast instinctively/reflexively (e.g. people who are bending the truth because of their own fears or biases). In practice, it's often quite difficult to distinguish between these two groups, especially in one-off or very occasional interactions, and intent is not particularly relevant to countering the local effects of a given attempt-to-confuse.
    (Working around malevolent intent is more of a strategic concern, and DADA is more about tactics.)
    As for "Defense," the first and foremost tool in one's toolkit is awareness. Recognition. Most dark arts get the bulk of their power from creeping in unnoticed—once you consciously realize that oh, this person is negging me (for instance), the spell loses most of its potency.
    If the full sequence gets written, there will be more to say about various active mental and interpersonal motions that one can make to fend off Dark attacks, but for the sake of this essay, the goal is to give you a concept like "square" which will help you recognize a bunch of different objects out in reality that have more or less of the square-nature.
    Preamble II: On The Healthy Use Of Labels
    I'm a big fan of reification—the idea that crystallizing a concept makes you better able to recognize that concept in multiple places, and port your observations between those places, and develop generally applicable models and plans.
    But there's a sad side effect of developing new conceptual handles, which is that, inevitably, some subset of people are going to act as if the concept is "real" and start fighting over "is it or isn't it?" in given instances.
    Getting an autism diagnosis as an adult is useful because it suddenly gives you a new way to interpret your experiences, and a new search term to dig up resources. It promotes to your attention a set of people to talk to and tools to try out; learning that you're autistic means "things which help lots of autistic people are unusually likely to be helpful for you," which is way better than doing a random walk through the vast unsorted library of generically potentially helpful things.
    Ditto, having a concept like "sealioning" can be super useful, because suddenly you can see the core mechanical nature of a Thing That Sometimes Happens, and recognize how it crops up in widely spaced contexts, and start to flesh out a set of reusable responses that you can port between those contexts rather than reinventing the wheel from scratch every time.
    What's bad is if you take your new conceptual handle "sealioning" and end up going "ah, okay, the thing to do here is to decide Which...
    30 min
  • EA - Center on Long-Term Risk: 2023 Fundraiser by stefan.torges
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Center on Long-Term Risk: 2023 Fundraiser, published by stefan.torges on December 9, 2022 on The Effective Altruism Forum.
    Summary
    Our goal: CLR’s goal is to reduce the worst risks of astronomical suffering (s-risks). Our concrete research programs are on AI conflict, Evidential Cooperation in Large Worlds (ECL), and s-risk macrostrategy. We ultimately want to identify and advocate for interventions that reliably shape the development and deployment of advanced AI systems in a positive way.
    Fundraising: We have had a short-term funding shortfall and a lot of medium-term funding uncertainty. Our minimal fundraising goal is $750,000. We think this is a particularly good time to donate to CLR for people interested in supporting work on s-risks, work on Cooperative AI, work on acausal interactions, or work on generally important longtermist topics.
    Causes of Conflict Research Group: In 2022, we started evaluating various interventions related to AI conflict (e.g., surrogate goals, preventing conflict-seeking preferences). We also started developing methods for evaluating conflict-relevant properties of large language models. Our priorities for next year are to continue developing and evaluating these, and to continue our work with large language models.
    Other researchers: In 2022, others researchers at CLR worked on topics including the implications of ECL, the optimal timing of AI safety spending, the likelihood of earth-originating civilization encountering extraterrestrials, and program equilibrium. Our priorities for the next year include continuing some of this work, alongside other work including on strategic modeling and agent foundations.
    S-risk community-building: Our s-risk community building programs received very positive feedback. We had calls or meetings with over 150 people interested in contributing to s-risk reduction. In 2023, we plan to at least continue our existing programs (i.e., intro fellowship, Summer Research Fellowship, retreat) if we can raise the required funds. If we can even hire additional staff, we want to expand our outreach function and create more resources for community members (e.g., curated reading lists, career guide, introductory content, research database).
    What CLR is trying to do and why
    Our goal is to reduce the worst risks of astronomical suffering (s-risks). These are scenarios where a significant fraction of future sentient beings are locked into intense states of misery, suffering, and despair. We currently believe that such lock-in scenarios most likely involve transformative AI systems. So we work on making the development and deployment of such systems safer.
    Concrete research programs:
    AI conflict: We want to better understand how we can prevent AI systems from engaging in catastrophic conflict. (The majority of our research efforts)
    Evidential Cooperation in Large Worlds (ECL): ECL refers to the idea that we make it more likely that other agents across the universe take actions that are good for our values by taking actions that are good according to their values. A potential implication is that we should act so as to maximize an impartial weighted sum of the values of agents across the universe.
    S-risk macrostrategy: In general, we want to better understand how we can reduce suffering in the long-term future. There might be causes or considerations that we have overlooked so far.
    Most of our work is research with the goal of identifying threat models and possible interventions. In the case of technical AI interventions (which is the bulk of our object-level work so far), we then plan to evaluate these interventions and advocate for their inclusion in AI development.
    Next to our research, we also run events and fellowships to identify and support people wanting to work on these problems.
    Fundraising
    Funding situation
    ...
    33 min
  • EA - r.i.c.e.'s neonatal lifesaving partnership is funded by GiveWell; a description of what we do by deanspears
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: r.i.c.e.'s neonatal lifesaving partnership is funded by GiveWell; a description of what we do, published by deanspears on December 9, 2022 on The Effective Altruism Forum.
    [The Forum found an unexpected way to help! A previous version of this post mistakenly suggested that a large donation from GiveWell risked changing r.i.c.e. from a public charity to a private foundation. We previously experienced this sort of risk with funding from the Bill and Melinda Gates Foundation. We are very excited to learn from a Forum poster that GiveWell poses no such risk! Thank you! While we are still accepting additional donations to improve the health of a larger number of babies, there is no additional urgency to maintain the charity status of r.i.c.e.]
    I’m writing on behalf of my team at r.i.c.e., which is honored to be highlighted in GiveWell’s “Our recommendations for giving in 2022” post. In this post, I present the details of our program that prevents neonatal deaths inexpensively by causing the implementation of a Kangaroo Mother Care program in India.
    A description of our work, in brief
    Some of you may be familiar with my work in population ethics or as Director of the Population Wellbeing Initiative at UT-Austin (or, more likely, for distributing utilitarianism t-shirts). Since long before this phase of my career, I’ve also been Executive Director of r.i.c.e., a 501(c)3 public charity working on early-life health in rural north India.
    In collaboration with the Government of Uttar Pradesh and an organization in India, we support a Kangaroo Mother Care program to promote neonatal survival in a context where low birth weight babies would not otherwise receive such lifesaving care. KMC is a well-established tool for preventing deaths that we did not invent. Instead, our contribution is managerial innovation: We developed a public-private partnership to cause the government’s KMC guidelines to in fact be implemented cost-effectively in a public hospital where many low birth weight babies are born. That public healthcare systems in developing countries fail to implement life-saving policies and programs is a well-known problem in global health and in development economics.
    Because KMC is known to prevent neonatal death, and because the collaboration that we fostered found new ways to overcome the barriers to implementation in a context where many babies are not otherwise receiving needed care, it is not surprising that we can prevent neonatal deaths inexpensively.
    Our statistics show that we save lives very cost-effectively; indeed, about as inexpensively as any life-saving program known to EA. These statistics led to investments this year by two EA funders: first by Founders Pledge and then by GiveWell, after in-depth investigations of our work and our evidence. We have big plans for 2023, including continuing to save lives, starting a formal impact evaluation, and doing the advocacy and partnership-building needed to scale up the program to more districts.
    If you would like to further support this work, please donate to r.i.c.e. at this link to PayPal. It will let you donate either one time or by establishing a recurring monthly donation. If you don’t like PayPal, you can mail a check to RICE Institute, Inc., 472 Old Colchester Rd, Amston, CT 06231.
    Longer set of details
    What is Kangaroo Mother Care and why is it good?
    Kangaroo Mother Care (KMC) is a way to inexpensively keep low birth weight babies (and other neonates) clean, fed, and warm. Here is GiveWell’s summary of Kangaroo Mother Care in general. It is a bit broader than our program. To emphasize: KMC has been well-known to save lives for years. Our workwomanlike contribution is merely to get it implemented in a place where there are lots of low birth weight babies, in part because mothers are themselves often underweigh...
    12 min
  • EA - Announcing BlueDot Impact by Dewi Erwan
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Announcing BlueDot Impact, published by Dewi Erwan on December 9, 2022 on The Effective Altruism Forum.
    BlueDot Impact is a non-profit running courses that support participants to develop the knowledge, community and network needed to pursue high-impact careers.
    The purpose of this new organisation is to increase the number of people working on solving some of the world’s most pressing problems in an informed way. We do this by building professional, scalable and high-quality courses. We aim to give participants the opportunity to engage deeply and critically with the literature on particular high-impact fields, meet and collaborate with others interested in the topic, and build an understanding of the opportunities to pursue a career in the space.
    We believe there are many great people who are interested in working on important problems and have the skills to contribute, but want additional support to take the career-switching leap. By bringing together communities of such people to engage with material on the field over a sustained period and explore the range of career opportunities available to them, we believe they are more likely to make this leap.
    BlueDot Impact spun out of Cambridge Effective Altruism, and was founded by the team who was primarily responsible for running previous rounds of the AGI Safety and Alternative Protein Fundamentals courses. We created a new organisation so that we could focus solely on the courses. Cambridge Effective Altruism is still around and doing great community-building work.
    We’re really excited about the amount of interest in the courses, and think they have great potential to build awesome communities around key issues. As such we have spent the last few months:
    Working with pedagogy experts to make discussion sessions more engaging
    Formalising our course design process with greater transparency for participants and facilitators
    Building systems to improve participant networking to create high-value connections
    Collating downstream opportunities for participants to pursue after the courses
    Forming a team that can continue to build, run and improve these courses over the long-term
    Learn more about BlueDot Impact here, and register your interest for future rounds of our courses here.
    Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.
    3 min
  • AF - Take 8: Queer the inner/outer alignment dichotomy. by Charlie Steiner
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Take 8: Queer the inner/outer alignment dichotomy., published by Charlie Steiner on December 9, 2022 on The AI Alignment Forum.
    As a writing exercise, I'm writing an AI Alignment Hot Take Advent Calendar - one new hot take, written every day for 25 days. Or until I run out of hot takes.
    I'm not saying to never say "inner alignment." But you had better be capable of not using that framing if you want to work on alignment.
    The inner/outer alignment framing is from Risks from Learned Optimization. You already know what I mean, but just to go through the motions: it describes a situations where there's two key optimization processes going on: the "outer" optimization process humans are using to create an AI, and the "inner" process the created AI is using to make plans. Outer alignment is when the outer process is aligned with humans, and inner alignment is when the inner process is aligned with the outer process. This is an outstandingly useful framing for thinking about certain kinds of AI development, especially model-free RL.
    However, this framing has a limited domain of validity. Sometimes it breaks down in a way that looks like adding degrees of freedom - as if humans and the created AI are two ends of a string, and the AI-optimization process is a point along the string. Then you can imagine holding the ends fixed but being able to wiggle the midpoint around.
    This looks like creating an AI that's still aligned with humans, but not because it's aligned with a creation process that is itself aligned with humans - instead, both processes are imperfect, and in order to get good outcomes you had to reason about the alignment of the end-product AI directly. This is how it is for most present-day "dumb" AI, except replace "aligned" with "useful and safe." One can also see shardites as arguing that this is what we should be doing.
    Other times, the inner/outer framing breaks down entirely, because there isn't a distinct two-part structure. The key example is reflectivity - using the AI to reflect on its own optimization process rapidly blurs the lines between what's "inner" or "outer."
    But it's not just obvious reflectivity, sometimes the breakdown seems like it was in the problem statement the whole time. Often when people try to solve one of inner or outer alignment entirely, they find that they've sneakily had to solve the other problem as well. In order to "really solve" outer alignment, you want the AI-optimization process to care about the generalization properties of the created AI beyond the training data. In order to "really solve" inner alignment, the created AI shouldn't just care about the raw outputs of the process that created it, it should care about the things communicated by the AI-optimization process in its real-world context.
    I endorse these attempts to "really" solve alignment. If you think that the inner/outer alignment framing is obvious, it's probably valuable for you to deliberately look for opportunities to blur the lines. Dream up AI-generating processes that care about the AI's inner properties, or AIs that learn to care about humans in a self-reflective process not well-described in terms of an intermediate AI-optimizer. Queer the inner/outer alignment dichotomy.
    Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.
    4 min
  • LW - Systems of Survival by Vaniver
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Systems of Survival, published by Vaniver on December 9, 2022 on LessWrong.
    In this 1992 dialogue, Jane Jacobs lays out two basic descriptive clusters (syndromes) of morality, each tied to a means of production, and describes them as ‘systems of survival’. That is, life requires inputs; those inputs can come from either taking, which is the foundation of what she calls the Guardian moral syndrome, or from trading, which is the foundation of what she calls the Commercial moral syndrome. It’s an attempt to understand work by looking at the aspirational attitudes that people have towards work, and understand those how those attitudes are grounded in the mechanics of that work. The dialogue explains the ideas and corollaries in some depth; here I present just the stripped core of the ideas.
    Jacobs identifies fifteen precepts for each syndrome (I have ordered the lists slightly differently than Jacobs, to highlight contrasts, but really they want to be a graph; also, I think it's easy to have your eyes skip over the lists, but deep understanding of this requires thinking about each precept individually as well as the connections between them, which the long dialogue is better at doing. I recommend getting the gist, reading the rest of the post, and then coming back to meditate on the lists). The lists ignore universally esteemed behavior; Jacobs has her character Kate list them as “cooperation, courage, moderation, mercy, common sense, foresight, judgment, competence, perseverance, faith, energy, patience, wisdom”.
    The Commercial Moral SyndromeThe Guardian Moral SyndromeExert prowessShun tradingBe loyalBe exclusiveTake vengeanceRespect hierarchyBe obedient and disciplinedAdhere to traditionTreasure honorShow fortitudeDeceive for the sake of the taskDispense largesseMake rich use of leisureBe ostentatiousBe fatalistic
    Shun force
    Come to voluntary agreements
    Be honest
    Collaborate easily with strangers and aliens
    Compete
    Respect contracts
    Use initiative and enterprise
    Be open to inventiveness and novelty
    Be efficient
    Promote comfort and convenience
    Dissent for the sake of the task
    Invest for productive purposes
    Be industrious
    Be thrifty
    Be optimistic
    The precepts are a compilation and refinement of “esteemed behavior” notes I’ve made over a period of some fifteen years. Initially, I conceived of the two precept groups as embodying “trader” and “raider” morality, and was laggard at recognizing that “raider” precepts are as morally valid as traders’, and are grounded in legitimate territorial concerns.
    Why two syndromes? The core idea is that combining two precepts might lead to different levels of synergy or discord (“Be honest” and “Respect contracts” strengthen each other; “be thrifty” and “be ostentatious” weaken each other), and so we can rank all possible combinations of precepts from most consistent to least, and separately we can rank each precept by how much it aids or hinders work of a particular type, and then we can find the optimal set for each type of work. Jacobs isn’t doing this work from scratch, she’s just describing what society has found so far, and seems somewhat optimistic about further innovation. In the absence of a third system of survival, then there should be a system that’s optimal for trading, a system that’s optimal for taking, and then lots of suboptimal systems.
    For example, why is ‘shun trading’ useful for Guardians? Well, the obvious thing for them to trade is secrets, loyalty, or favorable treatment, none of which the broader system wants them to trade. As well, while merchants might freely consent or not to contracts proposed by either party, a Guardian dealing with a non-Guardian might be tempted to lean on the threat of their prowess or the respect due their station, in a way that causes all consent to be suspect. Safer to cleanly separate...
    11 min