The Nonlinear Library

The Nonlinear Library

By The Nonlinear FundEducation
Download on the App Store

All The Nonlinear Library episodes

  • EA - [Draft Amnesty] Early unfinished draft on the case for a first principles, systematic scoping of meat alternatives by Linch
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: [Draft Amnesty] Early unfinished draft on the case for a first principles, systematic scoping of meat alternatives, published by Linch on December 19, 2022 on The Effective Altruism Forum.
    Epistemic status: Was revealed to me in a dream.
    More seriously, this is a post (actually, stringing together two posts) I wrote based very loosely on research I did in late 2021. I do not necessarily stand by all the claims in the current post, but hope that it can still be moderately helpful to at least some readers. I think this post has some structural issues. It was not cleaned up sufficiently for my personal standards of publication. It is also in a more conversational style than I endorse, and I’ve grown to be less confident in the core metaphors. Earlier drafts also have critiques that I did not get around to addressing. I also expect it to be more generally false. However in the spirit of draft amnesty day, I am publishing it rather than leave it languishing forever in my Google Drive (and my conscience).
    This is a Draft Amnesty Day draft. That means it’s not polished, it’s probably not up to my standards, the ideas are not thought out, and I haven’t checked everything. I was explicitly encouraged to post something unfinished! Commenting and feedback guidelines: I’m going with the default — please be nice. But constructive feedback is appreciated; please let me know what you think is wrong. Feedback on the structure of the argument is also appreciated.
    I think among effective animal advocates who are techno-optimists, the arguments for alternatives to factory farming are too narrow in scope, eg, plant-based meat vs cultured meat, or plant-based meat vs cultured meat vs non-food investments (eg corporate campaigns), or occasional considerations of other meat alternatives like yeast or mushrooms. I think this is too narrow in scope. I instead think we should have first-principles evaluation of strategies to sate people's desire for meat while avoiding animal suffering:
    Carefully considering all the desired criteria of what people value in conventional meat
    Enumerating all of the existing strategies, including the lesser-known ones
    Brainstorming/exploring entirely new strategies
    Carefully analyze the feasibility of each strategy
    I believe a significant fraction of this work can be done through armchair thinking by smart generalists, but we will eventually need empirical data, computational modeling, actual experimentation and varied domain expertise.
    In the post, I explore why I believe this is the correct strategy, using the motivating example of an extremely scientifically literate person in the 1800s trying to figure out flight. Birds are an existence proof that heavier-than-air flight is possible, but not a guarantee that we can get heavier-than-air flight quickly, and certainly not a proof that our existing attempts to get heavier-than-air flight is the right approach.
    While I am more optimistic about the general framework than any operational details, I would like to sketch out a path forwards for what a research agenda/plan might look like:
    Getting private feedback and carefully evaluating the quality of this initial plan, attracting funders and researchers as necessary <- We are here.
    Have 1-3 people work on the initial scoping of this research for up to 6 months, trying to analyze, dissect, and red-team this argument for clear empirical or conceptual issues
    Executive search for someone to lead a new org, or a team within an existing org (Rethink Priorities, New Science) to lead “New Meat Studies”
    Form an initial group of 3-20 tight-knit researchers that aims to generate and quickly falsify different lines of research into viable alternatives to conventional meat
    When a line of research looks like it cannot be clearly falsified with existing resources, spin off new no...
    30 min
  • EA - CEA Disambiguation by Jeff Kaufman
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: CEA Disambiguation, published by Jeff Kaufman on December 19, 2022 on The Effective Altruism Forum.
    Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.
    1 min
  • LW - The True Spirit of Solstice? by Raemon
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: The True Spirit of Solstice?, published by Raemon on December 19, 2022 on LessWrong.
    A year ago at the Winter Solstice afterparty, I had a 5 hour discussion with various folk about what things are or aren't appropriate for Solstice. Some example debates:
    "Is Solstice primarily a rationality holiday? An EA holiday? The broader secular community?"
    "How essential is the journey from light, into darkness, into light?"
    "Is it okay to have a Solstice where we don't sing Brighter Than Today?"
    "How important are singalongs vs speeches?"
    "How important is it for singalongs to sound polished, vs for them to feel like an organic part of the community? Is it appropriate to pay professional musicians?"
    "How important is transhumanism or x-risk?"
    "Is it good or bad to change lyrics over time?"
    "How important is it to celebrate Solstice on literal astronomical Solstice? If you don't, why are we calling it Solstice? Is it important for the name to be clear?"
    "Is it okay to have one solstice someday with a 'bad ending', where instead of climbing back out of the darkness hopefully, we just... sit with it, and accept that maybe it might be what the future holds?"
    Many of these answers are subjective. But are they entirely subjective? I think this is a super interesting question.
    I generally support individual solstice organizers doing whatever they think is right, and not worrying much about what other people think. But I think it's an interesting question whether or not a "True Spirit of Solstice" exists.
    You can have whatever arbitrary aesthetics you want. But, some aesthetics can be more or less self-consistent than others. And some aesthetics can be more, or less, in alignment with reality than others.
    It's important to be able to propagate facts into your aesthetics.
    Ancient, Eternal, and yet Personal
    Here are two reasons I think people want holidays:
    To be connected to something greater than themselves. To know that they are part of a story that started before them and will continue after them, something vast and eternal and powerful and beautiful.
    To celebrate something oddly specific about them. My tribe, my family. A costly signal that I am part of this ingroup instead of that other one over there.
    These are in tension. Holidays vary in how much of each they offer.
    Often, people want to know that they are not alone in their oddly-specific narrative. That their oddly-specific-ness is, in fact, powerful and vast or at least enduring. A central frustration of Solstice is that for many people, it is so close to being a holiday that encapsulates them. But, it is subtly off. Someone else's ritual often feels quaint, and you can appreciate it artistically and anthropologically. Ritual that almost succeeds but then does something weird you don't like is cringy and terrible.
    And, for now, Secular Solstice is frankly one of the only games in town. If it doesn't work for you, your alternatives are either to hold a much smaller event for people you're highly aligned with, or to go to very different events that are more pagan, or vaguely-unopinionatedly-humanist, or more social-justice-flavored.
    People often want a sense of grandness, of something greater than themselves. So there's something dissatisfying about splitting off to run a smaller solstice for highly aligned people.
    So we have a bunch of interest groups with slightly different aesthetics, all vaguely hoping to steer solstice in a direction that gives them more spiritual oomph that aligns with their beliefs, and an uneasy truce where you can't push a particular city's Solstice too far in any given direction without making people upset.
    When it comes to individual solstices, I'm highly in favor of the organizer putting together a program with a strong, unified vision. But a question that comes up sometimes is "What is the spirit ...
    15 min
  • AF - Event [Berkeley]: Alignment Collaborator Speed-Meeting by Alex Mennen
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Event [Berkeley]: Alignment Collaborator Speed-Meeting, published by Alex Mennen on December 19, 2022 on The AI Alignment Forum.
    If you’re working in alignment research, or would like to be, and you would like help finding researchers with whom to collaborate, this event is for you.
    Inspired by the structure of a speed-dating event, participants will spend the first two hours in short one-on-one conversations with each other, changing partners at regular intervals, all for the goal of assessing fit as collaborators. The event will end with one hour of free-for-all socializing.
    Please RSVP through Eventbrite or Facebook:Facebook::
    The event will take place at Constellation (2150 Shattuck Ave.) on Floor 11, on Jan 14, 2023.
    Instructions for access:Upon arrival at the building (at the Shattuck Ave. entrance) text phone number (850) 510-9052, and one of the event hosts will come down to let you up. Feel free to arrive as soon as 2:45 PM.
    Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.
    2 min
  • EA - How my thinking about doing good changed over the years by Quadratic Reciprocity
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: How my thinking about doing good changed over the years, published by Quadratic Reciprocity on December 18, 2022 on The Effective Altruism Forum.
    This is a Draft Amnesty Day draft. That means it’s not polished, it’s probably not up to my standards, the ideas are not thought out, and I haven’t checked everything. I was explicitly encouraged to post something unfinished! Commenting and feedback guidelines: I’m going with the default — please be nice. But constructive feedback is appreciated; please let me know what you think is wrong. Feedback on the structure of the post is also appreciated.
    I first came across effective altruism as a teenager a few years ago, and the core idea instantly clicked for me after reading one post about it. In this post, I will talk about some ways in which my thinking around doing good has evolved over the years as a young person with a strong interest in making the world better.
    The emotions I feel when thinking about others’ suffering are less intense. I don’t know if teenage-me would have predicted this. As a child, I remember crying a lot when watching videos on animal suffering, when I first confronted the idea of infinite hell I was depressed for an entire summer, I wanted to give all the money I received on my birthday to people who were less fortunate because I knew they needed it more.
    I think the change is partly from just getting used to it. The first time you confront the horrors of factory farming it is awful but by the hundredth time, it’s hard for my brain to naturally feel the same powerful emotions of sadness and anger. Partly, the change is from starting to believe that it isn’t actually that virtuous to feel strong emotions at others’ suffering. Some of that is from having been in the effective altruism community, where it is easy to feel that what matters are the results of what you do and not the emotions behind what you do. I still feel strong emotions of empathy for those who are suffering some of the time when I am feeling particularly introspective and emotional.
    However, and this is because of being in the effective altruism community, I am much more aware of my own ranking of what the biggest problems are and it is harder for me to direct a lot of empathy towards causes that feel less “big” compared to factory farming, extreme poverty, and existential risk - even though, in absolute terms, the suffering of people living in terrible conditions in rich countries is still massive.
    At the same time, my ability to live according to my values has increased. I haven’t eaten meat in a couple of years whereas as a child and young teenager, this was really difficult for me to do even though I really wanted to be vegetarian. I have more tools now to do what I think is right, and the biggest of them all is having a social community where there are others who take their beliefs seriously and try to do good.
    I am much less willing to try to hack my brain in order to force myself to do and feel things I endorse. I used to be much more ashamed of some of my feelings and actions and felt a strong desire to figure out how to trick my brain into being more willing to sacrifice myself for others, into working all the time and being more ambitious. This involved doing things adjacent to self-deception. This was a really bad idea and caused me lots of pain and frustration.
    Instead, the thing that worked for me is acknowledging that I have “selfish” desires, that sometimes I take actions that actively hurt others, and that I have things that I deeply care about besides just maximising the good. Having a better picture of myself and what I actually value allowed me to work with the “altruist” and “selfish” sides of me to do things like be able to enjoy spending money and time on things that make me happy without feeling guilty and then ...
    8 min
  • EA - 80,000 Hours wants to see more people trying out recruiting by Niel Bowerman
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: 80,000 Hours wants to see more people trying out recruiting, published by Niel Bowerman on December 18, 2022 on The Effective Altruism Forum.
    Tl;dr
    I'm excited to see more EAs try out recruiting for EA projects. 80k doesn't "have it covered", and we are keen to see more people and organisations test their fit with this kind of work.
    Recruiting has low "barriers to entry": you need to find a hiring manager who wants your help, and you need to have the time available to help them.
    Work hard to understand what the hiring manager is looking for.
    You don't need a mountain of centralised data in a single CRM to get started. You can start with referrals, spreadsheets, emails and conversations.
    Aims
    The aim of this article is to:
    Encourage more people to test their fit with recruiting, and reduce the extent to which 80,000 Hours is inadvertently "crowding out" other organisations and people from working in the recruiting space.
    Communicate some tentative "lessons learned" from our time recruiting in 2019-2020.
    Historical context
    In 2019 and 2020 I worked on a team with Peter McIntyre at 80,000 Hours, helping organisations working to solve some of the world's most pressing problems to identify candidates to hire. Below are some reflections and lessons learned from that period. At the start of 2021 we put on hold much of our efforts to help organisations hire in order to focus on increasing 80,000 Hours' capacity to have conversations with people who applied to speak with us. (We are hoping to increase our capacity on recruiting again in 2023.)
    We did ~534 calls with potential candidates in 2019 and 2020. We generated lists of names for roughly 174 roles. We spent just a few minutes on some of those lists, and up to a month on others. We think something like 23 people from the lists we generated were later hired by organisations, though we make no claims about counterfactual impact in this article.
    I've written this article for people who are considering trying out recruiting, but haven't done much recruiting before. I've put the article together pretty quickly, and expect to update parts of it in response to questions and comments, so please do fire away in the comments section. I've posted this on the forum as part of the draft amnesty days.
    Lots of hiring managers want help hiring
    There are a lot of hiring managers out there who are struggling to find great candidates to hire.
    Sometimes that's because the person they want doesn't exist, but sometimes it's because the hiring manager doesn't have the time to go find the right person.
    If the problem is that the hiring manager doesn't have the time to find the right person, then you may be able to help if you're willing to put in the time.
    Focus on building great models of what hiring managers want in a candidate
    Having buy-in from the hiring manager is crucial
    The best way to build up a model of what the hiring manager is looking for is to be able to ask them a lot of questions.
    Understand how they would trade off between the various attributes they want in the ideal candidate.
    If you aren't able to regularly email/message/speak with the hiring manager during the selection process, it will be a bunch harder for you to find a candidate that will ultimately get hired.
    We tried doing recruiting for a few roles in large organisations where we knew someone on the team but not the hiring manager. We generally had a much lower placement rate in these cases.
    Give a great experience to the hiring manager
    In order to get access to the hiring manager, it's usually valuable to be as helpful as you can initially. Write out detailed but concise descriptions of the candidates you're suggesting, put them into tiers, add LinkedIn links, and generally make the process easy for the hiring manager. Once you have more experience you can start cutt...
    11 min
  • EA - Working with the Beef Industry for Chicken Welfare by RobertY
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Working with the Beef Industry for Chicken Welfare, published by RobertY on December 18, 2022 on The Effective Altruism Forum.
    Historically, the US farmed animal welfare movement has seen itself as working in opposition to the entire animal agriculture industry. In doing so, we may have taken on a larger and more powerful enemy than we need to. I’ll explore an alternative approach of working with parts of the meat industry to mutually push for strong welfare protections in other parts, focusing on chicken and beef.
    Executive Summary
    Small animals account for a vast majority of the suffering in animal agriculture. From a cost-effectiveness standpoint, this has rightly caused the animal advocacy movement to more recently focus on chicken, fish, and invertebrates. Taking this one step further, if most of our focus is only on certain parts of animal agriculture, we may not need to see ourselves in opposition to the entire industry.
    In particular, I argue that the US beef industry could be a critical ally in the fight for farmed animal welfare. It may seem unlikely that the beef industry would be interested in partnering with animal welfare advocates, but a deeper analysis of the structure of the industry indicates that incentives may be more aligned than they first seem:
    Chickens don’t have the same welfare protections as cows, which is unfair to beef producers.
    Chicken is substantially cheaper than beef, partially because producers don’t have to treat chickens as well as cows. Chicken and beef compete for space on the plate, so if the price of chicken goes up, this will likely increase demand for beef.
    The beef industry is large and complex, with multiple different players whose incentives are not always aligned. The three main pillars of the beef sector are ranchers, feedlot operators, and the big meat packers. The meat packers have interests across chicken, pork, and beef, so will likely oppose animal welfare improvements in any area. However, ranchers and feedlot operators are generally only invested in beef, meaning that welfare improvements in chicken wouldn’t directly affect them.
    Additionally, I believe that cattle ranchers, who are generally small business owners, often do care about animal welfare. If framed in the right way, they could be inherently interested in seeing farmed animals treated better.
    Partnering with the beef industry could be useful in building broader coalitions than the animal welfare movement historically has been able to. Additionally, if the goal is to eventually pass stronger federal legislation for poultry welfare, having the beef industry on board could be a critical necessary step.
    Outline
    I) Small animals should dominate farmed animal welfare discussions
    II) The incentives of the beef industry are structurally aligned with the animal welfare movement
    Existing federal laws are unfair to beef producers
    Increasing the price of chicken will benefit beef producers
    Background on the structure of the beef and chicken industries in the US
    The incentives of cattle ranchers
    The incentives of feedlot operators
    How to find common ground with the beef industry
    III) How the beef industry might be helpful in the fight for animal welfare
    Beef industry support may open up the path to federal poultry welfare legislation
    IV) Counterarguments
    Historical Opposition
    Sustainability
    Abolitionism
    V) Why I’m focusing on chicken and beef
    VI) Call to Action
    I. Small animals should dominate farmed animal welfare discussions
    One of the most important recent theoretical developments in farmed animal advocacy has been a focus on small animals, mainly chicken, fish, and invertebrates. This was very influenced by EA-style thinking: chickens vastly outnumber other land animals because they’re smaller (In 2022, poultry accounted for 98% of animals slaughtered for the U...
    27 min
  • LW - Bad at Arithmetic, Promising at Math by cohenmacaulay
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Bad at Arithmetic, Promising at Math, published by cohenmacaulay on December 18, 2022 on LessWrong.
    n-Cohesive Rings
    Definition: Let n be a positive integer. We define an n-cohesive ring to be a commutative ring S such that, for every prime p dividing the characteristic of S, pn divides the order of the multiplicative group S×. We define an n-cohesive ideal of a ring R to be an ideal I of R such that the quotient ring R/I is an n-cohesive ring.
    Example: Z/25 is a 4-cohesive ring. The multiplicative group R× is the set {1,3,5,7,9,11,13,15,17,19,21,23,25,27,29,31}, which consists of the 16 elements of R that are relatively prime to 25. The order of the multiplicative group R× is 16, which is divisible by 24, so R is an n-cohesive ring for n=4.
    Example: Consider the ideal (8) of the ring Z. The multiplicative group of Z/I is {1,3,5,7}, whose order is 4. The highest power of 2 that divides the order of this group is 22, which means that I is a 2-cohesive ideal.
    The notion of an n-cohesive ring, and the dual notion of n-cohesive ideals, do not, to the best of my knowledge, appear in the mathematical literature. I know of no definitions off the top of my head that are equivalent to n-cohesiveness. The definition is rigorous, logically sound, and there exist nontrivial examples of n-cohesive ideals. A problem like "classify all 5-cohesive ideals of Z" strikes me as not completely trivial. A problem like "classify all 5-cohesive ideals of [insert number ring here]" strikes me as potentially very difficult (though I am not a number theorist). If someone came along and proved a strong classification result about n-cohesive ideals in number rings, they could probably publish that result in a mid-tier algebra or number theory journal. I could easily imagine handing it off as a research project to an undergraduate learning about unit groups, or maybe even a grad student who was particularly bored.
    The most interesting thing about the concept of n-cohesive ideals, however, is that it was not invented by a human.
    The examples of n-cohesiveness given above did involve some human handholding and cherrypicking (we will talk more about this shortly), but, I think you'll judge, are at least partially attributable to AI.
    Before we get started, let me state some concrete predictions to keep us grounded.
    By 2030, there will exist a paper whose topic was chosen by an AI, with at least some examples and theorems suggested by the AI (possibly after significant human cherrypicking), whose proofs are mainly human-written (possibly with some AI contribution, involving significant handholding), published in a pure mathematics journal of reasonable quality: 95%
    By 2030, there will exist an a correct proof primarily written by an AI, with at most minor human editing and corrections, published in a pure mathematics journal of reasonable quality: 30%.
    By 2030, there will exist a correct, original, wholly AI-written paper, whose topic was chosen by the AI, published in a pure mathematics journal of reasonable quality: <1%.
    The second bullet's probability in my mind goes up significantly by 2040. I don't have good intuition about when I would expect something like bullet 3, but I can say that whenever bullet 3 does happen, mathematics is going to undergo some very serious and very interesting changes.
    We're getting a bit ahead of ourselves, though. Let's talk about n-cohesive rings.
    Formal and Natural Mathematical Languages
    At this point, it is well-known that ChatGPT is terrible at arithmetic. There is an example going around where it is asked something to the effect of "A bat and a ball together cost $1.10, and the bat costs $1 more than the ball, how much does the ball cost?" and it often says something like $0.10. It is safe to say that nobody is going to be using ChatGPT as their pocket calculator...
    35 min
  • EA - Update on spending for CEA-run events by Eli Nathan
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Update on spending for CEA-run events, published by Eli Nathan on December 18, 2022 on The Effective Altruism Forum.
    TL;DR: Spending on events run and supported by CEA (including EA Global and EAGx conferences) will likely be reduced due to a decrease in available funding. This might influence travel grants, catering, volunteering, ticketing, and non-critical conference expenses.
    The CEA events team is responsible for numerous events in the EA community, including EA Global, EAGx, and various retreat programs. We (the CEA events team) expect to reduce spending on events we run in the coming year due to:
    The FTX situation
    The reduction in funds available to Open Philanthropy (partially due to a general stock market decline)
    The growth of the EA community — meaning that grantmakers now have more alternative funding opportunities. i.e., we’re no longer one of the very few things available for them to fund (this is a good thing!)
    At this stage, we’re still navigating the new funding landscape and we aren’t sure what this means going forwards, but some potential consequences include:
    Travel grant funding will likely be more restrictive. Previously we’ve funded people to travel to any EA conference they’ve been accepted to. We expect to retain some amount of travel funding moving forwards, but we’ll likely have to be much more conservative about how much we give and who we give it to. When planning around an event, we’d recommend you act under the assumption that we will not be able to grant your travel funding request (unless it has already been approved).
    Catering will likely be cut down. We’ll likely have to stop providing all three of breakfast, lunch, and dinner on each day for our conferences — we still expect to have some food or snacks available, but it’s currently unclear exactly what we’ll be able to provide.
    We might go back to a volunteer model for people working at EA Global (we trialed paying “volunteers” at the last two EA Globals).
    We might introduce a variable pricing ticketing system where we ask people with higher incomes to pay more for their tickets (we expect to still have free and reduced cost tickets available for students and those on lower incomes).
    We might need to limit capacity at certain events (whereas previously we always accepted people if they were above a certain bar).
    If you have any questions or concerns, you can email us at [email protected] or comment below (though we may not be able to respond to all comments).
    Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.
    3 min
  • LW - Positive values seem more robust and lasting than prohibitions by TurnTrout
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Positive values seem more robust and lasting than prohibitions, published by TurnTrout on December 17, 2022 on LessWrong.
    Originally a shortform comment.
    Imagine we train an AI on realistic situations where it can kill people, and penalize it when it does so. Suppose that we successfully instill a strong and widely activated "If going to kill people, then don't" value shard.
    Even assuming this much, the situation seems fragile. See, many value shards are self-chaining. In The shard theory of human values, I wrote about how:
    A baby learns "IF juice in front of me, THEN drink",
    The baby is later near juice, and then turns to see it, activating the learned "reflex" heuristic, learning to turn around and look at juice when the juice is nearby,
    The baby is later far from juice, and bumbles around until they're near the juice, whereupon she drinks the juice via the existing heuristics. This teaches "navigate to juice when you know it's nearby."
    Eventually this develops into a learned planning algorithm incorporating multiple value shards (e.g. juice and friends) so as to produce a single locally coherent plan.
    The juice shard chains into itself, as its outputs cause the learning process to further reinforce and generalize the juice-shard. This shard reinforces itself across time and thought-steps.
    But a "don't kill" shard seems like it should remain... stubby? Primitive? The "don't kill" shard can't self-chain into not doing something. If you're going to kill someone, and then don't because of the don't-kill shard, and that avoids predicted negative reward... Then maybe the "don't kill" shard gets reinforced and generalized a bit because it avoided negative reward (and so reward was higher than predicted, which I think would trigger e.g. a reinforcement event in people).
    But—on my current guesses and intuitions—that shard doesn't become more sophisticated, it doesn't become reflective, it doesn't "agentically participate" in the internal shard politics (e.g. the agent's "meta-ethics", deciding what kind of agent it "wants to become"). Other parts of the agent want things, they want paperclips or whatever, and that's harder to do if the agent isn't allowed to kill anyone.
    Crucially, the no-killing injunction can probably be steered around by the agent's other values. While the obvious route of lesioning the no-killing shard might be reflectively-predicted by the world model to lead to more murder, and therefore bid against by the no-killing shard... There are probably ways to get around this obstacle. Other value shards (e.g. paperclips and cow-breeding) might bid up lesioning plans which are optimized so as to not make the killing a salient plan feature to the reflective world-model, and thus, the plan does not activate the no-killing shard.
    This line of argumentation is a point in favor of the following: Don't embed a shard which doesn't want to kill. Make a shard which wants to protect / save / help people. That can chain into itself across time.
    Other points:
    Deontology seems most durable to me when it can be justified on consequentialist grounds. Perhaps this is one mechanistic reason why.
    This is one point in favor of the "convergent consequentialism" hypothesis, in some form.
    I think that people are not usually defined by negative values (e.g. "don't kill"), but by positives, and perhaps this is important.
    Which I won't actually detail right now.
    Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.
    4 min