The Nonlinear Library

The Nonlinear Library

By The Nonlinear FundEducation
Download on the App Store

All The Nonlinear Library episodes

  • LW - Open Thread With Experimental Feature: Reactions by jimrandomh
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Open Thread With Experimental Feature: Reactions, published by jimrandomh on May 24, 2023 on LessWrong.
    This open thread introduces an experimental extension of LessWrong's voting system: reactions. Unlike votes, reactions are public; hovering over the reactions will show a list of users who reacted. For now, this feature is only for comments on this post in particular; after collecting feedback, we might roll out more broadly, or make significant alterations, or scrap it entirely. Reactions to comments in this thread will be preserved while discussions here are active, but they may be lost later if the feature changes in an incompatible way. Using this feature in various ways is planned to have karma minimums, but for this experimental post, those karma minimums are temporarily reduced to zero.
    These are similar to the reactions found on other sites such as Slack and Discord, but with a few twists. The first difference is the palette of reactions offered.
    The palette of available reactions, and the first page of reactions in particular, is explicitly a claim about what we think people should care about in comments (both positive and negative). On LessWrong, that means ways of relating to comments epistemically, reasons why comments are good and bad contributions.
    Reactions currently can only be applied to comments (not posts), only on this thread (for experimentation purposes), and only on entire comments, not individual comment sections. Depending on feedback, we may roll it out to comments on all posts, allow reactions to posts not just comments, and maybe implement something to allow reacting to specific parts of comments and not just to the whole thing.
    Antireacts
    Unlike other sites that have reactions, LessWrong has antireactions. An antireaction is like downvoting a reaction, or placing -1 reactions. This doesn't affect anyone's karma, but will show your name in the hover-over as someone who disagreed with the react. If there are at least as many antireactions of a given type as there are reactions of that type, then the icon won't be shown to people who don't hover over the reactions icon.
    The idea here is that, since we are trying to promote reactions that have more semantic content than the reactions you would find on most sites, many of the reactions in the palette are capable of being incorrect in a way that purely emotive reactions aren't.
    Motivation
    In my experience, seeing a list of names that I respect, marked as having reacted positively to my writing, feels much more motivating than a high karma number. Conversely, seeing negative reactions from names I don't respect feels less bad than a low karma number. And seeing a mixed reaction, with positive reactions from people I recognize and negative reactions from people I don't, creates a feeling of community.
    Other sites on the internet (most notably Facebook) try to use reactions to bias interactions towards positive vibes. I theorize that rationality culture is accidentally doing the opposite; that is, LessWrong has a problem with asymmetrically muted signal. When people see content they agree with and want to support, that mostly gets expressed through upvotes; when they see content they disagree with, that's more likely to result in writing a comment. This biases the perceived reception in a negative direction.
    Reactions are meant to serve as a middle ground between voting and commenting. This may capture feedback that would have been lost otherwise. In the case of positive feedback, this makes it more salient and motivating. In the case of negative feedback, I expect (though I could be mistaken) that receiving a reaction that expresses some specific criticism will at least be better, from an author's perspective, than a negative score with no replies.
    The default palette of reactions contains many...
    5 min
  • EA - Who does work you are thankful for? by Nathan Young
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Who does work you are thankful for?, published by Nathan Young on May 23, 2023 on The Effective Altruism Forum.
    I think that the other side of criticism is community support. So who are you grateful is doing what they are doing?
    Perhaps pick people who you think don't get complimented very much or don't get complimented as much as they get criticised.
    Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.
    1 min
  • AF - Aligned AI via monitoring objectives in AutoGPT-like systems by Paul Colognese
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Aligned AI via monitoring objectives in AutoGPT-like systems, published by Paul Colognese on May 24, 2023 on The AI Alignment Forum.
    Thanks to Arun Jose, Joseph Bloom, and Johannes Treutlein for feedback/discussions.
    Introduction
    The release of AutoGPT prompted discussions related to the potential of such systems to turn non-agentic LLMs into agentic systems that pursue goals, along with the dangers that could follow. The relation of such systems to the alignment problem has also been explored.
    In this short post, we investigate a threat model that comes from AutoGPT-like systems pursuing unaligned objectives and explore the potential for alignment via oversight. We briefly consider some key properties of such systems, and then discuss the idea that these systems’ high-level cognition might be interpretable by default and so might allow for sufficient oversight to ensure the system is aligned. Finally, we consider a couple of reasons for why the high-level cognition might be obscured from oversight and highlight ways of preventing this obscuration.
    Background
    AutoGPT-like systems, or Scaffolded LLMs, are systems that:
    wrap a programmatic scaffold around an LLM core and chain together a number of individual LLM calls to achieve some larger and more complex task than can be accomplished in a single prompt.
    This programmatic scaffold allows for information from the inputs/outputs of LLM calls to be stored in long-term memory as well as used as inputs to tools/plugins.
    We will state a few assumptions we’ll make about these AutoGPT-like systems. See this post for a similar context.
    For the purposes of this post, we assume that a single LLM call is incapable of successfully pursuing objectives. That is, the threat comes from the LLM being used within the scaffolded system.
    We assume that LLMs in the system must generate prompts/instructions for other versions of itself (factored cognition/bureaucratic approach)
    When information is passed from one LLM call/module/plugin to another, we say that this information is passed through an “information channel”. In particular, an overseer can observe this information (in the form of token sequences, etc.)
    The threat model we’ll look at is one in which we have an agentic AutoGPT-like system that pursues unaligned objectives.
    We don’t discuss the likelihood of this threat model being realized but note that these systems may suffer from inefficiencies that come from bureaucratic systems and hence may not be competitive. See this Zvi’s post for an exploration of the future of such systems.
    Oversight
    We assume that there is some overseer that wants the AutoGPT-like system to pursue an aligned objective. If the unaligned objective is detectable by the overseer, then the overseer can (hopefully) shut down the system before the objective is successfully pursued (including the objective of the system escaping the overseer’s oversight). This oversight may ensure that the system remains aligned and never pursues unaligned objectives.
    We now make explicit the key hope for oversight that comes from the modular-by-default structure of AutoGPT-like systems. The system is modular because information must be passed from one LLM call to another (or other modules like memory, etc.) Furthermore, because of the way the modular structure is encoded (via a programmatic scaffolding) and the fact that by default information is passed from one LLM call/module to another via natural language, a human overseer (or an LLM assistant) can, in theory, monitor every information channel for information pertaining to the objective or high-level cognition of the system.
    This kind of oversight fails if either:
    Information about the objective doesn’t pass through the information channels, or
    Information about the objective is passed through the information channels but ...
    8 min
  • LW - I don't want to talk about AI by KirstenH
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: I don't want to talk about AI, published by KirstenH on May 22, 2023 on LessWrong.
    This is a post I wrote for my blog, EA Lifestyles.
    like any good ea, I try to have a scout mindset. I don’t like to lie to myself or to other people. I try to be open to changing my mind.
    but that kind of intellectual honesty only works if you don’t get punished for being honest. in order to think clearly about a topic, all of the potential outcomes have to be okay - otherwise you just end up with mental gymnastics trying to come up with the answer you want.
    and in some cases - like existential risks from ai - none of the potential outcomes of thinking deeply about it are especially attractive.
    if I look into ai safety and come to believe the world’s going to end in ten years, that would be super depressing, especially if I can’t do much to contribute. so that outcome, while tolerable, isn’t especially attractive.
    you might think, “but maybe you’ll figure out that ai isn’t really a risk after all! wouldn’t that be reassuring?”
    let’s think through what’s going to happen if I investigate ai safety and realize it’s not that important:
    a lot of arguments. people in ea want to hear ideas that are different to their own - it’s a credit to their epistemic humility! I encountered this a lot when I first got into ea in 2017. I remember being cornered at a new year’s eve party, being interrogated in detail on my views on ai safety. it convinced me the guy I was talking to cares deeply about being correct about ai, but it wasn’t a lot of fun.
    a lot of re-education. after hearing my views, I’m guessing people will want to share their own. at church, when my views deviated from the norm, I would usually get a lot of interested questions followed by the same 2 or 3 recommended readings. I’d expect the same thing here.
    a lot of judgment. I’m not a scientist. if I tried to form my own view on ai safety, it might be really stupid. or it might just be kind of weird. either way, there would probably be at least a few people who would think less of me.
    and maybe - MAYBE - I could convince other people to shift their resources to something else. if I were right, that would be very positive; if I were wrong, it would be very negative. but as social sciences major, my chances of being both right and persuasive on ai safety seem astronomically low.
    if I wanted the best possible outcome for me personally, I’d just memorize two or three sentences from one of Ajeya or Holden’s blog posts, and quote them when I’m asked about my views. “I agree with Ajeya that in the coming 15-30 years, the world could plausibly develop “transformative AI”: AI powerful enough to bring us into a new, qualitatively different future, via an explosion in science and technology R&D” sounds pretty good and I think it would impress most of the people I talk to.
    so in summary, while forming an inside view on ai might be very altruistic of me, I just can’t bring myself to do it. it would take a long time and it’s hard for me to imagine any good coming from it. the next time someone asks me what I think about ai safety at a new year’s eve party, I plan to blithely respond, “I’ve never really thought about it. Would you like another drink?”
    Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.
    3 min
  • EA - Don Efficace is hiring a CEO by Don Efficace
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Don Efficace is hiring a CEO, published by Don Efficace on May 23, 2023 on The Effective Altruism Forum.
    Don Efficace is a French effective giving association that aims to enable donors to support charitable programs deemed to be highly effective by independent evaluators in the area of global health and poverty, with an aim to expand to include climate change, and animal welfare.
    Our scope and purpose are similar to those of other effective and successful national giving organizations (eg. Effektiv Spenden in Germany, Ayuda Efectiva in Spain, Doneer Effectief in the Netherlands). The costs of Don Efficace are currently funded by private donors and Giving What We Can, so 100% of common donations fund charitable programs.
    We are recruiting for the position of Executive Director. In this strategic role for the development of Don Efficace in France, you will have the autonomy to create your own team, and collaborate with the Board of Directors, which is composed of internationally recognized experts with experience in various fields. The main task is to develop a fundraising strategy with French donors, including the media presence. You will also be in charge of overseeing the operational aspects such as the development of the website, communication tools or means, budget, recruitment, etc.
    Responsibilities:
    Raising funds for charitable programmes with proven effectiveness
    Engage with the French community and the media to promote understanding of the importance of impact and the value of evidence in charitable giving
    Inform the general public about the wide variations in effectiveness of different programs, and the ability of donors to increase their charitable impact based on evidence
    Working alongside other stakeholders (e.g., Giving What We Can, and organizations involved in charitable programmes)
    Developing the community of donors seeking to give effectively in France
    Managing operations in Don Efficace (budget tracking, meetings, reporting to donors, etc.)
    Recruiting and managing a small team of staff and volunteers
    The ideal candidate would have:
    Strong interpersonal and communication skills, including teamwork but also convincing people to support a project you believe in
    Ability to work independently and take initiative
    Having a growth mindset, strategic and iterative thinker
    Strong taste for fast-paced projects and small structures
    Strong interest in projects that aim to make a real impact
    Excellent written and spoken French
    Sufficient English for written and spoken communication
    3-5 years of experience, ideally some in fundraising and management
    Open to the values of Effective Giving: transparency, efficiency, and an evidence-based approach to maximize positive impact
    Salary, benefits and location:
    We are flexible on the availability of candidates and can accept different formats: full time (CDI), part time, job sharing and contractual arrangements, remote work (full remote acceptable) or on-site in Paris (CET time zone), suitable for family and life commitments. The position requires infrequent participation in meetings compatible with different time zones (approximately 2x/month).
    Compensation: ~45 k€ gross per year (+ a variable part), to be negotiated according to experience and location.
    Application:
    To apply, email [email protected] a CV (including at least two references) and cover letter explaining your fit with the job.
    We will review applications as we receive them. We would prefer to find someone able to start by September 2023 (but can be flexible for the right person).
    For any questions, contact [email protected].
    We are an equal opportunity employer and value diversity within our organization. We do not discriminate on the basis of ethnicity, religion, color, national origin, gender, sexual orientation, age, marital status, ...
    5 min
  • EA - Some governance research ideas to prevent malevolent control over AGI and why this might matter a hell of a lot by Jim Buhler
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Some governance research ideas to prevent malevolent control over AGI and why this might matter a hell of a lot, published by Jim Buhler on May 23, 2023 on The Effective Altruism Forum.
    Epistemic status: I spent only a few weeks reading/thinking about this. I could have asked more people to give me feedback so I could improve this piece but I’d like to move on to other research projects and thought throwing this out there was still a good idea and might be insightful to some.
    Summary
    Many power-seeking actors will want to influence the development/deployment of artificial general intelligence (AGI). Some of them may have malevolent(-ish) preferences which they could satisfy on massively large scales if they succeed at getting some control over (key parts of the development/deployment of) AGI. Given the current rate of AI progress and dissemination, the extent to which those actors are a prominent threat will likely increase.
    In this post:
    I differentiate between different types of scenarios and give examples.
    I argue that 1) governance work aimed at reducing the influence of malevolent actors over AGI does not necessarily converge with usual AGI governance work – which is as far as I know – mostly focused on reducing risks from “mere” uncautiousness and/or inefficiencies due to suboptimal decision-making processes, and 2) the expected value loss due to malevolence, specifically, might be large enough to constitute an area of priority in its own right for longtermists.
    I, then, list some research questions that I classify under the following categories:
    Breaking down the conditions for an AGI-related long-term catastrophe from malevolence
    Redefining the set of actors/preferences we should worry about
    Steering clear from information/attention hazards
    Assessing the promisingness of various interventions
    How malevolent control over AGI may trigger long-term catastrophes?
    (This section is heavily inspired by discussions with Stefan Torges and Linh Chi Nguyen. I also build on Das Sarma and Wiblin’s (2022) discussion.
    We could divide the risks we should worry about into those two categories: Malevolence as a risk factor for AGI conflict and Direct long-term risks from malevolence.
    Malevolence as a risk factor for AGI conflict
    Clifton et al. (2022) write:
    Several recent research agendas related to safe and beneficial AI have been motivated, in part, by reducing the risks of large-scale conflict involving artificial general intelligence (AGI). These include the Center on Long-Term Risk’s research agenda, Open Problems in Cooperative AI, and AI Research Considerations for Human Existential Safety (and this associated assessment of various AI research areas). As proposals for longtermist priorities, these research agendas are premised on a view that AGI conflict could destroy large amounts of value, and that a good way to reduce the risk of AGI conflict is to do work on conflict in particular.
    In a later post from the same sequence, they explain that one of the potential factors leading to conflict is conflict-seeking preferences (CSPs) such as pure spite or unforgivingness. While AGIs might develop CSPs by themselves in training (e.g., because there are sometimes advantages to doing so; see, e.g., Abreu and Sethi 2003), they might also inherit them from malevolent(-ish) actors. Such an actor would also be less likely to want to reduce the chance of CSPs arising by “accident”.
    This actor can be a legitimate decisive person/group in the development/deployment of AGI (e.g., a researcher at a top AI lab, a politician, or even some influencer whose’ opinion is highly respected), but also a spy/infiltrator or external hacker (or something in between these last two).
    Direct long-term risks from malevolence
    For simplicity, say we are concerned about the risk of some AGI ending up with...
    35 min
  • LW - Coercion is an adaptation to scarcity; trust is an adaptation to abundance by Richard Ngo
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Coercion is an adaptation to scarcity; trust is an adaptation to abundance, published by Richard Ngo on May 23, 2023 on LessWrong.
    There’s a sense in which I’m postulating a trillion dollar bill lying on the ground. If the skills I’ve talked about so far in this sequence would lead to so much more flourishing, why haven’t they become far more common? I’ve already given a partial answer: that our evolutionary environment was much more dangerous than our current environment. But I want to extend this to a more general answer: that coercion is an adaptation to scarcity; and that we only very recently left the era in which scarcity was the dominant feature of people’s lives.
    Under conditions of scarcity, you don’t have enough slack that you can afford to take risks. If misbehavior from any individual in the group would risk the lives of many others, you have to coerce them into staying in line; if a loss on any gamble would leave you ruined, then you need to avoid taking those gambles even when they’re winning in expectation. (Poker players think a lot about this when trying to manage their bankroll—if a game has high-enough stakes that they’d be out of money if they lost, they have to avoid it, even if they expect to make money there on average.)
    By contrast, in an abundant environment, you can take the optimal long-term strategy, even if there’s a risk that it’ll leave you way down in the short term. In particular, you can put effort into building trust with others, even though that leaves you more vulnerable to being let down or betrayed. With that trust, you can receive a huge range of gains from cooperation.
    Western societies are incredibly abundant in many ways. As a citizen, you face almost zero risk of starvation, dying in a war, or exile from your country; meanwhile deaths from most diseases and accidents are dramatically lower than in the past. There’s more career flexibility than there ever has been before: there are many routes to success, including self-employment. And society is far richer than it’s ever been before: the median person in a western society is incredibly wealthy both by historical standards and by the standards of most people across the world. Even welfare recipients are still well off by historical and global standards.
    What does it look like to internalize a feeling of abundance? It might involve not forcing yourself to study when you don’t feel like it; or being less defensive when people criticize you; or deciding to be less harsh on yourself about the possibility that you might fail. The more slack you have, the more of a risk you can take by doing this—in each case it’s a gamble, but one which more often than not pays off in the long run by reducing internal conflict and allowing you to act more coherently towards your goals. Similarly, excitement about the future is a gamble: if you get excited about something which doesn’t work out, you’ll be left in a worse position than if your expectations had stayed low the whole time. The way that rich people can get higher returns to investment because they can absorb more risk is analogous to how fear forces us to take suboptimal actions.
    What if you think, like I do, that we live at the hinge of history, and our actions could have major effects on the far future—and in particular that there’s a significant possibility of existential risk from AGI? I agree that this puts us in more of a position of scarcity and danger than we otherwise would be (although I disagree with those who have very high credence in catastrophe). But the more complex the problems we face, the more counterproductive scarcity mindset is. In particular, AGI safety requires creative paradigm-shifting research, and large-scale coordination; those are both hard to achieve from a scarcity mindset. In other words, coercion at a ...
    7 min
  • LW - 'Fundamental' vs 'applied' mechanistic interpretability research by Lee Sharkey
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: 'Fundamental' vs 'applied' mechanistic interpretability research, published by Lee Sharkey on May 23, 2023 on LessWrong.
    When justifying my mechanistic interpretability research interests to others, I've occasionally found it useful to borrow a distinction from physics and distinguish between 'fundamental' versus 'applied' interpretability research.
    Fundamental interpretability research is the kind that investigates better ways to think about the structure of the function learned by neural networks. It lets us make new categories of hypotheses about neural networks. In the ideal case, it suggests novel interpretability methods based on new insights, but is not the methods themselves.
    Examples include:
    A Mathematical Framework for Transformer Circuits (Elhage et al., 2021)
    Toy Models of Superposition (Elhage et al., 2022)
    Polysemanticity and Capacity in Neural Networks (Scherlis et al., 2022)
    Interpreting Neural Networks through the Polytope Lens (Black et al., 2022)
    Causal Abstraction for Faithful Model Interpretation (Geiger et al., 2023)
    Research agenda: Formalizing abstractions of computations (Jenner, 2023)
    Work that looks for ways to identify modules in neural networks (see LessWrong 'Modularity' tag).
    Applied interpretability research is the kind that uses existing methods to find the representations or circuits that particular neural networks have learned. It generally involves finding facts or testing hypotheses about a given network (or set of networks) based on assumptions provided by theory.
    Examples include
    Steering GPT-2-XL by adding an activation vector (Turner et al., 2023)
    Discovering Latent Knowledge in Language Models (Burns et al., 2022)
    The Singular Value Decompositions of Transformer Weight Matrices are Highly Interpretable (Millidge et al., 2022)
    In-context Learning and Induction Heads (Olsson et al., 2022)
    We Found An Neuron in GPT-2 (Miller et al., 2023)
    Language models can explain neurons in language models (Bills et al., 2023)
    Acquisition of Chess Knowledge in AlphaZero (McGrath et al., 2021)
    Although I've found the distinction between fundamental and applied interpretability useful, it's not always clear cut:
    Sometimes articles are part fundamental, part applied (e.g. arguably 'A Mathematical Framework for Transformer Circuits' is mostly theoretical, but also studies particular language models using new theory).
    Sometimes articles take generally accepted 'fundamental' -- but underutilized -- assumptions and develop methods based on them (e.g. Causal Scrubbing, where the key underutilized fundamental assumption was that the structure of neural networks can be well studied using causal interventions).
    Other times the distinction is unclear because applied interpretability feeds back into fundamental interpretability, leading to fundamental insights about the structure of computation in networks (e.g. the Logit Lens lends weight to the theory that transformer language models do iterative inference).
    Why I currently prioritize fundamental interpretability
    Clearly both fundamental and applied interpretability research are essential. We need both in order to progress scientifically and to ensure future models are safe.
    But given our current position on the tech tree, I find that I care more about fundamental interpretability.
    The reason is that current interpretability methods are unsuitable for comprehensively interpreting networks on a mechanistic level. So far, our methods only seem to be able to identify particular representations that we look for or describe how particular behaviors are carried out. But they don't let us identify all representations or circuits in a network or summarize the full computational graph of a neural network (whatever that might mean). Let's call the ability to do these things 'comprehensive interpretability' .
    We ...
    6 min
  • EA - Review of Animal Liberation Now by Richard Y Chappell
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Review of Animal Liberation Now, published by Richard Y Chappell on May 23, 2023 on The Effective Altruism Forum.
    Animal Liberation Now releases today! I received an advance copy for review, so will share some thoughts and highlights. (It feels a bit presumptuous to “review” such a classic text—obviously you should read it, no-one needs to await my verdict in order to know that—but hopefully there’s still some value in my sharing a few thoughts and highlights that stood out to me.)
    As Singer notes in his publication announcement, he considers it “a new book, rather than just a revision, because so much of the material in the book is new.” I’m embarrassed to admit that I never actually got around to reading the original Animal Liberation (aside from the classic first chapter, widely anthologized as ‘All Animals are Equal’, and commonly taught in intro ethics classes). So I can’t speak to any differences, except to note that the present book is very much “up to date”, focusing on describing the current state of animal experimentation and agriculture, and (in the final chapter) engaging with recent philosophical defenses of speciesism.
    Empirical Details
    This book is not exactly an enjoyable read. It describes, clearly and dispassionately, humanity’s abusive treatment of other animals. It’s harrowing stuff. To give just one example, consider our treatment of broiler chickens: they have been bred to grow so large they cannot support themselves or walk without pain (p. 118):
    The birds may try to avoid the pain by sitting down, but they have nothing to sit on except the ammonia-laden litter, which, as we saw earlier, is so corrosive that it can burn their bodies. Their situation has been likened to that of someone with arthritic leg joints who is forced to stand up all day. [Prof.] Webster has described modern intensive chicken production as “in both magnitude and severity, the single most severe, systematic example of man’s inhumanity to another sentient animal.”
    Their parents—breeder birds—are instead starved to keep their weight at a level that allows mating to occur, and for the birds to survive longer—albeit in a state of hunger-induced aggression and desperation. In short, we’ve bred these birds to be physically incapable of living happy, healthy lives. It’s abominable.
    Our treatment of dairy cows is also heartbreaking:
    Dairy producers must ensure that their cows become pregnant every year, for otherwise their milk will dry up. Their babies are taken from them at birth, an experience that is as painful for the mother as it is terrifying for the calf. The mother often makes her feelings plain by constant calling and bellowing for her calf—and this may continue for several days after her infant calf is taken away. Some female calves will be reared on milk substitutes to become replacements of dairy cows when they reach the age, at around two years, when they can produce milk. Some others will be sold at between one to two weeks of age to be reared for beef in fattening pens or feedlots. The remainder will be sold to veal producers. (p. 155)
    A glimmer of hope is offered in the story of niche dairy farms that produce milk “without separating the calves from their mothers or killing a single calf.” (p. 157) The resulting milk is more expensive, since the process is no longer “optimized” purely for production. But I’d certainly be willing to pay more to support a less evil (maybe even positively good!) treatment of farm animals. I dearly hope these products become more widespread.
    The book also relates encouraging legislation, especially in the EU and New Zealand, constraining the mistreatment of animals in various respects. The U.S. is more disheartening for the most part, but here’s one (slightly) positive note (p. 282):
    In the U.S. the joint impact of the changes in stat...
    14 min
  • LW - My May 2023 priorities for AI x-safety: more empathy, more unification of concerns, and less vilification of OpenAI by Andrew Critch
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: My May 2023 priorities for AI x-safety: more empathy, more unification of concerns, and less vilification of OpenAI, published by Andrew Critch on May 24, 2023 on LessWrong.
    I have a mix of views on AI x-risk in general — and on OpenAI specifically — that no one seems be able to remember, due to my views not being easily summarized as those of a particular tribe or social group or cluster. For some of the views I consider most neglected and urgently important at this very moment, I've decided to write them here, all-in-one-place to avoid presumptions that being "for X" means I'm necessarily "against Y" for various X and Y.
    Probably these views will be confusing to read, especially if you're implicitly trying to pin down "which side" of some kind of debate or tribal affiliation I land on. I don't tend to choose my beliefs in a way that's strongly correlated with or caused by the people I affiliate with. As a result, I apologize in advance if I'm not easily remembered as "for" or "against" any particular protest or movement or statement, even though I in fact have pretty clear views on most topics in this space... the views just aren't correlated according to the usual social-correlation-matrix.
    Anyhoo:
    Regarding "pausing": I think pausing superintelligence development using collective bargaining agreements between individuals and/or states and/or companies is a good idea, along the lines of FLI's "Pause Giant AI Experiments", which I signed early and advocated for.
    Regarding OpenAI, I feel overall positively about them:
    I think OpenAI has been a net-positive influence for reducing x-risk from AI, mainly by releasing products in a sufficiently helpful-yet-fallible form that society is now able to engage in less-abstract more-concrete public discourse to come to grips AI and (soon) AI-risk.
    I've found OpenAI's behaviors and effects as an institution to be well-aligned with my interpretations of what they've said publicly. That said, I'm also sympathetic to people other than me who expected more access to models or less access to models than what OpenAI has ended up granting; but my personal assessment, based on my prior expectations from reading their announcements, is "Yeah, this is what I thought you told us you'd do... thanks!". I've also found OpenAI's various public testimonies, especially to Congress, to move the needle on helping humanity come to grips with AI x-risk in a healthy and coordinated way (relative to what would happen if OpenAI made their testimony and/or products less publicly accessible). I also like their charter, which creates tremendous pressure on them from their staff and the public to behave in particular ways. This leaves me, on-net, a fan of OpenAI.
    Given their recent post on Governance of Superintelligence, I can't tell if their approach to superintelligence is something I do or will agree with, but I expect to find that out over the next year or two, because of the openness of their communications and stance-taking. And, I appreciate the chance to for me, and the public, to engage in dialogue with them about it.
    I think the world is vilifying OpenAI too much, and that doing so is probably net-negative for existential safety. Specifically, I think people are currently over-targeting OpenAI with criticism that's easy to formulate because of the broad availability of OpenAI's products, services, and public statements. This makes them more vulnerable to attack than other labs, and I think piling onto them for that is a mistake from an x-safety perspective, in the "shooting the messenger" category. I.e., over-targeting OpenAI with criticism right now is pushing present and future companies toward being less forthright in ways that OpenAI has been forthright, thereby training the world to have less awareness of x-risk and weaker collective orien...
    13 min