The Nonlinear Library

The Nonlinear Library

By The Nonlinear FundEducation
Download on the App Store

All The Nonlinear Library episodes

  • EA - 'Artificial Intelligence Governance under Change' (PhD dissertation) by MMMaas
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: 'Artificial Intelligence Governance under Change' (PhD dissertation), published by MMMaas on September 15, 2022 on The Effective Altruism Forum.
    This is a link for my PhD dissertation, ‘Artificial Intelligence Governance under Change: Foundations, Facets, Frameworks’. I originally submitted this to the University of Copenhagen’s Faculty of Law in September 2020, and presented it in April 2021 (see also defense presentation slides & handout).
    TLDR: this dissertation discusses approaches and choices in regime design for the global governance of advanced (and transformative) AI. To do so, it draws on concepts and frameworks from the fields of technology regulation (‘sociotechnical change’; ‘governance disruption’), international law, and global governance studies (‘regime complexity’).
    In slogan form, the project explores how we may govern a changing technology, in a changing world, using governance systems that may themselves be left changed.
    In slightly more detail: the project discusses how AI governance regimes and institutions may need to be adapted, to take into account three facets of ‘change’ that will characterize or at least influence (T)AI governance in the coming few decades:
    the ways in which (especially under continual multipolar deployment scenarios) ongoing changes in AI capabilities and downstream applications will tend to create constant adaptation pressure on earlier AI-focused treaties, institutions, or legal norms, and unclarity for _when _and _why _new instruments are needed (lens: sociotechnical change)
    the ways in which AI systems may be increasingly used within international law, to support (or to contest) the establishment, monitoring, enforcement and/or arbitration of international treaty regimes, thereby shifting the possibility frontier on global cooperation (lens: governance disruption)
    The ways in which the architecture of global governance is undergoing structural changes over the last two decades, in ways that affect what kind of instruments (e.g. comprehensive formal treaty regimes vs. fragmented ecologies of informal institutions) will be more or less viable in governing (T)AI (lens: regime complexity).
    Sections that may be of particular interest to this audience, include:
    Chapter 2.1 (a balanced argument on the likely stakes of AI, written for an unfamiliar–and likely skeptical–audience);
    Chapter 2.2 (characterization of AI as global governance problem) & 2.3. (overview of AI governance avenues and instruments under international law);
    Chapter 4.4. (overview of distinct types of ‘problem logics’ of six types of AI governance challenges)
    Chapter 5.1 & 5.2 (history & taxonomy of the ways in which new technologies can disrupt, automate, or augment international law, with application to AI)
    Chapter 6.1 & 6.2 (primer on ‘regime complexity’ framework and approach to fragmented global governance)
    Chapter 7 (step-by-step framework discussing how to analyze an (emerging/proposed) AI governance regime in terms of its: (1) origins and foundations; (2) topology and organisation; (3) evolution over time; (4) effects of its organization, and (5) strategies to maintain AI governance regime efficacy, resilience and coherence;
    Why post this now? Obviously, a tremendous amount has happened in the two years since I submitted this–in terms of technical AI progress; -in global AI governance developments; and in community-internal debates over Transformative AI timelines,[1] -risks,[2] and -governance.[3] Nonetheless, I felt I’d share this now, as I believe that:
    most of the dissertation’s analysis, especially its AI governance-specific claims, have held up very well, fairly well, or functionally well;[4]
    the project introduces a set of more general governance design frameworks, with links to existing bodies of work on institutional design and legal automa...
    5 min
  • LW - Transhumanism, genetic engineering, and the biological basis of intelligence. by fowlertm
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Transhumanism, genetic engineering, and the biological basis of intelligence., published by fowlertm on September 14, 2022 on LessWrong.
    I recently had a fascinating conversation with Dr. Steven Hsu, theoretical physicist and serial entrepreneur, about computational genomics and genetic engineering.
    There were a number of really interesting takeaways which I think would be of interest to LWians.
    Apparently, genetic engineering in the form of embryo selection is very far along. They can already select for things like eye color and propensity for diseases. The only reason they're not selecting for intelligence yet is because there is such a strong stigma and because there's not currently enough labeled training data for the algorithms.
    The algorithms involved are simpler than you might think, it mostly appears to be basic regression models regularized with LI penalties (as those handle sparse data much better)
    Dr. Hsu does think it's worth worrying about this technology's potential to worsen inequality, and we kick the ethics of that around for a little bit.
    Epigenetics plays less of a role in determining life outcomes than many of us have been led to believe.
    We also discuss whether AGI or genetic engineering will lead to the first superhuman intelligence.
    Hope you enjoy it!
    Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.
    2 min
  • LW - What's the longest a sentient observer could survive in the Dark Era? by Raemon
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: What's the longest a sentient observer could survive in the Dark Era?, published by Raemon on September 15, 2022 on LessWrong.
    Periodically I look at this graph, and I'm like, "holy shit, that is so much Nothing. Wut. Like, they converted it to log-scale so it would fit into my brain and it's still so much Nothing, Jezus Christ."
    Eliezer's Dath Ilan verse has this song for people who died before the invention of cryonics:
    Even if the stars should die in heavenOur sins can never be undoneNo single death wil be forgivenWhen fades at last the last lit sun.Then in the cold and silent blackAs light and matter endWe’ll have ourselves a last look backAnd toast an absent friend.
    And I find myself wondering "What are the physical limits on 'some sentient computation surviving into the dark era?" Assuming we don't learn anything new about physics, or acausal-trade our way into another universe or whatever, what's the farthest to the right on this timeline you could possibly get, if you goal was to have some computation that, as late as possible, looks around at the universe one last time, comprehends it, and says "yep, still full of infinite blackness. Goodbye, world."
    Vaguely relevant things I'm vaguely aware of include:
    Something something you can somehow harvest energy from black holes. Does this plausibly get us to the end of the Black Hole Era or is that energy eventually too weaksauce?
    Space is, like, really cold (especially when All Has Become Darkness), so any time-capsule robot would face an uphill battle not radiating it's energy into the void.
    On long enough timescales, protons (probably?) decay, and... like, your robot would eventually... just... disintegrate?
    If anyone knows enough physics to satisfy my random curiosity, uh, I'd appreciate it.
    Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.
    2 min
  • EA - It’s not effective to call everything effective and how (not) to name a new organisation by James Odene [User-Friendly]
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: It’s not effective to call everything effective and how (not) to name a new organisation, published by James Odene [User-Friendly] on September 15, 2022 on The Effective Altruism Forum.
    User-Friendly is an EA-focussed marketing agency supporting organisations with a wide range of marketing, branding and communications requirements. As such, we come across many misconceptions regarding what the criteria should be for a successful brand and try to mitigate the movement from continuously making the same mistakes.
    It is our aim moving forward to not only support organisations with their marketing, branding and communications needs, but to use this platform to bring well-established, robust, industry-standard marketing principles into this space, to ensure that all EA marketing is as impactful as possible.
    As the movement rapidly grows, many new organisations are emerging each month, all in need of a distinctive brand and name. I will use this forum entry to highlight some common mistakes when naming an organisation and share key ways to ensure that your chosen name will work for you, not against you.
    The number one job of a brand name is to be distinctive, not descriptive.
    Think about the biggest and most successful brands out there; Apple, Google, Amazon, Microsoft - none of which use their name to describe their service or product.
    The brand name isn’t a product description, it’s a mark of identification, a decision simplifier, a way of narrowing down the decision field for your target audience in a way that makes sure you’re the one they think of. The Von Restorff Effect (the cognitive bias towards things that stand out) should be considered your best friend when it comes to branding.
    Do you know why Apple was named Apple, and not ‘Effective Computer Solutions’? Because it was a distinct and simplistic choice, far more memorable than anything that may have explained the product. It was also Job’s contention that they would benefit from a company name beginning with ‘A’ as it would appear earlier in the phonebook than a competitor. Overly simplistic right? Maybe not.
    Comically, Apple was actually initially called, ‘Apple Computer Co’, but as the only distinct element - Apple stuck and the rest was binned.
    Play where you can own the field
    Okay, so you’re starting a new consultancy organisation, why not call it, for example, ‘Effective Consultancy’? I’ll use this example as a springboard to both express why this might not be a good idea, and suggest alternative criteria to consider when choosing a name.
    This name is not distinct or memorable
    This name will be far less ‘sticky’ than something more distinct. The primary role of a brand is to drive top of mind and support organisations to win against a huge host of competition for the attention of your target audience. Our brains are misers, they like easy decisions, and often, what comes to mind first or easiest is more likely to be chosen (see more on system 1 and 2 thinking). Sure, this name explains the service, but will it be the first thing that comes to mind when I come to needing this service? It is less likely than if the name was more distinct?
    As this movement grows, and there are more and more organisations using ‘Effective this’ and ‘Effective that’, the name becomes increasingly muddied, blurred and entangled in a host of very similar sounding names. This is not helpful to either the success of the individual organisation nor the success of the EA movement as a whole.
    The name is not ownable
    Not only will this organisation struggle to carve out its own unique place in the mind of the target audience, but equally the organisation will see the same challenge within the brands online presence. If all ‘Effective X’s’ try to optimise their SEO and run Google ads (both of which they almost certainly should be doing)...
    9 min
  • AF - AGI safety researchers should focus (only/mostly) on deceptive alignment by Marius Hobbhahn
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: AGI safety researchers should focus (only/mostly) on deceptive alignment, published by Marius Hobbhahn on September 15, 2022 on The AI Alignment Forum.
    Comment: after I wrote the first draft of this post, Evan Hubinger published “How likely is deceptive alignment” in which he argues that deceptive alignment is the default outcome of NNs trained with SGD. The post is very good and I recommend everyone to read it. As a consequence, I rewrote my post to cover less of the “why is deceptive alignment likely” to the more high-level arguments of “why should we focus on deceptive alignment” and adopted Evan’s nomenclature to prevent confusion.
    I’d like to thank Lee Sharkey, Richard Ngo and Evan Hubinger for providing feedback on a draft of this post.
    TL;DR: No matter from which angle I look at it, I always arrive at the conclusion that deceptive alignment is either a necessary component or greatly increases the harm of bad AI scenarios. This take is not new and many(most?) people in the alignment community seem to already believe it but I think there are reasons to write this post anyway. Firstly, newer members of the alignment community are sometimes not aware (at least I wasn’t in the beginning) that more senior people often implicitly talk about deceptive alignment when they talk about misalignment.
    Secondly, there seems to be some disagreement about whether a powerful misaligned AI is deceptive by default or whether such a thing as a “corrigibly aligned” AI can even exist. I hope this post clarifies the different positions.
    Epistemic status: Might have reinvented the wheel. Most of the content is probably not new for most people within the alignment community. Hope it is helpful anyway.
    Update: after a discussion in the comments, I want to make some clarifications:1. My definition of deception is a bit inconsistent throughout the post. I'm not sure what the best definition is but I think it is somewhere between "We have no clue what the model is doing" (which doesn't include active deception) to "The model is actively trying to hide something from us". Both seem like important failure modes.2. I don't think deception is orthogonal to understanding other failure modes like getting what we measure. Deception can be a component of other failure modes. 3. This post should not be interpreted as "everything that isn't direct work on deception is bad" and more like "we should think about how other research relates to deception". For example, AI forecasting still seems super valuable to me.
    However, I think one of the main sources of value from AI forecasting comes from having better models of future AI capabilities and those might be used to predict when the model becomes deceptive and what happens if it does. 4. I'm not sure about all of this. I still find many aspects of alignment confusing and hard to grasp but I'm mildly confident in the statement "most failure modes look much worse when you add deception" and thus my takeaway is something like "deception is not everything but probably a good thing to work on right now".
    Definition - deceptive alignment
    By deceptive alignment, I mean an AI system that seems aligned to human observers and passes all relevant checks but is, in fact, not aligned and ultimately aims to achieve another non-aligned goal. In Evan’s post, this means that the NN has actively made an incomplete proxy of the true goal a terminal goal. Note, that the AI is aware of the fact that we wanted it to achieve a different goal and therefore actively acts in ways that humans will perceive as aligned. If the AI accidentally followed a different goal, e.g. due to a misunderstanding or a lack of capabilities, this is not deceptive alignment but described as corrigible alignment in Evan’s post. Corrigible alignment essentially means that the AI currently has an inc...
    20 min
  • EA - EA Organization Updates: September 2022 by Lizka
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: EA Organization Updates: September 2022, published by Lizka on September 14, 2022 on The Effective Altruism Forum.
    These monthly posts originated as the "Updates" section of the EA Newsletter. Organizations submit their own updates, which we edit for clarity.
    Job listings that these organizations highlighted are at the top of this post. Some of the jobs have extremely pressing deadlines.
    You can see previous updates on the "EA Organization Updates (monthly series)" topic page, or in our repository of past newsletters. Notice that there’s also an “org update” tag, where you can find more news and updates that are not part of this consolidated series.
    The organizations are in alphabetical order, starting with L this month.
    Job listings
    Consider also exploring jobs listed on “Job listing (open).”
    Applications closing soon
    80,000 Hours:
    Full Stack Web Developer, (London / Remote, apply by 26 September)
    Centre for Effective Altruism:
    Project Manager: Community Health (Remote, apply by 30 September)
    Retreats Associate (Remote, apply by 10 October)
    EA Global Events Associate (Remote, apply by 10 October)
    Community Events Associate (Remote, apply by 10 October)
    Effective Ventures Operations
    Operations Assistant (Preferred Oxford, London or Boston / Remote, apply by 19 September)
    Executive Assistant (Preferred Oxford / Remote, apply by 19 September)
    Project Manager Oxford EA Professional Hub (Oxford UK, apply by 19 September)
    Senior Bookkeeper / Accountant (Preferred Oxford, London or Boston / Remote, apply by 19 September)
    Finance Associate (Preferred Oxford/ Remote, apply by 7 October)
    Operations Associate - Salesforce Admin (Preferred Oxford/ Remote, apply by 12 October)
    Forethought Foundation:
    Communications Director (Oxford / Remote, apply by 2 October)
    Researcher / Writer (Oxford / Remote, apply by 2 October)
    Other roles
    Against Malaria Foundation:
    Junior and Senior Operations Manager
    Donations Administrator
    Junior and Senior Software Engineer
    Alignment Research Center:
    Generalist Technical Researcher and a webdev Engineer
    GiveWell
    Operations Assistant (Oakland, CA)
    Senior Researchers and Senior Research Associates (Remote / Oakland, CA)
    Content Editors (Remote / Oakland, CA)
    Quantified Uncertainty Research Institute (QURI)
    Technical Product Manager (Berkeley / Remote)
    Full Stack Software Engineer (Berkeley / Remote)
    Programming Language or Scientific Software Engineer (Berkeley / Remote)
    Berkeley Existential Risk Initiative:
    Deputy Director (NYC / Remote)
    Rethink Priorities:
    Multiple research team roles (Remote)
    Organizational updates
    Legal Priorities Project
    The Legal Priorities Project ran its Legal Priorities Summer Institute from Sep 3–8 in Oxford. 35 talented law students and graduates from 18 different countries and all 5 continents gathered to discuss issues related to future generations and existential risk. Participants with various degrees of familiarity with longtermism participated in workshops, talks, and group discussions with invited experts on issues relating to legal longtermism.
    LPP also announced the winning submissions from their Cost–Benefit Analysis Writing Competition. The winning submissions can be found here.
    LPP funded two student reading groups at ITAM, with curricula centering on The Precipice and the Introduction to EA Program, respectively.
    Quantified Uncertainty Research Institute
    QURI released Squiggle, a free new web–based programming language for probabilistic estimation. Squiggle is made especially for effective altruist cost–effectiveness estimation. Squiggle is currently in public “Early Access”, but should be usable for many effective altruist use cases.
    Rethink Priorities (RP)
    Research:
    Epoch – an AI research initiative that RP is incubating – did a deep–dive investigation of machine learning model size trends and hypothesized rea...
    20 min
  • LW - When does technical work to reduce AGI conflict make a difference?: Introduction by JesseClifton
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: When does technical work to reduce AGI conflict make a difference?: Introduction, published by JesseClifton on September 14, 2022 on LessWrong.
    This is a pared-down version of a longer draft report. We went with a more concise version to get it out faster, so it ended up being more of an overview of definitions and concepts, and is thin on concrete examples and details. Hopefully subsequent work will help fill those gaps.
    Sequence summary
    Some researchers are focused on reducing the risks of conflict between AGIs. In this sequence, we’ll present several necessary conditions for technical work on AGI conflict reduction to be effective, and survey circumstances under which these conditions hold. We’ll also share some thoughts on promising directions for research and intervention to prevent AGI conflict.
    This post
    We give a breakdown of necessary conditions for technical work on AGI conflict reduction to make a difference: “AGIs won’t always avoid conflict, despite it being materially costly” and “intent alignment is either insufficient or unnecessary for conflict reduction work to make a difference.” (more)
    When would AGIs engage in conflict?
    To assess the claim that AGIs would figure out how to avoid conflict, we give a breakdown (which we believe is exhaustive) of the causes of conflict between rational agents. Some kinds of conflict can be avoided if the agents are sufficiently capable of credible commitment and disclosure of private information. Two barriers to preventing conflict by these means are (i) strategic pressures early in multipolar AI takeoff that make it risky to implement cooperation-improving technologies, and (ii) fundamental computational hurdles to credible commitment and disclosure. Other causes of conflict — miscoordination and conflict-prone preferences — can’t be solved by credible commitment and information disclosure alone. (more)
    We give examples of technical work aimed at each of the causes of conflict surveyed in (2a). (more)
    When is intent alignment sufficient or necessary to reduce AGI conflict?
    We survey conditions under which intent-aligned AGIs could engage in conflict that is not endorsed by their overseers. Scenarios include: locking in architectures that can’t solve informational and commitment problems; the unintentional corruption of human preferences by AGIs; and catastrophes which leave human overseers unable to prevent intent-aligned AGIs from acting on misunderstandings of their preferences. (more)
    We consider whether it is possible to shape an AGI’s conflict behavior even if it is misaligned. We suggest that coarse modifications to an AI’s training distribution could make a difference by causing it to acquire a less conflict-prone (e.g., spiteful or risk-seeking) mesa-objective before it starts “playing the training game”. (more)
    We tentatively conclude that two of the most promising directions for technical research to reduce AGI conflict are (a) prescriptive work on bargaining aimed at preventing intent-aligned AIs from locking in catastrophic bargaining decisions, and (b) conceptual and empirical work on the origins of conflict-prone preferences, which we might be able to prevent even if AGI becomes misaligned at an early stage. Work on measuring and training cooperative behaviors in contemporary large language models could feed into both of these research streams. (more)
    This sequence assumes familiarity with intermediate game theory.
    Necessary conditions for technical work on AGI conflict to have a counterfactual impact
    Could powerful AI systems engage in catastrophic conflict? And if so, what are the best ways to reduce this risk? Several recent research agendas related to safe and beneficial AI have been motivated, in part, by reducing the risks of large-scale conflict involving artificial general intelligence (AGI). ...
    14 min
  • EA - Forecasting thread: How does AI risk level vary based on timelines? by elifland
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Forecasting thread: How does AI risk level vary based on timelines?, published by elifland on September 14, 2022 on The Effective Altruism Forum.
    Crossposted to LessWrong
    While there have been many previous surveys asking about the chance of existential catastrophe from AI and/or AI timelines, none as far as I'm aware have asked about how the level of AI risk varies based on timelines. But this seems like an extremely important parameter for understanding the nature of AI risk and prioritizing between interventions.
    Contribute your forecasts below. I'll write up my forecast rationales in an answer and encourage others to do the same.
    Elicit Prediction (forecast.elicit.org/binary/questions/726u_r_XC)
    Elicit Prediction (forecast.elicit.org/binary/questions/8HnkL7Ekf)
    Elicit Prediction (forecast.elicit.org/binary/questions/dlPDzREFM)
    Elicit Prediction (forecast.elicit.org/binary/questions/EHL2INyQq)
    Elicit Prediction (forecast.elicit.org/binary/questions/AETDAUmNF)
    Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.
    2 min
  • EA - Future Matters #5: supervolcanoes, AI takeover, and What We Owe the Future by Pablo
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Future Matters #5: supervolcanoes, AI takeover, and What We Owe the Future, published by Pablo on September 14, 2022 on The Effective Altruism Forum.
    Even if we think the prior existence view is more plausible than the total view, we should recognize that we could be mistaken about this and therefore give some value to the life of a possible future. The number of human beings who will come into existence only if we can avoid extinction is so huge that even with that relatively low value, reducing the risk of human extinction will often be a highly cost-effective strategy for maximizing utility, as long as we have some understanding of what will reduce that risk.
    Katarzyna de Lazari-Radek & Peter Singer
    Future Matters is a newsletter about longtermism. Each month we collect and summarize longtermism-relevant research, share news from the longtermism community, and feature a conversation with a prominent researcher. You can also subscribe on Substack, listen on your favorite podcast platform and follow on Twitter. Future Matters is also available in Spanish.
    Research
    William MacAskill’s What We Owe the Future was published, reaching the New York Times Bestseller list in its first week and generating a deluge of media for longtermism. We strongly encourage readers to get a copy of the book, which is filled with new research, ideas, and framings, even for people already familiar with the terrain. In the next section, we provide an overview of the coverage the book has received so far.
    In Samotsvety's AI risk forecasts, Eli Lifland summarizes the results of some recent predictions related to AI takeover, AI timelines, and transformative AI by a group of seasoned forecasters. In aggregate, the group places 38% on AI existential catastrophe, conditional on AGI being developed by 2070, and 25% on existential catastrophe via misaligned AI takeover by 2100. Roughly four fifths of their overall AI risk is from AI takeover. They put 32% on AGI being developed in the next 20 years.
    John Halstead released a book-length report on climate change and longtermism and published a summary of it on the EA Forum. The report offers an up-to-date analysis of the existential risk posed by global warming. One of the most important takeaways is that extreme warming seems significantly less likely than previously thought: the probability of >6°C warming was thought to be 10% a few years ago, whereas it now looks <1% likely. (For much more on this topic, see our conversation with John that accompanied last month’s issue.)
    In a similar vein, the Good Judgment Project asked superforecasters a series of questions on Long-term risks and climate change, the results of which are summarized by Luis Urtubey (full report here).
    The importance of existential risk reduction is often motivated by two claims: that the value of humanity’s future is vast, and that the level of risk is high. David Thorstad’s Existential risk pessimism and the time of perils notes that these stand in some tension, since the higher the overall risk, the shorter humanity’s expected lifespan. This tension dissolves, however, if one holds that existential risk will decline to near-zero levels if humanity survives the next few centuries of high risk. This is precisely the view held by most prominent thinkers on existential risk, e.g. Toby Ord (see The Precipice) and Carl Shulman (see this comment).
    In Space and existential risk, the legal scholar Chase Hamilton argues that existential risk reduction should be a central consideration shaping space law and policy. He outlines a number of ways in which incautious space development might increase existential risk, pointing out that our current laissez-faire approach fails to protect humanity against these externalities and offering a number of constructive proposals. We are in a formativ...
    30 min
  • LW - ACT-1: Transformer for Actions by Daniel Kokotajlo
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: ACT-1: Transformer for Actions, published by Daniel Kokotajlo on September 14, 2022 on LessWrong.
    ACT-1 can take a high-level user request and execute it. The user simply types a command into the text box and ACT-1 does the rest. In this example, this requires repeatedly taking actions and observations over a long time horizon to fulfill a single goal.
    While we’re excited that these systems can transform what people can do on a computer, we clearly see that they have the potential to cause harm if misused or misaligned with user preferences. Our goal is to build a company with large-scale human feedback at the center — models will be evaluated on how well they satisfy user preferences, and we will iteratively evaluate how well this is working as our product becomes more sophisticated and load-bearing.
    Daniel's commentary: To be clear, this is very much just a cool demo, not much better than WebGPT as far as I can tell, and not surprising at all that this level of capability is possible.
    Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.
    2 min