The Nonlinear Library

The Nonlinear Library

By The Nonlinear FundEducation
Download on the App Store

All The Nonlinear Library episodes

  • LW - Transcript: Yudkowsky on Bankless follow-up Q&A by vonk
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Transcript: Yudkowsky on Bankless follow-up Q&A, published by vonk on February 28, 2023 on LessWrong.
    This follow-up Q&A took place shortly after the podcast was released. It talks more about the hosts' reactions to the episode, clears some questions about AI takeover pathways & alignment difficulties (like "why can't we just ask AIs to help solve the alignment?"); OpenAI/Silicon Valley & what should these companies be doing instead; Eliezer's take on doomerism; what would a surviving distant future look like. Let me know if you can clear up some of the [??] places (here is the original transcript alongside audio).michaelwong.eth: Good afternoon. Good morning, wherever you are. Got to be one of those, I bet. It's another Monday, minting Monday with Bankless. So I hope that you guys got to listen to the episode this morning about AI. And I have a hard time pronouncing this gentleman's name, but I think it's Eliezer. So I got Lucas on the call. I got Ryan on the call. I got David on the call. What's up everybody?
    David Hoffman: Yo, yo, how are you feeling? Alright?
    Ryan Sean Adams: Hey, still live. How you feeling? David?
    David: [laugs] Pretty good. Pretty good. Just you know, everyday trying to push existential dread to the background.
    Ryan: Yeah, me too. Especially since last Monday when we recorded this thing. Mike, Lucas, How're you guys doing?
    0x_Lucas: Doing pretty good. Also kind of going through my own mini existential crisis right now and just trying to survive. One day at a time.
    michaelwong.eth: I'm living large. I didn't know that the Mr. Roboto part of that song was so late in the in the song. So thanks, everybody for sticking with me through that. But it's kind of relevant today. A little bit relevant. So we're gonna jump into that in just a moment.
    Ryan: Guys, let's can we get into logistics first. So what are we doing here today, Lucas and Mike?
    0x_Lucas: Yeah, absolutely. So we are on our Monday mint number six of the year. So for those familiar, we mint our flagship Bankless podcast every Monday as a limited edition collectible on sound protocol. So you can go ahead and mint these at collectibles.bankless.com and part of it we like to host these little live Twitter spaces just so everyone has a live [??]. Ryan and David'd love to kind of do a debrief on the episode. And hopefully we have Eliezer joining us and I'd also probably butchered his name, but yeah, hopefully he is able to join us in the next few minutes here, but overall, just wanted to debrief on the episode talk about the men. And yeah, get your guys's thoughts.
    Ryan: Well, I'm definitely gonna be minting this one. That's for sure. And I hope the AI doesn't kill me for it in the future. This is a pretty unique episode, in my mind, David. This is one that caught me by the most surprise I think of anything we recorded. In that, we had an agenda, and then it took a different direction. It was a very interesting direction to pursue but what I wasn't quite ready for. I went and I spent the weekend actually I listened this episode again. I actually enjoy hearing it more than I think I enjoyed actually recording it, for whatever reason – some of the things sunk in a little bit better – but how did you receive it on the other side of this, David?
    David: Yeah, so the AI alignment problem was like a rabbit hole that I remember going down in my like, I think like college days. And so this is always like something that I had in the back of my head. And I think that's that's like why you and I have reacted differently to it. And I know you you went down the rabbit hole too. But it's like it was just something that I thought kind of everyone knew about and we just all understood that it was like, futile. It was like a thought experiment that was futile to really like reason about because there is no solu...
    43 min
  • EA - Autonomy and Manipulation in Social Movements by SeaGreen
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Autonomy and Manipulation in Social Movements, published by SeaGreen on February 27, 2023 on The Effective Altruism Forum.
    In this essay, I want to share a perspective I have been applying to evaluate movement-building efforts that have helped me understand a feeling that there is “something off”. This is not supposed to be a normative judgement about people building social movements, just a lens that has changed the way I evaluate my own behaviour.
    Examples of optimisation in social movements
    Suppose you are running a retreat (a sort of themed 3-ish day group residential work trip) aimed at getting more people interested in a social movement. You mean well: the social movement seems like an important one and having more people interested in it should help more people down the line. You want to do a good job, to get as many people as interested in the movement as possible, so you try to work out how to optimise for this goal. Here are some things you might say:
    “In our experience, young people are more open-minded, so we should focus on reaching out to them”.
    “We should host the retreat in a remote location that’s fun and free from distraction”.
    “Let’s try to build a sense of community around this social movement: this will make people feel more supported, motivated and inspired”.
    “We will host presentations and discussions for people at the retreat. People will learn better surrounded by people also interested in the ideas”.
    Framed in this way, these suggestions sound fairly innocuous and are probably an effective way to get people to be more interested in the social movement. However, there seems to be something fishy about them. Here is each thing framed in another way.
    “Younger people are more susceptible to our influence, so we should focus on reaching out to them”.1
    “Let's host the event in a remote location that separates people from other social pressures, and the things that ground them in their everyday lives”.
    “Let’s build strong-social bonds, dependent on believing in the ideas of this movement, increasing the cost of changing their values down the road”.
    “We can present the ideas of the movement in this unusual social context, in which knowledge of the ideas corresponds directly to social status: we, the presenters, are the most knowledgable and authoritative and the attendees who are most ‘in-crowd’ will know most about the ideas”.2
    Either set of framings can describe why the actions are effective. In truth, I think the first set is overly naive, and the second is probably too cynical. Further, I understand that there are plenty of settings in which the cynical framings could apply, and they could be hard to avoid. That said, I think they point to useful concepts that can be useful “flags” to check one’s behaviour against.
    How I understand autonomy and manipulation
    I want to put forward conceptions of “autonomy” and “manipulation”. Although I don’t claim these capture exactly how every person uses the words, or that they refer to any natural kind, I do think having these concepts available to you is useful. Since these concepts were clarified to me, I have frequently used them as a perspective to look at my behaviour, and frequently they have changed my actions.
    As I understand it, a person’s choice or action is more autonomous when they are able to make it via a considered decision process in accordance with their values.3 The most autonomous decisions are made with time for consideration, accurate, sufficient and balanced information, and free from social or emotional pressure. Here is an example of an action that is less autonomous:
    I don’t act very autonomously when I scroll to watch my 142nd TikTok of the day. Had I distanced myself and reflected, I would have chosen to go for a walk instead, but the act of scrolling is so fast that I never engaged...
    14 min
  • LW - $20 Million in NSF Grants for Safety Research by Dan H
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: $20 Million in NSF Grants for Safety Research, published by Dan H on February 28, 2023 on LessWrong.
    After a year of negotiation, the NSF has announced a $20 million request for proposals for empirical AI safety research.
    Here is the detailed program description.
    The request for proposals is broad, as is common for NSF RfPs. Many safety avenues, such as transparency and anomaly detection, are in scope:
    "reverse-engineering, inspecting, and interpreting the internal logic of learned models to identify unexpected behavior that could not be found by black-box testing alone"
    "Safety also requires... methods for monitoring for unexpected environmental hazards or anomalous system behaviors, including during deployment."
    Note that research that has high capabilities externalities is explicitly out of scope:
    "Proposals that increase safety primarily as a downstream effect of improving standard system performance metrics unrelated to safety (e.g., accuracy on standard tasks) are not in scope."
    Thanks to OpenPhil for funding a portion the RfP---their support was essential to creating this opportunity!
    Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.
    2 min
  • AF - $20 Million in NSF Grants for Safety Research by Dan H
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: $20 Million in NSF Grants for Safety Research, published by Dan H on February 28, 2023 on The AI Alignment Forum.
    After a year of negotiation, the NSF has announced a $20 million request for proposals for empirical AI safety research.
    Here is the detailed program description.
    The request for proposals is broad, as is common for NSF RfPs. Many safety avenues, such as transparency and anomaly detection, are in scope:
    "reverse-engineering, inspecting, and interpreting the internal logic of learned models to identify unexpected behavior that could not be found by black-box testing alone"
    "Safety also requires... methods for monitoring for unexpected environmental hazards or anomalous system behaviors, including during deployment."
    Note that research that has high capabilities externalities is explicitly out of scope:
    "Proposals that increase safety primarily as a downstream effect of improving standard system performance metrics unrelated to safety (e.g., accuracy on standard tasks) are not in scope."
    Thanks to OpenPhil for funding a portion the RfP---their support was essential to creating this opportunity!
    Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.
    2 min
  • EA - Why I think it's important to work on AI forecasting by Matthew Barnett
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Why I think it's important to work on AI forecasting, published by Matthew Barnett on February 27, 2023 on The Effective Altruism Forum.
    Note: this post is a transcript of a talk I gave at EA Global: Bay Area 2023.
    These days, a lot of effective altruists are working on trying to make sure AI goes well. But I often worry that, as a community, we don’t yet have a clear picture of what we’re really working on.
    The key problem is that predicting the future is very difficult, and in general, if you don’t know what the future will look like, it’s usually hard to be sure that any intervention we do now will turn out to be highly valuable in hindsight.
    When EAs imagine the future of AI, I think a lot of us tend to have something like the following picture in our heads.
    At some point, maybe 5, 15, 30 years from now, some AI lab somewhere is going to build AGI. This AGI is going to be very powerful in a lot of ways. And we’re either going to succeed in aligning it, and then the future will turn out to be bright and wonderful, or we’ll fail, and the AGI will make humanity go extinct, and it’s not yet clear which of these two outcomes will happen yet.
    Alright, so that’s an oversimplified picture. There’s lots of disagreement in our community about specific details in this story. For example, we sometimes talk about whether there will be one AGI or several. Or about whether there will be a fast takeoff or a slow takeoff.
    But even if you’re confident about some of these details, I think there are plausibly some huge open questions about the future of AI that perhaps no one understands very well.
    Take the question of what AGI will look like once it’s developed.
    If you asked an informed observer in 2013 what AGI will look like in the future, I think it’s somewhat likely they’d guess it’ll be an agent that we’ll program directly to search through a tree of possible future actions, and select the one that maximizes expected utility, except using some very clever heuristics that allows it to do this in the real world.
    In 2018, if you asked EAs what AGI would look like, a decent number of people would have told you that it will be created using some very clever deep reinforcement learning trained in a really complex and diverse environment.
    And these days in 2023, if you ask EAs what they expect AGI to look like, a fairly high fraction of people will say that it will look like a large language model: something like ChatGPT but scaled up dramatically, trained on more than one modality, and using a much better architecture.
    That’s just my impression of how people’s views have changed over time. Maybe I’m completely wrong about this. But the rough sense I’ve gotten while in this community is that people will often cling to a model of what future AI will be like, which frequently changes over time. And at any particular time, people will often be quite overconfident in their exact picture of AGI.
    In fact, I think the state of affairs is even worse than how I’ve described it so far. I’m not even sure if this particular question about AGI is coherent. The term “AGI” makes it sound like there will be some natural class of computer programs called “general AIs” that are sharply distinguished from this other class of programs called “narrow AIs”, and at some point – in fact, on a particular date – we will create the “first” AGI. I’m not really sure that story makes much sense.
    The question of what future AI will look like is a huge question, and getting it wrong could make the difference between a successful research program, and one that never went anywhere. And yet, it seems to me that, as of 2023, we still don’t have very strong reasons to think that the way we think about future AI will end up being right on many of the basic details.
    In general I think that uncertainty about the future of ...
    16 min
  • EA - Every Generator Is A Policy Failure [Works in Progress] by Lauren Gilbert
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Every Generator Is A Policy Failure [Works in Progress], published by Lauren Gilbert on February 27, 2023 on The Effective Altruism Forum.
    This article was spun out of a shallow investigation for Open Philanthropy; I thought it might be of interest to GHW folks.
    Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.
    1 min
  • EA - Milk EA, Casu Marzu EA by Jeff Kaufman
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Milk EA, Casu Marzu EA, published by Jeff Kaufman on February 27, 2023 on The Effective Altruism Forum.
    Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.
    1 min
  • LW - Fertility Rate Roundup #1 by Zvi
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Fertility Rate Roundup #1, published by Zvi on February 27, 2023 on LessWrong.
    Previously: On Car Seats as Contraception
    [Editor’s Note: This post assumes the perspective that more people having more children is good, actually. I will not be engaging with any of the arguments against this, of any quality, whether they be ‘AI or climate change is going to kill everyone’ or ‘people are bad actually,’ other than to state here that I strongly disagree. AI content will continue later this week.]
    A common theme in childhood roundups has been that existing efforts by governments, to increase the number of children born in various countries, have all been pathetically small in magnitude. The amounts spent and methods used pale in comparison to what is at stake. We reliably see signs that the policies work, even when poorly designed and implemented – the same way that when policies attempt to reduce the birth rate, those work as well.
    The core problem is the dose is too low.
    Yes. If you give parents money, more people choose to be parents.And the amount necessary to make this happen is, if you crunch the numbers, an amount rapidly aging societies can’t afford not to pay.
    The other theme is, as I discuss in On Car Seats as Contraception, that there are lots of other government policies that have much bigger impacts on the felt costs of having and raising children, including the lifestyle and status impacts of raising children.
    This is a roundup of related efforts that have crossed my desk recently, to illustrate that this is a highly solvable problem.
    Childcare
    Child care in America continues to be super expensive. People who understand economics understand that this is true because we combine large purchasing subsidies with onerous baseline requirements that drive up costs. Whereas you could (at least partly) solve this problem in the style of Vermont, by doing much less of both these things – removing price barriers for the bottom half and removing subsidizes for at least the top half, instead Giving Parents Money mostly in the form of lower taxes.
    I also continue to think that it is madness to subsidize and massively favor professionally provided child care over family provided child care, where as far as I can tell most everyone agrees children are better off with family provided child care, which we are now essentially pricing out of the market. While in other contexts, of course, massively favoring family provided care via the tax code.
    Alternatively, you could do what DC does (direct).
    Requiring a college degree to provide child care is one of those places I fail the ITT.
    Meanwhile Department of State proposes gutting the Au Pair program. They of course refer to this as things like ‘strengthening the educational component.’ By all reports I’ve seen the au pair program is insanely great as it is except it isn’t large enough. Win for everyone involved.
    If prospective parents confidently knew they could participate in the Au Pair program, I would predict a substantial increase in the birth rate. This could be a full plan. Houses or apartments and locations could be chosen with this in mind, and life could be much easier to manage and predict.
    A study in Finland finds that paying moms to stay home results in them staying home substantially more and working substantially less, including in the long term, whereas subsidizing child care and thus work (and by extension effectively taxing staying at home) has the opposite effect (although the paper’s results seem to not technically be ‘significant’, so salt as needed). You don’t say. You can either prefer to have mothers stay home or prefer to have mothers work, and people respond to incentives. You can get whatever change you want if you care enough. You do need to choose.
    Parental Leave
    At People’s Policy Project, Matt Bruenig...
    18 min
  • AF - Counting-down vs. counting-up coherence by Tsvi Benson-Tilsen
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Counting-down vs. counting-up coherence, published by Tsvi Benson-Tilsen on February 27, 2023 on The AI Alignment Forum.
    [Metadata: crossposted from. First completed 25 October 2022.]
    Counting-down coherence is the coherence of a mind viewed as the absence of deviation downward in capability from ideal, perfectly efficient agency: the utility left on the table, the waste, the exploitability.
    Counting-up coherence is the coherence of a mind viewed as the deviation upward in capability from a rock: the elements of the mind, and how they combine to perform tasks.
    What determines the effects of a mind?
    Supranormally capable minds can have large effects. To control those effects, we'd have to understand what determines the effects of a mind.
    Pre-theoretically, we have the idea of "values", "aims", "wants". The more capable a mind is, the more it's that case that what the mind wants, is what will happen in the world; so the mind's wants, its values, determine the mind's effect on the world.
    A more precise way of describing the situation is: "Coherent decisions imply consistent utilities". A mind like that is incorrigible: if it knows it will eventually be more competent than any other mind at pushing the world towards high-utility possibilities, then it does not defer to any other mind. So to understand how a mind can be corrigible, some assumptions about minds and their values may have to be loosened.
    The question remains, what are values? That is, what determines the effects that a mind has on the world, besides what the mind is capable of doing or understanding? This essay does not address this question, but instead describes two complementary standpoints from which to view the behavior of a mind insofar as it has effects.
    Counting-down coherence
    Counting-down coherence is the coherence of a mind viewed as the absence of deviation downward in capability from ideal, perfectly efficient agency: the utility left on the table, the waste, the exploitability.
    Counting-down coherence could also be called anti-waste coherence, since it has a flavor of avoiding visible waste, or universal coherence, since it has a flavor of tracking how much a mind everywhere conforms to certain patterns of behavior.
    Some overlapping ways of describing counting-down incoherence:
    Exploitable, Dutch bookable, pumpable for resources. That is, someone could make a set of trades with the mind that leaves the mind worse off, and could do so repeatedly to pump the mind for resources. See Garrabrant induction.
    VNM violating. Choosing between different outcomes, or different probabilities of different outcomes, in a way that doesn't satisfy the Von Neumann–Morgenstern axioms, leaves a mind open to being exploited by Dutch books. See related LessWrong posts.
    Doesn't maximize expected utility. A mind that satisfies the VNM axioms behaves as though it maximizes the expected value of a fixed utility function over atomic (not probabilistic) outcomes. So deviating from that policy exposes a mind to Dutch books.
    Missed opportunities. Leaving possible gains on the table; failing to pick up a $20 bill lying on the sidewalk.
    Opposing pushes. Working at cross-purposes to oneself; starting to do X one day, and then undoing X the next day; pushing and pulling on the door handle at the same time.
    Internal conflict. At war with oneself; having elements of oneself that try to harm each other or interfere with each other's functioning.
    Inconsistent beliefs, non-Bayesian beliefs. Sometimes acting as though X and sometimes acting as though not-X, where X is something that is either true or false. Or some more complicated inconsistency, or more generally failing to act as though one has a Bayesian belief state and belief revisions. Any of these also open one up to being Dutch booked.
    Inefficient allocation. Choosing to inve...
    22 min
  • EA - Help GiveDirectly kill "teach a man to fish" by GiveDirectly
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Help GiveDirectly kill "teach a man to fish", published by GiveDirectly on February 27, 2023 on The Effective Altruism Forum.
    We need your creative ideas to solve a problem: how to convince the world of the wisdom of giving directly. Will you submit to our proverb contest?
    Hi, we need your creative ideas to solve a problem: how to convince the world of the wisdom of giving directly. Will you submit to our proverb contest? The most common critique of giving cash without conditions is a fear of dependency, which comes in the form of: “Give a man a fish, feed him for a day. Teach a man to fish, feed him for a lifetime.”
    We’ve tried to disabuse folks of this paternalistic idea by showing that often people in poverty know how to fish but cannot afford the boat. Or they don’t want to fish; they want to sell cassava. Also, we’re not giving fish; we’re giving money, and years after getting it, people are better able to feed themselves. Oh, and even if you do teach them skills, it’s less effective than giving cash. Phew!
    Yet, despite our efforts, the myth remains.
    The one thing we haven’t tried: fighting proverb with (better) proverb. That’s where you come in. We’re crowdsourcing ideas that capture the dignity and logic of giving directly. SUBMIT YOUR DIRECT GIVING PROVERB (and add your ideas to the comments too!)
    The best suggestions are not a slogan, but a saying — simple, concrete, evocative (e.g.). Submit your ideas by next Friday, March 3, and then we'll post the top 3 ideas on Twitter for people to vote on the winner.
    The author of the winning adage will win a video call with a GiveDirectly staff member to learn more about our work one-on-one. Not feeling creative? Share with your friends who are.
    Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.
    2 min