The Nonlinear Library

The Nonlinear Library

By The Nonlinear FundEducation
Download on the App Store

All The Nonlinear Library episodes

  • EA - Quick PSA: 8000hours.org (not 80000hours.org) is a malicious scam site by Will Bradshaw
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Quick PSA: 8000hours.org (not 80000hours.org) is a malicious scam site, published by Will Bradshaw on January 13, 2023 on The Effective Altruism Forum.
    Just what the title says: 8000hours.org is a malicious scam/phishing site. If you mean to go to 80000hours.org and come across something strange/confusing, be aware and take care to keep yourself safe. (EDIT: And maybe think twice about going to the scam site yourself; one user contacted me to say that their browser had autocompleted information into its fields.)
    I've accidentally gone to this site instead of 80000hours.org a few times over the past year, and each time it's been a different scam, so it seems to change hands/tactics frequently. Some of them were convincing enough that I'm worried they'd catch an unwary user.
    I've notified 80K about this; this post isn't meant to be a complaint or criticism about them, just to make others aware of the danger. That said, people from other orgs reading this might do well to think about what other domains close to their own someone might use to prey on their users, and what they might do to pre-emptively prevent this.
    Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.
    2 min
  • EA - The Capability Approach by ryancbriggs
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: The Capability Approach, published by ryancbriggs on January 13, 2023 on The Effective Altruism Forum.
    This post outlines the capability approach to thinking about human welfare. I think that this approach, while very popular in international development, is neglected in EA. While the capability approach has problems, I think that it provides a better approach to thinking about improving human welfare than approaches based on measuring happiness or subjective wellbeing (SWB) or approaches based on preference satisfaction. Finally, even if you disagree that the capability approach is best, I think this post will be useful to you because it may clarify why many people and organizations in the international development or global health space take the positions that they do. I will be drawing heavily on the work of Amartya Sen, but I will often not be citing specific texts because I’m an academic and getting to write without careful citations is thrilling.
    This post will have four sections. First, I will describe the capability approach. Second, I will give some simple examples that illustrate why I think that aiming to maximize capabilities is the best way to do good for people. I’ll frame these examples in opposition to other common approaches, but my goal here is mostly constructive and to argue for the capability approach rather than against maximizing, for example, SWB. Third, I will describe what I see as the largest downsides to the capability approach as well as possible responses to these downsides. Fourth and finally, I will explain my weakly-held theory that a lot of the ways that global health or international development organizations, including GiveWell, behave owes to the deep (but often unrecognized) influence of the capability approach on their thought.
    The capability approach
    The fundamental unit of the value in the capability approach is a functioning, which is anything that you can be or do. Eating is a functioning. Being an EA is a functioning. Other functionings include: being a doctor, running, practicing Judaism, sleeping, and being a parent. Capabilities are options to be or do a functioning. The goal of the capability approach is not to maximize the number of capabilities available to people, it is instead to maximize the number of sets of capabilities. The notion here is that if you maximized simply the number of capabilities then you might enable someone to be: a parent or employed outside the home. But someone might want to do both. If you’re focusing on maximizing the number of sets of capabilities then you’ll end up with: parent, employed, both parent and employed, and neither.
    The simple beauty of this setup is that it is aiming to maximize the options that people have available to them, from which they then select the group of functionings that they want most. This is why one great book about this approach is entitled “Development as Freedom.” The argument is that development is the process of expanding capabilities, or individual freedom to live the kind of life that you want.
    I will come to criticisms later on, but one thing people may note is that this approach will lead to a lot of sets of capabilities and we will need some way to rank them or condense the list. In theory, we would want to do this based on how much people value each capability set. I will discuss this issue in more detail in the third section.
    Examples of why I love the capability approach
    Here I’ll lay out a few examples that show why I think the capability approach is the best way to think about improving human welfare.
    First, in opposition to preference-satisfaction approaches, the capability approach values options not taken. I think this accords with most of our intuitions, and that it takes real work for economics to train it out of people. Here are two examples...
    16 min
  • EA - Economic Theory & Global Prioritization (summer 2023): Apply now! by trammell
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Economic Theory & Global Prioritization (summer 2023): Apply now!, published by trammell on January 13, 2023 on The Effective Altruism Forum.
    Background: I'm an economics DPhil student at Oxford and research associate at GPI.
    Last summer, I organized a course on “Topics in Economic Theory and Global Prioritization”. It aimed to provide a rigorous introduction to a selection of topics in economic theory that appear especially relevant to the project of doing the most good. It was designed primarily for economics graduate students, and strong, late-stage undergraduate students, considering careers in global priorities research.A summary of how it went, including links to the 2022 syllabus, can be found here.
    Applications are now open for summer 2023! It will probably be run similarly to how it was run in 2022, with minor changes summarized in the post linked above.A provisional syllabus and program outline for 2023 can be found here.
    Application deadline: February 18 (11:59pm GMT)When you will hear back: March 4 or earlierLocation: Oxford, UKCourse dates: August 12–25 (+optional unstructured week to September 2)
    The course is sponsored by the Forethought Foundation. If accepted, your transportation to and from Oxford, and accommodation in Oxford for the duration of the course, will be provided.
    Click here for more info and to apply.
    Please don’t hesitate to email [email protected], or comment below, if you have any comments or questions.
    Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.
    2 min
  • AF - [ASoT] Simulators show us behavioural properties by default by Arun Jose
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: [ASoT] Simulators show us behavioural properties by default, published by Arun Jose on January 13, 2023 on The AI Alignment Forum.
    I’m trying out making a few posts with less polish and smaller scope, to iterate more quickly on my thoughts and write about some interesting ideas in isolation before having fully figured them out. Expect middling confidence in any conclusions drawn, and occasionally just chains of reasoning without fully contextualized conclusions.
    Let’s say we prompt a language model to do chain-of-thought reasoning while answering some question. What are we actually seeing? One answer I’ve seen people use, often implicitly, is along the lines that we’re seeing the underlying reasoning being done by the model to get at the answer.
    I think this take is wrong, and in this post I’ll explain the frame I generally use for this.
    Languages describe processes
    How do we use text? Well, broadly we use it to describe our own thoughts, or to describe something that’s happening externally in the real world. In both of these cases, language is used as logs of real, mechanistic processes.
    To be clear, I think these are very good and comprehensive logs of many relevant processes. Contrary to some contexts in which I’ve heard this description used, I think that large language models trained on these logs learn the processes underlying them pretty well, and end up understanding and being able to use the same (or pretty similar) mechanics at the limit.
    The key point I want to make with the idea that text primarily functions as reports on processes is that this is what we’re observing with a language model’s outputs. One way to think about it is that our prompt shines a lens onto some specific part of a simulation, and the language model just keeps describing everything under the lens.
    In other words, when we prompt simulators, what we should expect to see are the behavioural characteristics of the simulation, not the mechanistic characteristics of the underlying processes.
    Focusing the lens onto the mechanics
    I use the terms “behavioural and mechanistic characteristics” in a broader sense than is generally used in the context of something in our world - I expect we can still try to focus our lens on the mechanics of some simulacrum (they’re plausibly also part of the simulation, after all), I just expect it to be a lot harder because you have to make sure the lens actually is focused on the underlying process. One way to view this is that you’re still getting reports on the simulation from an omniscient observer, you just have to craft your prompt such that that the observer knows to describe the pretty small target of the actual mechanistic process.
    So just saying “explain your reasoning step-by-step” doesn’t work - for example, consider the experiments showing that incorrect chains of thought in a few-shot setting result in better performance than some human-engineered chain-of-thought prompts. You aren’t seeing the internal computation of some process that thinks in wildly wrong ways and gets to the correct answer anyway, you’re just giving information to the underlying mechanics of the model on what process it should be simulating, and you never really see that process’ computations. What you do end up seeing is something like the incoherent ramblings of a simulacrum that thinks in the right way and doesn’t express these thoughts or is really bad at expressing these thoughts.
    In other words, your prompt wasn’t well-specified enough to report on true underlying mechanics. I expect getting this right to be hard for a number of reasons, some isomorphic to the problems facing ELK, but a pretty underrated one in this context in my opinion is that in some cases (probably not the ones where we compute basic math) the actual computations being done by the target simulacrum m...
    6 min
  • EA - Tyler Cowen on effective altruism (December 2022) by peterhartree
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Tyler Cowen on effective altruism (December 2022), published by peterhartree on January 13, 2023 on The Effective Altruism Forum.
    In December 2022, Tyler Cowen gave a talk on effective altruism. It was hosted by Luca Stroppa and Theron Pummer at the University of St Andrews.
    You can watch the talk on YouTube, listen to the podcast version, or read the transcript below.
    Transcript
    The transcript was generated by OpenAI Whisper. I made a couple of minor edits and corrections for clarity.
    Hello everyone, and thank you for coming to this talk. Thank you Theron for the introduction.
    I find effective altruism is what people actually want to talk about, which is a good thing. So I thought I would talk about it as well. And I'll start by giving two big reasons why I'm favorably inclined, but then work through a number of details where I might differ from effective altruism.
    So let me give you what I think are the two big pluses. They're not the only pluses. But to me, they're the two reasons why in the net ledger, it's strongly positive.
    The first is that simply as a youth movement, effective altruism seems to attract more talented young people than anything else I know of right now by a considerable margin. And I've observed this by running my own project, Emergent Ventures for Talented Young People. And I just see time and again, the smartest and most successful people who apply get grants. They turn out to have connections to the EA movement. And that's very much to the credit of effective altruism. Whether or not you agree with everything there, that to me is a more important fact about the movement than anything else you might say about it. Unlike some philosophers, I do not draw a totally rigorous and clear distinction between what you might call the conceptual side of effective altruism and the more sociological side. They're somewhat intertwined and best thought of as such.
    The second positive point that I like about effective altruism is simply that what you might call traditional charity is so terrible, such a train wreck, so poorly conceived and ill thought out and badly managed and run that anything that waves its arms and says, hey, we should do better than this, again, whether or not you agree with all of the points, that has to be a big positive. So whether or not you think we should send more money to anti-malaria bed nets, the point is effective altruism is forcing us all to rethink what philanthropy should be. And again, that for me is really a very significant positive.
    Now before I get to some of the more arcane points of difference or at least different emphasis, let me try to outline some core propositions of effective altruism, noting I don't think there's a single dominant or correct definition. It's a pretty diverse movement. I learned recently there's like a sub movement, effective altruism for Christians. I also learned there's a sub sub movement, effective altruism for Quakers. So I don't think there's any one way to sum it all up, but I think you'll recognize these themes as things you see appearing repeatedly.
    So my first group of themes will be those where contemporary effective altruism differs a bit from classic utilitarianism. And then I'll give two ways in which effective altruism is fairly similar to classical utilitarianism.
    So here are three ways I think current effective altruism has evolved from classical utilitarianism and is different:
    The first is simply an emphasis on existential risk, the notion that the entire world could end, world of humans at least, and this would be a very terrible thing. I don't recall having read that, say, in Bentham or in John Stuart Mill. It might be in there somewhere, but it certainly receives far, far more emphasis today than it did in the 19th century.
    The second point, which I think is somewhat in c...
    29 min
  • AF - Tracr: Compiled Transformers as a Laboratory for Interpretability | DeepMind by Cinera Verinia
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Tracr: Compiled Transformers as a Laboratory for Interpretability | DeepMind, published by Cinera Verinia on January 13, 2023 on The AI Alignment Forum.
    Abstract
    Interpretability research aims to build tools for understanding machine learning (ML) models. However, such tools are inherently hard to evaluate because we do not have ground truth information about how ML models actually work. In this work, we propose to build transformer models manually as a testbed for interpretability research. We introduce Tracr, a "compiler" for translating human-readable programs into weights of a transformer model. Tracr takes code written in RASP, a domain-specific language (Weiss et al. 2021), and translates it into weights for a standard, decoder-only, GPT-like transformer architecture. We use Tracr to create a range of ground truth transformers that implement programs including computing token frequencies, sorting, and Dyck-n parenthesis checking, among others. To enable the broader research community to explore and use compiled models, we provide an open-source implementation of Tracr at this https URL.
    Twitter thread by Dr. Jim Fan
    Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.
    2 min
  • EA - [Linkpost] FLI alleged to have offered funding to far right foundation by Jens Nordmark
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: [Linkpost] FLI alleged to have offered funding to far right foundation, published by Jens Nordmark on January 13, 2023 on The Effective Altruism Forum.
    This seems concerning. It is claimed that the Future of Life Institute, run by MIT professor Max Tegmark, offered but did not pay out a grant to a Swedish far-right foundation. The character of this foundation and its associates is well-known in Sweden. Expo is an old and respected watchdog organization specialized on neo-nazism and related movements.
    Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.
    1 min
  • EA - Thread for discussing Bostrom's email and apology by Lizka
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Thread for discussing Bostrom's email and apology, published by Lizka on January 13, 2023 on The Effective Altruism Forum.
    The Forum is getting a bit swamped with discussions about Bostrom's email and apology. We’re making this thread where you can discuss the topic.
    All other posts on this topic will be marked as “Personal Blog” — people who opt in or have opted into seeing “Personal Blog” posts will see them on the Frontpage, but others won’t; they’ll see them only in Recent Discussion or in All Posts. (If you want to change your "Personal Blog" setting, you can do that by following the instructions here.)
    (Please also feel free to give us feedback on this thread and approach. This is the first time we’ve tried this in response to events that dominate Forum discussion. You can give feedback by commenting on the thread, or by reaching out to [email protected].)
    If you choose to participate in this discussion, please remember Forum norms. Chiefly,
    Be kind.
    Stay civil, at the minimum. Don’t sneer or be snarky. In general, assume good faith. We may delete unnecessary rudeness and issue warnings or bans for it.
    Substantive disagreements are fine and expected. Disagreements help us find the truth and are part of healthy communication.
    Please try to remember that most people on the Forum are here for collaborative discussions about doing good.
    Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.
    2 min
  • EA - Should the forum be structured such that the drama of the day doesn't occur on the front page? by Chris Leong
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Should the forum be structured such that the drama of the day doesn't occur on the front page?, published by Chris Leong on January 13, 2023 on The Effective Altruism Forum.
    I feel kind of hypocritical here, as I ended up commenting on a bunch of the posts related to the drama of the day, but here goes anyway...
    I think it's important for people to be able to criticize Effective Altruism. One of the things I love about this community is its openness to criticism.
    At the same time, I'm starting to worry that constantly having all this drama play out on the front page of the forum is very distracting. But what would be even more worrying is if we've now reached a certain size/level of attention where this is the new normal going forward.
    So I guess I feel it's gotten to the point where I feel that we have to discuss how to balance these twin interests. I think this is incredibly challenging. If we change how this site works to address this issue, I want to these policies be fair to people holding different viewpoints and on different sides of these issues. And this is tricky, if we decided "let's move drama of the day discussion to a separate section of the site", well then maybe that just leads to a lot of arguments about what counts as drama and people feeling their issues aren't being heard or that they're being treated unfairly.
    I don't actually know if there's any policy or site mechanics shift I would reflectively endorse after thinking through the consequences. But maybe someone thinks that they have a solution to this?
    Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.
    2 min
  • AF - Some Arguments Against Strong Scaling by Joar Skalse
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Some Arguments Against Strong Scaling, published by Joar Skalse on January 13, 2023 on The AI Alignment Forum.
    There are many people who believe that we will be able to get to AGI by basically just scaling up the techniques used in recent large language models, combined with some relatively minor additions and/or architectural changes. As a result, there are people in the AI safety community who now predict timelines of less than 10 years, and structure their research accordingly. However, there are also people who still believe in long(er) timelines, or at least that substantial new insights or breakthroughts will be needed for AGI (even if those breakthroughts in principle could happen quickly). My impression is that the arguments for the latter position are not all that widely known in the AI safety community. In this post, I will summarise as many of these arguments as I can.
    I will almost certainly miss some arguments; if so, I would be grateful if they could be added to the comments. My goal with this post is not to present a balanced view of the issue, nor is it to present my own view. Rather, my goal is just to summarise as many arguments as possible for being skeptical of short timelines and the "scaling is all you need" position.
    This post is structured into four sections. In the first section, I give a rough overview of the scaling is all you need-hypothesis, together with a basic argument for that hypothesis. In the second section, I give a few general arguments in favour of significant model uncertainty when it comes to arguments about AI timelines. In the third section, I give some arguments against the standard argument for the scaling is all you need-hypothesis, and in the fourth section, I give a few direct arguments against the hypothesis itself. I then end the post on a few closing words.
    Glossary:
    LLM - Large Language ModelSIAYN - Scaling Is All You Need
    The View I'm Arguing Against
    In this section, I will give a brief summary of the view that these arguments oppose, as well as provide a standard justification for this view. In short, the view is that we can reach AGI by more or less simply scaling up existing methods (in terms of the size of the models, the amount of training data they are given, and/or the number of gradient steps they take, etc). One version says that we can do this by literally just scaling up transformers, but the arguments will apply even if we relax this to allow scaling of large deep learning-based next-token predictors, even if they would need be given a somewhat different architecture, and even if some extra thing would be needed, etc.
    Why believe this? One argument goes like this:
    (1) Next-word prediction is AI complete. This would mean that if we can solve next-word prediction, then we would also be able to solve any other AI problem. Why think next-word prediction is AI complete? One reason is that human-level question answering is believed to be AI-complete, and this can be reduced to next-word prediction.
    (2) The performance of LLMs at next-word prediction improves smoothly as a function of the parameter count, training time, and amount of training data. Moreover, the asymptote of this performance trend is on at least human performance.
    () Hence, if we keep scaling up LLMs we will eventually reach human-level performance at next-word prediction, and therefore also reach AGI.
    An issue with this argument, as stated, is that GPT-3 already is better than humans at next-word prediction. So are both GPT-2 and GPT-1, in fact, see this link. This means that there is an issue with the argument, and that issue is that human-level performance on next-word prediction (in terms of accuracy) evidently is insufficient to attain human-level performance in question answering.
    There are at least two ways to amend the argument:
    (3) In...
    26 min