The Nonlinear Library

The Nonlinear Library

By The Nonlinear FundEducation
Download on the App Store

All The Nonlinear Library episodes

  • AF - A proposed method for forecasting transformative AI by Matthew Barnett
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: A proposed method for forecasting transformative AI, published by Matthew Barnett on February 10, 2023 on The AI Alignment Forum.
    In 2021, I proposed measuring progress in the perplexity of language models and extrapolating past results to determine when language models were expected to reach roughly "human-level" performance. Here, I build on that approach by introducing a more systematic and precise method of forecasting progress in language modeling that employs scaling laws to make predictions.
    The full report for this forecasting method can be found in this document. In this blog post I'll try to explain all the essential elements of the approach without providing excessive detail regarding the technical derivations.
    This approach can be contrasted with Ajeya Cotra's Bio Anchors model, providing a new method for forecasting the arrival of transformative AI (TAI). I will tentatively call it the "Direct Approach", since it makes use of scaling laws directly to make predictions about compute requirements for AI.
    Naturally, the Direct Approach is a very speculative framework and might end up being useless for forecasting TAI (in fact, I consider this the most likely outcome). Nonetheless, I'm hopeful that something like it can serve as a better foundation than current TAI timelines models, which I currently think are likely even worse. Note that there may be errors in the report and Colab notebook, as they were not extensively fact-checked.
    Some background
    In a nutshell, this approach is simply about taking the cross-entropy loss of an autoregressive model and trying to find a way of interpreting that quantity qualitatively: that is, something we can put on a chart and extrapolate until the quantity reaches a natural threshold that we identify with something important.
    In my 2021 post about predicting language model performance, I drew a trendline through a plot of language model perplexities on various benchmarks and noted when the trendline went through estimates of "human-level" perplexity. This approach felt reasonable to me at the time, but I now think it too easily hand-waved away some important details.
    The error of omission I committed in my old approach becomes more apparent when you think about language model performance from the perspective of scaling laws, for example the parametric scaling law from Hoffmann et al. 2022:
    Here, we see cross-entropy loss as a function of parameters N and training tokens D seen during training. Notably, if we take the limit as the number of parameters and training tokens goes to infinity, then we're left with E. Theoretically, E corresponds to the "entropy of natural text" under certain assumptions, which is precisely the thing I identified with "roughly human-level" performance in my previous post. In other words, if we take this scaling law naively, it seems as though it will take infinite compute to reach human-level performance.
    I believe the resolution to this apparent issue is to say that "human-level" performance will not be obtained when loss hits E, but rather some small level above E. How close to E is enough? Well, that's the question we tried to answer with this report.
    Summary of the Direct Approach
    We begin by considering a language task, which in this post will be scientific research for illustration. For simplicity, let's imagine that this task consists of writing high-quality research papers or reports, although more nuanced specifications are possible.
    Of course, real scientific research involves more than merely writing research papers. It involves proposing hypotheses, devising experiments, and collecting data, but for now, let's imagine that we can simplify all these steps into one step that involves writing high quality research papers. This simplification may not be entirely unrealistic, since if the ...
    18 min
  • AF - Why I’m not working on {debate, RRM, ELK, natural abstractions} by Steve Byrnes
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Why I’m not working on {debate, RRM, ELK, natural abstractions}, published by Steve Byrnes on February 10, 2023 on The AI Alignment Forum.
    [For background & spelling out the acronyms in the title, see: Debate (AI safety technique), Recursive Reward Modeling, Eliciting Latent Knowledge, Natural Abstractions.]
    When I say “Why I’m not working on X”, I am NOT trying to find a polite & diplomatic way to say “Nobody should work on X because X is unhelpful for AGI safety”. Hmm, OK, well, maybe it’s just a little bit that. But really, I don’t feel strongly. Instead, I think:
    A lot of disagreement about what a solution to technical AGI safety looks like is really downstream of disagreements about questions like “How will AGI be built? What will it look like? How will it work?”
    Nobody really knows the answers to those questions.
    So we should probably be contingency-planning, by going through any possible answers to those questions that at least some reasonable person finds plausible, and doing AGI safety research conditional on those answers being correct.
    But still, I have my own opinions about the answers to those questions, and obviously I think my opinions are right, and I am not going to work on something unless it makes sense on my own models. And since people ask me from time to time, it seems worth explaining why the various research programs in the post title do not seem to be a good use of time, on my own models of how AGI will be developed and what AGI will look like.
    I wrote this post quickly and did not run it by the people I’m (sorta) criticizing. Do not assume that I described anything fairly and correctly. Please leave comments, and I’ll endeavor to update this post or write a follow-up in the case of major errors / misunderstandings / mind-changes. (But maybe not until after the weekend.)
    (By the way: If I’m not working on any of those research programs, then what am I working on? See here. I listed six other projects that seem particularly great to me here, and there are many others besides.)
    1. Background
    1.1 “Trying” to figure something out seems both necessary & dangerous
    (Partly self-plagiarized from here.)
    Let’s compare two things: “trying to get a good understanding of some domain by building up a vocabulary of new concepts and their relations” versus “trying to win a video game”. At a high level, I claim they have a lot in common!
    In both cases, there are a bunch of possible “moves” you can make (you could think the thought “what if there’s some analogy between this and that?”, or you could think the thought “that’s a bit of a pattern; does it generalize?”, etc. etc.), and each move affects subsequent moves, in an exponentially-growing tree of possibilities.
    In both cases, you’ll often get some early hints about whether moves were wise, but you won’t really know that you’re on the right track except in hindsight.
    And in both cases, I think the only reliable way to succeed is to have the capability to repeatedly try different things, and learn from experience what paths and strategies are fruitful.
    Therefore (I would argue), a human-level concept-inventing AI needs “RL-on-thoughts”—i.e., a reinforcement learning system, in which “thoughts” (edits to the hypothesis space / priors / world-model) are the thing that gets rewarded.
    Next, consider some of the features that we plausibly need to put into this RL-on-thoughts system, for it to succeed at a superhuman level:
    Developing and pursuing instrumental subgoals—for example, suppose the AI is “trying” to develop concepts that will make it superhumanly competent at assisting a human microscope inventor. We want it to be able to “notice” that there might be a relation between lenses and symplectic transformations, and then go spend some compute cycles developing a better understanding of symplectic transform...
    15 min
  • AF - Conditioning Predictive Models: Open problems, Conclusion, and Appendix by Evan Hubinger
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Conditioning Predictive Models: Open problems, Conclusion, and Appendix, published by Evan Hubinger on February 10, 2023 on The AI Alignment Forum.
    This is the final of seven posts in the Conditioning Predictive Models Sequence based on the paper “Conditioning Predictive Models: Risks and Strategies” by Evan Hubinger, Adam Jermyn, Johannes Treutlein, Rubi Hudson, and Kate Woolverton. Each post in the sequence corresponds to a different section of the paper.
    7. Open problems
    We think that there are a wide variety of ways—both experimental and theoretical—in which our analysis could be expanded upon. Here, we’ll try to briefly lay out some of the future directions that we are most excited about—though note that this is only a sampling of some possible future directions, and is thus a highly incomplete list:
    Are pre-trained LLMs well-modeled as predictive models or agents?
    As pre-trained model scale increases, do markers of agentic behavior increase as well?
    See “Discovering Language Model Behaviors with Model-Written Evaluations” for some initial results on this question.
    To what extent do LLMs exhibit distributional generalization?
    Distributional generalization seems like evidence of acting as a generative/predictive model rather than just optimizing cross-entropy loss.
    To the extent that current LLMs are doing some sort of prediction, can we find evidence of that in their internal structure?
    Is the RLHF conditioning hypothesis true?
    How do markers of agentic behavior change as the amount of RLHF done increases, and under different RLHF fine-tuning regimes?
    See “Discovering Language Model Behaviors with Model-Written Evaluations” for some initial results on this question.
    For anything that an RLHF model can do, is there always a prompt that gets a pre-trained model to do the same thing? What about a soft prompt or a prompt chain?
    In addition to validating the extent to which RLHF models can be mimicked using techniques that are more clearly implementing a conditional, a positive result here could also provide an alternative to RLHF that allows us to get the same results without relying on the RLHF conditioning hypothesis at all.
    More generally, how similar are RLHF fine-tuned models to pre-trained models with fine-tuned soft prompts?
    The idea here being that a soft prompt is perhaps more straightforward to think of as a sort of conditional.
    To what extent do RLHF fine-tuned models exhibit distributional generalization?
    Relevant here for the same reason as in the pre-training case.
    To what extent can you recover the original pre-trained distribution/capabilities from an RLHF fine-tuned model?
    If an RLHF model no longer successfully solves some prediction task by default, how easy is it to turn back on that capability via additional fine-tuning, or did the RLHF destroy it completely?
    If it is generally possible to do this, it is some evidence that the original pre-trained distribution is still largely maintained in the RLHF model.
    How do markers of agentic behavior change as we change the RL reward? Is it very different between human-like and random rewards? What happens if we exactly invert the standard helpfulness reward?
    This can help test whether agency is coming from the specific choice of RL reward or the general process of RLHF.
    How do RLHF fine-tuned models differ from their own preference model, especially regarding markers of agentic behavior?
    To the extent that fine-tuned models get closer to their preference models as scale increases, preference models can serve as a proxy for future RLHF models.
    Are there ways of changing standard RLHF techniques to make them more likely to produce conditionals rather than agents?
    How do alternative, more myopic RL training schemes—such as the one described here—affect markers of agentic behavior? Can we use such techniques...
    18 min
  • LW - Cyborgism by NicholasKees
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Cyborgism, published by NicholasKees on February 10, 2023 on LessWrong.
    Thanks to Garrett Baker, David Udell, Alex Gray, Paul Colognese, Akash Wasil, Jacques Thibodeau, Michael Ivanitskiy, Zach Stein-Perlman, and Anish Upadhayay for feedback on drafts, as well as Scott Viteri for our valuable conversations.
    Executive summary: This post proposes a strategy for safely accelerating alignment research. The plan is to set up human-in-the-loop systems which empower human agency rather than outsource it, and to use those systems to differentially accelerate progress on alignment.
    Introduction: An explanation of the context and motivation for this agenda.
    Automated Research Assistants: A discussion of why the paradigm of training AI systems to behave as autonomous agents is both counterproductive and dangerous.
    Becoming a Cyborg: A proposal for an alternative approach/frame, which focuses on a particular type of human-in-the-loop system I am calling a “cyborg”.
    Failure Modes: An analysis of how this agenda could either fail to help or actively cause harm by accelerating AI research more broadly.
    Testimony of a Cyborg: A personal account of how Janus uses GPT as a part of their workflow, and how it relates to the cyborgism approach to intelligence augmentation.
    Terminology
    GPT: Large language models trained on next-token prediction. Most plans to accelerate research (including this one) revolve around leveraging GPTs specifically. I will mostly be using “GPT'' to gesture at the base models which have not been augmented using reinforcement learning.
    Autonomous Agent: An AI system which can be well modeled as having goals or preferences, and deliberately selects actions in order to achieve them (with limited human assistance).
    Capabilities research: Research which directly improves the capabilities of AI systems and thereby brings us closer to being able to train and deploy more powerful autonomous agents.
    Simulator: A class of AI system (of which GPT is a member). Simulators are generative predictive models, where the model makes a prediction (probability distribution) about how the state of a system will evolve, and then the state is updated by sampling from that prediction/distribution. The result is a process which “simulates” the training distribution, the limit of such a process being a system which faithfully generates trajectories sampled from the distribution implied by the training data.
    Disempowerment: The process of humans losing control of the long-term future to a powerful autonomous agent (or agents). This includes anything from our civilization being hijacked to outright human extinction.
    Introduction
    There is a lot of disagreement and confusion about the feasibility and risks associated with automating alignment research. Some see it as the default path toward building aligned AI, while others expect limited benefit from near term systems, expecting the ability to significantly speed up progress to appear well after misalignment and deception. Furthermore, progress in this area may directly shorten timelines or enable the creation of dual purpose systems which significantly speed up capabilities research.
    OpenAI recently released their alignment plan. It focuses heavily on outsourcing cognitive work to language models, transitioning us to a regime where humans mostly provide oversight to automated research assistants. While there have been a lot of objections to and concerns about this plan, there hasn’t been a strong alternative approach aiming to automate alignment research which also takes all of the many risks seriously.
    The intention of this post is not to propose an end-all cure for the tricky problem of accelerating alignment using GPT models. Instead, the purpose is to explicitly put another point on the map of possible strategies, and to add nuance to ...
    59 min
  • AF - Cyborgism by Nicholas Kees Dupuis
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Cyborgism, published by Nicholas Kees Dupuis on February 10, 2023 on The AI Alignment Forum.
    Thanks to Garrett Baker, David Udell, Alex Gray, Paul Colognese, Akash Wasil, Jacques Thibodeau, Michael Ivanitskiy, Zach Stein-Perlman, and Anish Upadhayay for feedback on drafts, as well as Scott Viteri for our valuable conversations.
    Executive summary: This post proposes a strategy for safely accelerating alignment research. The plan is to set up human-in-the-loop systems which empower human agency rather than outsource it, and to use those systems to differentially accelerate progress on alignment.
    Introduction: An explanation of the context and motivation for this agenda.
    Automated Research Assistants: A discussion of why the paradigm of training AI systems to behave as autonomous agents is both counterproductive and dangerous.
    Becoming a Cyborg: A proposal for an alternative approach/frame, which focuses on a particular type of human-in-the-loop system I am calling a “cyborg”.
    Failure Modes: An analysis of how this agenda could either fail to help or actively cause harm by accelerating AI research more broadly.
    Testimony of a Cyborg: A personal account of how Janus uses GPT as a part of their workflow, and how it relates to the cyborgism approach to intelligence augmentation.
    Terminology
    GPT: Large language models trained on next-token prediction. Most plans to accelerate research (including this one) revolve around leveraging GPTs specifically. I will mostly be using “GPT'' to gesture at the base models which have not been augmented using reinforcement learning.
    Autonomous Agent: An AI system which can be well modeled as having goals or preferences, and deliberately selects actions in order to achieve them (with limited human assistance).
    Capabilities research: Research which directly improves the capabilities of AI systems and thereby brings us closer to being able to train and deploy more powerful autonomous agents.
    Simulator: A class of AI system (of which GPT is a member). Simulators are generative predictive models, where the model makes a prediction (probability distribution) about how the state of a system will evolve, and then the state is updated by sampling from that prediction/distribution. The result is a process which “simulates” the training distribution, the limit of such a process being a system which faithfully generates trajectories sampled from the distribution implied by the training data.
    Disempowerment: The process of humans losing control of the long-term future to a powerful autonomous agent (or agents). This includes anything from our civilization being hijacked to outright human extinction.
    Introduction
    There is a lot of disagreement and confusion about the feasibility and risks associated with automating alignment research. Some see it as the default path toward building aligned AI, while others expect limited benefit from near term systems, expecting the ability to significantly speed up progress to appear well after misalignment and deception. Furthermore, progress in this area may directly shorten timelines or enable the creation of dual purpose systems which significantly speed up capabilities research.
    OpenAI recently released their alignment plan. It focuses heavily on outsourcing cognitive work to language models, transitioning us to a regime where humans mostly provide oversight to automated research assistants. While there have been a lot of objections to and concerns about this plan, there hasn’t been a strong alternative approach aiming to automate alignment research which also takes all of the many risks seriously.
    The intention of this post is not to propose an end-all cure for the tricky problem of accelerating alignment using GPT models. Instead, the purpose is to explicitly put another point on the map of possible strategies, ...
    59 min
  • AF - FLI Podcast: Connor Leahy on AI Progress, Chimps, Memes, and Markets (Part 1/3) by remember
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: FLI Podcast: Connor Leahy on AI Progress, Chimps, Memes, and Markets (Part 1/3), published by remember on February 10, 2023 on The AI Alignment Forum.
    We often prefer reading over listening to audio content, and have been testing transcribing podcasts using our new tool at Conjecture, Verbalize, with some light editing and formatting. We're posting highlights and transcripts of podcasts in case others share our preferences, and because there is a lot of important alignment-relevant information in podcasts that never made it to LessWrong.If anyone is creating alignment-relevant audio content and wants to transcribe it, get in touch with us and we can give you free credits!
    The podcast episode transcribed in this post is available here.
    Topics covered include:
    Defining artificial general intelligence
    What makes humans more powerful than chimps?
    Would AIs have to be social to be intelligent?
    Importing humanity's memes into AIs
    How do we measure progress in AI?
    Gut feelings about AI progress
    Connor's predictions about AGI
    Is predicting AGI soon betting against the market?
    How accurate are prediction markets about AGI?
    Books cited in the episode include:
    The Incerto Series by Nassim Nicholas Taleb
    The Selfish Gene, Richard Dawkins
    Various books on primates and animal intelligence by Frans De Wall
    Inadequate Equilibria by Eliezer Yudkowsky
    Highlights
    On intelligence in humans and chimps:
    We are more social because we're more intelligent and we're more intelligent because we are more social. These things are not independent variables. So at first glance, if you look at a human brain versus a chimp brain, it's basically the same thing. You see like all the same kind of structures, same kind of neurons, though a bunch of parameters are different. You see some more spindle cells, it's bigger. Human brain just has more parameters, it's just GPT-3 versus GPT-4...
    But really, the difference is, is that humans have memes. And I mean this in the Richard Dawkins sense of evolved, informational, programmatic virtual concepts that can be passed around between groups. If I had to pick one niche, what is the niche that humans are evolved for?
    I think the niche we're evolved for is memetic hosts.
    On benchmarks and scaling laws:
    Benchmarks are actually coordination technologies. They're actually social technologies. What benchmarks are fundamentally for is coordination mechanisms. The kind of mechanisms you need to use when you're trying to coordinate groups of people around certain things....
    So we have these scaling laws, which I think a lot of people misunderstand. So scaling laws give you these nice curves which show how the loss of performance on the model smoothly decreases as they get larger. These are actually terrible, and these actually tell you nothing about the model. They tell you what one specific number will do. And this number doesn't mean anything. There is some value in knowing the loss. But what we actually care about is can this model do various work? Can it do various tasks? Can it reason about its environment? Can it reason about its user?...
    So currently there are no predictive theories of intelligence gain or task. There is no theory that says once it reaches 74.3 billion parameters, then it will learn this task. There's no such theory. It's all empirical. And we still don't understand these things at all. I think there's, so another reason I'm kind of against benchmarks, and I'm kind of being a bit pedantic about this question is because I think they're actively misleading in the sense that people present them as if they mean something, but they just truly, truly don't. A benchmark in a vacuum means nothing.
    On the dangerous of having a good metric of progress towards AGI:
    So this is an interesting question. And not just from a scientific perspective, but it's als...
    1 hr 8 min
  • EA - Straightforwardly eliciting probabilities from GPT-3 by NunoSempere
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Straightforwardly eliciting probabilities from GPT-3, published by NunoSempere on February 9, 2023 on The Effective Altruism Forum.
    I explain two straightforward strategies for eliciting probabilities from language models, and in particular for GPT-3, provide code, and give my thoughts on what I would do if I were being more hardcore about this.
    Straightforward strategies
    Look at the probability of yes/no completion
    Given a binary question, like “At the end of 2023, will Vladimir Putin be President of Russia?” you can create something like the following text for the model to complete:
    Then we can compare the relative probabilities of completion to the “Yes,” “yes,” “No” and “no” tokens. This requires a bit of care. Note that we are not making the same query 100 times and looking at the frequencies, but rather asking for the probabilities directly:
    You can see a version of this strategy implemented here.
    A related strategy might be to look at what probabilities the model assigns to a pair of sentences with opposite meanings:
    “Putin will be the president of Russia in 2023”
    “Putin will not be the president of Russia in 2023.”
    For example, GPT-3 could assign a probability of 9 10^-N to the first sentence and 10^-N to the second sentence. We could then interpret that as a 90% probability that Putin will be president of Russia by the end of 2023.
    But that method has two problems:
    The negatively worded sentence has one word more, and so it might systematically have a lower probability
    GPT-3’s API doesn’t appear to provide a way of calculating the likelihood of a whole sentence.
    Have the model output the probability verbally
    You can directly ask the model for a probability, as follows:
    Now, the problem with this approach is that, untweaked, it does poorly.
    Instead, I’ve tried to use templates. For example, here is a template for producing reasoning in base rates:
    Many good forecasts are made in two steps.
    Look at the base rate or historical frequency to arrive at a baseline probability.
    Take into account other considerations and update the baseline slightly.
    For example, we can answer the question “will there be a schism in the Catholic Church in 2023?” as follows:
    There have been around 40 schisms in the 2000 years since the Catholic Church was founded. This is a base rate of 40 schisms / 2000 years = 2% chance of a schism / year. If we only look at the last 100 years, there have been 4 schisms, which is a base rate of 4 schisms / 100 years = 4% chance of a schism / year. In between is 3%, so we will take that as our baseline.
    The Catholic Church in Germany is currently in tension and arguing with Rome. This increases the probability a bit, to 5%.
    Therefore, our final probability for “will there be a schism in the Catholic Church in 2023?” is: 5%
    For another example, we can answer the question “${question}” as follows:
    That approach does somewhat better. The problem is that sometimes the base rate approach isn’t quite relevant, because sometimes we have neither a historical record—e.g,. global nuclear war. And sometimes we can't straightforwardly rely on the lack of a historical track record: VR headsets haven’t really been adopted in the mainstream, but their price has been falling and their quality rising, so making a forecast solely looking at the historical lack of adoption might lead one astray.
    You can see some code which implements this strategy here.
    More elaborate strategies
    Various templates, and choosing the template depending on the type of question
    The base rate template is only one of many possible options. We could also look at:
    Laplace rule of succession template: Since X was first possible, how often has it happened?
    “Mainstream plausibility” template: We could prompt a model to simulate how plausible a well-informed member of the public thinks that an eve...
    7 min
  • EA - “Community” posts have their own section, subforums are closing, and more (Forum update February 2023) by Lizka
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: “Community” posts have their own section, subforums are closing, and more (Forum update February 2023), published by Lizka on February 10, 2023 on The Effective Altruism Forum.
    TL;DR: We’re kicking off a test where “Community” posts don’t go on the Frontpage with other posts but have their own section below the fold. We’re also closing subforums and focusing on improving “core topic” pages to let people go deeper on specific sub-fields in EA. And we’re sharing a few other updates.
    More detailed summary:
    We’re running a test: “Community” posts have a separate section on the Frontpage ⬇️
    We’re shutting down subforums, and pivoting to “core topics” ⬇️
    Other changes ⬇️
    You can now notify users by tagging or “mentioning” them ⬇️
    You can tag shortform posts that are related to core topics ⬇️
    You can upload social preview images to your posts ⬇️
    Coauthors can edit posts ⬇️
    “Community” posts have a separate section on the Frontpage — a test
    We recently shared that we might test separating community discussion from posts on other topics, and outlined two approaches we might try. Based on the feedback we got and other considerations, we’re going forward with version 2: a section for “Community” posts on the Frontpage.
    What this means
    Posts that are primarily impactful via the EA community as a phenomenon (posts that aren’t significantly relevant to a non-meta organization, field of research, etc., including posts about the community) get the “Community” tag (as before), which moves them off the top of the Frontpage and into a “Community” section below the fold (on the same page).
    Moderators and Forum facilitators will be applying and monitoring this tag. Other users cannot modify the tag.
    Readers can change this by modifying their tag filters (e.g. to still include "Community" posts).
    Discussions on “Community” posts will also not be showing up in the “Recent Discussion” section. (Readers can change this back by going to their account settings and looking in the "Site customizations" section.)
    We will run this test for around a month, and track engagement with different kinds of posts, feedback we get, subjective evaluations of how discussions go, and more.
    If you have any feedback on this change (or any others), we’d love to hear it. You can comment on this post or email [email protected].
    You can see how this works by scrolling through the Frontpage right now.
    Why we're doing this
    You can see our full reasoning here (and in the comments). In brief, we are worried about a way in which the voting structure on the Forum leads to more engagement with “Community” posts than users endorse, we’ve been hearing user feedback on related issues for a long time, and we’ve been having lots of conversations on hypotheses that we’d like to test.
    Transitioning from subforums to “core topics”
    Very short TL;DR: we’re closing the subforums that we were testing, and pivoting to developing more polished and easy-to-use topic pages that help readers explore those topics.
    Slightly less brief:
    We were testing hosting subforums on the Forum. We launched subforums for Effective Giving, Bioethics, Software Development, and Forecasting & Estimation.
    We faced some issues with the subforums and decided to discontinue that project (at least for the near future — we might revisit subforums). (See more on what we learned.) The existing subforums are closing (and most will become “core topic” pages).
    Content from the subforums — discussion threads, posts, etc. — will still be available on the Forum. (Posts on the topic pages, discussions in the "Posts" tab.)
    We still want to build better topic-specific spaces on the Forum, and are developing “core topic” pages that we hope will let people keep up with content on those topics or go deeper into topic-specific content.
    (We also hope to suppor...
    10 min
  • LW - Covid 2/9/22: Interferon λ by Zvi
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Covid 2/9/22: Interferon λ, published by Zvi on February 9, 2023 on LessWrong.
    There are two big pieces of pandemic news this week.
    One is that there is growing alarm about the dangers of avian flu, or H5N1. Likely mammal-to-mammal transmission has been identified, a lot of wild animals are infected, and there is a real danger that this will become a pandemic. I will cover this in its own post.
    The other is that there are trial results for a new Covid treatment, Interferon λ. It looks super effective. Cuts hospitalizations and deaths in half, with no major side effects.
    Despite this, there is zero expectation by anyone that it will be available any time soon, and little surprise about this reaction. You see, the trial was done by academics and was in Brazil and Canada. So it doesn’t count. Sorry.
    Executive Summary
    We have a new drug that cuts Covid risk of hospitalizations and deaths by about half with no major side effects.
    You can’t have it. Maybe ever.
    Because FDA. FDA Delenda Est.
    Let’s run the numbers.
    The Numbers
    Predictions
    Predictions from Last Week: 210k cases (-8%) and 3,150 deaths (-8%)
    Results: 238k cases (-6%) and 3,052 deaths (-11%)
    Predictions for Next Week: 220k cases (-7%) and 2,750 deaths (-10%).
    Last week I forgot to manually adjust Florida for cases, which I’ve fixed. That’s why the percentages above don’t match. Also means my prediction last week for cases was worse than I thought – cases were actually slightly up and I predicted a drop.
    I expect deaths to continue to drop on schedule. Valentine’s Day is not a reporting-relevant holiday, although it is still a date it is important not to forget.
    Deaths
    Cases
    Interferon λ
    As noted up top: It looks super effective. Cuts hospitalizations and deaths in half, with no major side effects. Despite this, there is zero expectation by anyone that it will be available any time soon, and little surprise about this reaction.
    Here is their results summary.
    A total of 933 patients were assigned to receive pegylated interferon lambda (2 were subsequently excluded owing to protocol deviations) and 1018 were assigned to receive placebo. Overall, 83% of the patients had been vaccinated, and during the trial, multiple SARS-CoV-2 variants had emerged. A total of 25 of 931 patients (2.7%) in the interferon group had a primary-outcome event, as compared with 57 of 1018 (5.6%) in the placebo group, a difference of 51% (relative risk, 0.49; 95% Bayesian credible interval, 0.30 to 0.76; posterior probability of superiority to placebo, >99.9%). Results were generally consistent in analyses of secondary outcomes, including time to hospitalization for Covid-19 (hazard ratio, 0.57; 95% Bayesian credible interval, 0.33 to 0.95) and Covid-19–related hospitalization or death (hazard ratio, 0.59; 95% Bayesian credible interval, 0.35 to 0.97). The effects were consistent across dominant variants and independent of vaccination status.
    Among patients with a high viral load at baseline, those who received pegylated interferon lambda had lower viral loads by day 7 than those who received placebo. The incidence of adverse events was similar in the two groups.
    I am completely unsurprised both by our government denying people access to life saving medicine as a matter of course. And also completely unsurprised by the complete indifference of the public to our government indefinitely denying everyone access to life-saving medicine.
    Paxlovid was the last gasp of the old ‘people are dying and we upended our lives to try and stop this, perhaps we should unnecessarily delay less than usual in deploying life saving medicine’ principle.
    This is standard procedure. The FDA exists to deny people life saving medicine. For years. Instead, during that time, they die. No rush. If there isn’t enough money to justify paying to get through the process, t...
    21 min
  • EA - EA Community Builders’ Commitment to Anti-Racism & Anti-Sexism by Rockwell
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: EA Community Builders’ Commitment to Anti-Racism & Anti-Sexism, published by Rockwell on February 9, 2023 on The Effective Altruism Forum.
    In light of recent events in the EA community, several professional EA community builders have been working on a statement for the past few weeks: EA Community Builders’ Commitment to Anti-Racism & Anti-Sexism. You can see the growing list of signatories at the link.
    We have chosen to be a part of the effective altruism community because we agree that the world can and should be a better place for everyone in it. We have chosen to be community builders because we recognize that lasting, impactful change comes out of collective effort. The positive change we want to see in the world requires a diverse set of actors collaborating within an inclusive community for the greater good.
    But inclusive, diverse, collaborative communities need to be protected, not just built. Bigoted ideologies, such as racism and sexism, are intrinsically harmful. They also fundamentally undermine the very collaborations needed to produce a world that is better for everyone in it.
    We unequivocally condemn racism and sexism, including “scientific” justifications for either, and believe they have no place in the effective altruism community. As community builders within the effective altruism space, we commit to practicing and promoting anti-racism and anti-sexism within our communities.
    If you are the leader/organizer of an EA community building group (including national and city groups, professional groups, affinity groups, and university groups), you can add your signature and any additional commentary specific to you/your organization (that will display as a footnote on the statement) by filling out this form.
    Thank you to the many community builders who contributed to the creation of this document.
    Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.
    2 min