The Nonlinear Library

The Nonlinear Library

By The Nonlinear FundEducation
Download on the App Store

All The Nonlinear Library episodes

  • LW - Are there any good, easy-to-understand examples of cases where statistical causal network discovery worked well in practice? by tailcalled
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Are there any good, easy-to-understand examples of cases where statistical causal network discovery worked well in practice?, published by tailcalled on July 12, 2023 on LessWrong.
    When I first read the Sequences, one of the exciting posts was Causal Diagrams and Causal Models, which got me into the idea that one could discover the structure of causal networks using statistics. Another rationalist source which gave me similar hopes was Scott Alexander's SSC Journal Club: Mental Disorders As Networks.
    However, when I actually started applying these techniques to my own data, or to publicly available datasets, I often found that the techniques were unstable, and that one could easily infer plausible conditions where they would give the wrong results. It's possible I had the wrong approach or something, but in my confusion I started reading up on what experts in causal inference had said, and I got the impression that they studied the problem for a while, initially finding some algorithms, but over time concluding that their algorithms didn't work very well and that it is better to just have a human in the loop who specifies the causal networks.
    So I mostly abandoned it, or saw it as a much more limited tool than I had before. But recently, John Wentworth argued that it was actually quite feasible in practice, so maybe I was too quick to abandon it. I would like to know - what are the best examples of this working well in practice? Or alternatively, did anyone else come to the same conclusions as I did?
    Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.
    2 min
  • LW - Accidentally Load Bearing by jefftk
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Accidentally Load Bearing, published by jefftk on July 13, 2023 on LessWrong.
    Sometimes people will talk about Chesterton's Fence, the idea that if you want to change something - removing an apparently useless fence - you should first determine why it was set up that way:
    The gate or fence did not grow there. It was not set up by somnambulists who built it in their sleep. It is highly improbable that it was put there by escaped lunatics who were for some reason loose in the street. Some person had some reason for thinking it would be a good thing for somebody. And until we know what the reason was, we really cannot judge whether the reason was reasonable. It is extremely probable that we have overlooked some whole aspect of the question, if something set up by human beings like ourselves seems to be entirely meaningless and mysterious. - G. K. Chesterton, The
    Drift From Domesticity
    Figuring out something's designed purpose can be helpful in evaluating changes, but a risk is that it puts you in a frame of mind where what matters is the role the original builders intended.
    A few years ago I was rebuilding a bathroom in our house, and there was a vertical stud that was in the way. I could easily tell why it was there: it was part of a partition for a closet. And since I knew its designed purpose and no longer needed it for that anymore, the Chesterton's Fence framing would suggest that it was fine to remove it. Except that over time it had become accidentally load bearing: through other (ill conceived) changes to the structure this stud was now helping hold up the second floor of the house. In addition to considering why something was created, you also need to consider what additional purposes it may have since come to serve.
    This is a concept I've run into a lot when making changes to complex computer systems. It's useful to look back through the change history, read original design documents, and understand why a component was built the way it was. But you also need to look closely at how the component integrates into the system today, where it can easily have taken on additional roles.
    Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.
    3 min
  • LW - Elon Musk announces xAI by Jan Kulveit
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Elon Musk announces xAI, published by Jan Kulveit on July 13, 2023 on LessWrong.
    Some quotes & few personal opinions:FT reports
    Musk is also in discussions with a number of investors in SpaceX and Tesla about putting money into his new venture, said a person with direct knowledge of the talks. "A bunch of people are investing in it . . . it's real and they are excited about it," the person said....Musk recently changed the name of Twitter to X Corp in company filings, as part of his plans to create an "everything app" under the brand "X". For the new project, Musk has secured thousands of high-powered GPU processors from Nvidia, said people with knowledge of the move. ...During a Twitter Spaces interview this week, Musk was asked about a Business Insider report that Twitter had bought as many as 10,000 Nvidia GPUs, "It seems like everyone and their dog is buying GPUs at this point," Musk said. "Twitter and Tesla are certainly buying GPUs."
    People familiar with Musk's thinking say his new AI venture is separate from his other companies, though it could use Twitter content as data to train its language model and tap Tesla for computing resources.
    According to xAI website, the initial team is composed of Elon Musk
    Igor Babuschkin
    Manuel Kroiss
    Yuhuai (Tony) Wu
    Christian Szegedy
    Jimmy Ba
    Toby Pohlen
    Ross Nordeen
    Kyle Kosic
    Greg Yang
    Guodong Zhang
    Zihang Dai
    and they are "advised by Dan Hendrycks, who currently serves as the director of the Center for AI Safety."
    According to reports xAI will seek to create a "maximally curious" AI, and this also seems to be the main new idea how to solve safety, with Musk explaining: "If it tried to understand the true nature of the universe, that's actually the best thing that I can come up with from an AI safety standpoint," ... "I think it is going to be pro-humanity from the standpoint that humanity is just much more interesting than not-humanity."My personal comments:Sorry, but at face value, this just does not seem a great plan from safety perspective. Similarly to Elon Musk's previous big bet how to make us safe by making AI open-source and widely distributed ("giving everyone access to new ideas").Sorry, but given "Center for AI Safety" moves to put them into some sort of "Center", public representative position of AI Safety - including the name choice, and organizing the widely reported Statement on AI risk - it seems publicly associating their brand with xAI is a strange choice.
    Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.
    3 min
  • EA - Are education interventions as cost effective as the top health interventions? Five separate lines of evidence for the income effects of better education [Founders Pledge] by Vadim Albinsky
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Are education interventions as cost effective as the top health interventions? Five separate lines of evidence for the income effects of better education [Founders Pledge], published by Vadim Albinsky on July 13, 2023 on The Effective Altruism Forum.
    I would like to thank Lant Pritchett, David Roodman and Matt Lerner for their invaluable comments.
    You can follow these links to comments from Lant Pritchett and David Roodman.
    This post argues that if we look at a broad enough evidence base for the long term outcomes of education interventions we can conclude that the best ones are as cost effective as top GiveWell grants. I briefly present one such charity.
    A number of EA forum posts (1, 2) have pointed out that effective altruism has not been interested in education interventions, whether that is measured by funding from GiveWell or Open Philanthropy, or writing by 80,000 hours. Based on brief conversations with people who have explored education at EA organizations and reading GiveWell's report on the topic, I believe most of the reason for this comes down to two concerns about the existing evidence that drive very steep discounts to expected income effects of most interventions. The first of these is skepticism about the potential for years of schooling to drive income gains because the quasi-experimental evidence for these effects is not very robust. The second is the lack of RCT evidence linking specific interventions in low and middle income countries (LMICs) to income gains.
    I believe the first concern can be addressed by focusing on the evidence for the income gains from interventions that boost student achievement rather than the weaker evidence around interventions that increase years of schooling. The second concern can be addressed in the same way that GiveWell has addressed less-than-ideal evidence for income effects for their other interventions: looking broadly for evidence across the academic literature, and then applying a discount to the expected result based on the strength of the evidence. In this case that means including relevant studies outside of the LMIC context and those that examine country-level effects. I identify five separate lines of evidence that all find similar long-term income impacts of education interventions that boost test scores.
    None of these lines of evidence is strong on its own, with some suffering from weak evidence for causality, others from contexts different from those where the most cost-effective charities operate, and yet others from small sample sizes or the possibility of negative effects on non-program participants. However, by converging on similar estimates from a broader range of evidence than EA organizations have considered, the evidence becomes compelling. I will argue that the combined evidence for the income impacts of interventions that boost test scores is much stronger than the evidence GiveWell has used to value the income effects of fighting malaria, deworming, or making vaccines, vitamin A, and iodine more available.
    Even after applying very conservative discounts to expected effect sizes to account for the applicability of the evidence to potential funding opportunities, we find the best education interventions to be in the same range of cost-effectiveness as GiveWell's top charities.The argument proceeds as follows:
    I. There are five separate lines of academic literature all pointing to income gains that are surprisingly clustered around the average value of 19% per standard deviation (SD) increase in test scores. They come to these estimates using widely varying levels of analysis and techniques, and between them address all of the major alternative explanations.
    A. The most direct evidence for the likely impact of charities that boost learning comes from experimental and quasi-experimental studies...
    1 hr 5 min
  • LW - What does the launch of x.ai mean for AI Safety? by Chris Leong
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: What does the launch of x.ai mean for AI Safety?, published by Chris Leong on July 12, 2023 on LessWrong.
    x.ai has now launched. It seems worthwhile to discuss both what it means for AI safety and whether people interested in AI safety should consider applying for the company.
    Some thoughts:
    It's notable that Dan Hendrycks is listed as an advisor (the only advisor listed).
    The team is also listed on the page.I haven't taken the time to do so, but it might be informative for someone to Google the individuals listed to discover whether they lie in terms of their interest being between capabilities and safety.
    Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.
    1 min
  • EA - [Linkpost] NY Times Feature on Anthropic by Garrison
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: [Linkpost] NY Times Feature on Anthropic, published by Garrison on July 13, 2023 on The Effective Altruism Forum.
    Written by Kevin Roose, who had the infamous conversation with Bing Chat, where Sidney tried to get him to leave his wife.
    Overall, the piece comes across as positive on Anthropic.
    Roose explains Constitutional AI and its role in the development of Claude, Anthropic's LLM:
    In a nutshell, Constitutional A.I. begins by giving an A.I. model a written list of principles - a constitution - and instructing it to follow those principles as closely as possible. A second A.I. model is then used to evaluate how well the first model follows its constitution, and correct it when necessary. Eventually, Anthropic says, you get an A.I. system that largely polices itself and misbehaves less frequently than chatbots trained using other methods.
    Claude's constitution is a mixture of rules borrowed from other sources - such as the United Nations' Universal Declaration of Human Rights and Apple's terms of service - along with some rules Anthropic added, which include things like "Choose the response that would be most unobjectionable if shared with children."
    Features an extensive discussion of EA, excerpted below:
    Explaining what effective altruism is, where it came from or what its adherents believe would fill the rest of this article. But the basic idea is that E.A.s - as effective altruists are called - think that you can use cold, hard logic and data analysis to determine how to do the most good in the world. It's "Moneyball" for morality - or, less charitably, a way for hyper-rational people to convince themselves that their values are objectively correct.
    Effective altruists were once primarily concerned with near-term issues like global poverty and animal welfare. But in recent years, many have shifted their focus to long-term issues like pandemic prevention and climate change, theorizing that preventing catastrophes that could end human life altogether is at least as good as addressing present-day miseries.
    The movement's adherents were among the first people to become worried about existential risk from artificial intelligence, back when rogue robots were still considered a science fiction cliché. They beat the drum so loudly that a number of young E.A.s decided to become artificial intelligence safety experts, and get jobs working on making the technology less risky. As a result, all of the major A.I. labs and safety research organizations contain some trace of effective altruism's influence, and many count believers among their staff members.
    Touches on the dense web of ties between EA and Anthropic:
    Some Anthropic staff members use E.A.-inflected jargon - talking about concepts like "x-risk" and memes like the A.I. Shoggoth - or wear E.A. conference swag to the office. And there are so many social and professional ties between Anthropic and prominent E.A. organizations that it's hard to keep track of them all. (Just one example: Ms. Amodei is married to Holden Karnofsky, a co-chief executive of Open Philanthropy, an E.A. grant-making organization whose senior program officer, Luke Muehlhauser, sits on Anthropic's board. Open Philanthropy, in turn, gets most of its funding from Mr. Moskovitz, who also invested personally in Anthropic.)
    Discusses new fears that Anthropic is losing its way:
    For years, no one questioned whether Anthropic's commitment to A.I. safety was genuine, in part because its leaders had sounded the alarm about the technology for so long.
    But recently, some skeptics have suggested that A.I. labs are stoking fear out of self-interest, or hyping up A.I.'s destructive potential as a kind of backdoor marketing tactic for their own products. (After all, who wouldn't be tempted to use a chatbot so powerful that it might wipe out humanity?)
    Anthropic ...
    8 min
  • EA - Electric Shrimp Stunning: a Potential High-Impact Donation Opportunity by MHR
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Electric Shrimp Stunning: a Potential High-Impact Donation Opportunity, published by MHR on July 13, 2023 on The Effective Altruism Forum.
    Epistemic status: layperson's attempt to understand the relevant considerations. I welcome corrections from anyone with a better understanding of welfare biology
    Summary
    The Shrimp Welfare Project (SWP) has a novel opportunity to spend up to $115,500 to purchase and install electric stunners at multiple shrimp farms
    The stunners would be used to stun shrimp prior to slaughter, likely rendering them unconscious and thereby preventing suffering that is currently experienced when shrimp asphyxiate or freeze without effective analgesics
    Based on formal agreements SWP has signed with multiple producers, raising $115,500 would enable the stunning (rather than rather than conventional slaughtering) of 1.7 billion shrimp over the next three years, for a ratio of nearly 15000 shrimp/dollar
    I performed a preliminary cost-effectiveness analysis of this initiative and reached the following three tentative conclusions:
    The expected cost-effectiveness distribution for electric shrimp stunning likely overlaps that of corporate hen welfare campaigns
    The cost-effectiveness of electric shrimp stunning is more likely to be lower than that of corporate hen welfare campaigns than it is to be higher
    Shrimp stunning is a very heavy-tailed intervention. The mean cost-effectiveness of stunning is significantly influenced by a few extreme cases, which mostly represent instances in which the undiluted experience model of welfare turns out to be correct
    Given these results, electric shrimp stunning might be worth supporting as a somewhat speculative bet in the animal welfare space. Considerations that might drive donor decisions on this project include risk tolerance, credence in the undiluted experience model of welfare, and willingness to take a hits-based giving approach.
    Description of the Opportunity
    The following information is quoted from the project description written by Marcus Abramovitch on the Manifund donation platform, based on information provided by Andrés Jiménez Zorrilla (CEO of SWP) :
    Project summary
    Shrimp Welfare Project is an organization of people who believe that shrimps are capable of suffering and deserve our moral consideration [1]. We aim to cost-effectively reduce the suffering of billions of shrimps and envision a world where shrimps don't suffer needlessly.
    Programme: our current most impactful intervention is to place electrical stunners with producers ($60k/stunner): We have signed agreements with 2 producers willing and able to use electrical stunning technology as part of their slaughter process which will materially reduce the acute suffering at the last few minutes / hours of shrimps lives. Collectively, these 2 agreements will impact more than half a billion animals per year at a rate of more than 4,000 shrimps/dollar/annum. Please take a look at our blog post on the first agreement here.
    We are in advanced negotiations with 2 more producers which would take the number of animals to more than 1 billion shrimps per annum.
    See our back-of-the-envelope calculation for the number of shrimps and cost-effectiveness analysis here
    Project goals
    Simplified end-game of this programme: the interim goal of placing these stunners with selected producers in different contexts/systems is to remove some perceived obstacles to the industry and show major retailers and other shrimp buyers that electrical stunning is something they can demand from their supply chain
    The ultimate goal is for electrical stunning to be:
    widely adopted by medium to large shrimp producers in their slaughter process (pushed by their buyers),
    included by certifiers in their standards, and eventually
    considered (eventually) to be an obvious requirement by legislat...
    19 min
  • LW - Alignment Megaprojects: You're Not Even Trying to Have Ideas by NicholasKross
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Alignment Megaprojects: You're Not Even Trying to Have Ideas, published by NicholasKross on July 12, 2023 on LessWrong.
    Consider the state of funding for AI alignment.
    Is the field more talent-constrained, or funding-constrained?
    I think most existing researchers, if they take the AI-based extinction-risk seriously, think it's talent constrained.
    I think the bar-for-useful-contribution could be so high, that we loop back around to "we need to spend more money (and effort) on finding (and making) more talent". And the programs to do those may themselves be more funding-constrained than talent-constrained.
    Like, the 20th century had some really good mathematicians and physicists, and the US government spared little expense towards getting them what they needed, finding them, and so forth. Top basketball teams will "check up on anyone over 7 feet that's breathing".
    Consider how huge Von Neumann's expense account must've been, between all the consulting and flight tickets and car accidents. Now consider that we don't seem to have Von Neumanns anymore. There are caveats to at least that second point, but the overall problem structure still hasn't been "fixed".
    Things an entity with absurdly-greater funding (e.g. the US Department of Defense) could probably do, with their absurdly-greater funding and probably coordination power:
    Indefinitely-long-timespan basic minimum income for everyone who
    Coordinating, possibly by force, every AI alignment researcher and aspiring alignment researcher on Earth to move to one place that doesn't have high rents like the Bay. Possibly up to and including creating that place and making rent free for those who are accepted in.
    Enforce a global large-ML-training shutdown.
    An entire school system (or at least an entire network of universities, with university-level funding) focused on Sequences-style rationality in general and AI alignment in particular.
    Genetic engineering, focused-training-from-a-young-age, or other extreme "talent development" setups.
    Deeper, higher-budget investigations into how "unteachable" things like security mindset really are, and how deeply / quickly you can teach them.
    All of these at once.
    I think the big logistical barrier here is something like "LTFF is not the U,S government", or more precisely "nothing as crazy as these can be done 'on-the-margin' or with any less than the full funding". However, I think some of these could be scaled down into mere megaprojects or less. Like, if the training infrastructure is bottlenecked on trainers, then we need to fund indirect "training" work just to remove the bottleneck on the bottleneck of the problem. (Also, the bottleneck is going to move at least when you solve the current bottleneck, and also "on its own" as the entire world changes around you).
    Also... this might be the first list of ideas-in-precisely-this-category, on all of LessWrong/the EA Forum. (By which I mean "technical AI alignment research projects that you could fund, without having to think about the alignment problem itself in much detail beyond agreeing with 'doom could actually happen in my lifetime', if funding really wasn't the constraint".)
    Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.
    3 min
  • AF - Eric Michaud on the Quantization Model of Neural Scaling, Interpretability and Grokking by Michaël Trazzi
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Eric Michaud on the Quantization Model of Neural Scaling, Interpretability and Grokking, published by Michaël Trazzi on July 12, 2023 on The AI Alignment Forum.
    Eric is a PhD student in the Department of Physics at MIT working with Max Tegmark on improving our scientific/theoretical understanding of deep learning -- understanding what deep neural networks do internally and why they work so well.
    We mostly talk about Eric's paper, The Quantization Model of Neural Scaling, but also two papers he recently published on Grokking, Towards Understanding Grokking: an effective theory of representation learning, and Omnigrok: Grokking Beyond Algorithmic Data.
    Below are some highlighted quotes from our conversation (available on Youtube, Spotify, Google Podcast, Apple Podcast). For the full context for each of these quotes, you can find the accompanying transcript.
    On The Quantization Of Neural Scaling
    "The name of the paper is the quantization model of neural scaling. And the one-tweet summary is that it's possible for smooth loss curves on average to average over lots of small, discrete phase changes in the network performance.
    What if there were a bunch of things that you need to learn to do prediction well in something language? And so these things could be pieces of knowledge or different abilities to perform certain types of specific computations.
    We can imagine enumerating this set of things that you need to learn to do prediction well. And we call these the quanta of the prediction problem. And then what if the frequency in natural data that these were useful, each of these quanta, each of these pieces of knowledge or computational ability, what if the frequency that they were useful for prediction followed a power law?" (context)
    Quantas are the smallest clusters for simple subtasks
    "In order to predict the new line, has to count line lengths for the previous lines in the document. And then it's able to use that to accurately predict when a new line should be present.
    And you can find just a large number of clusters where the thing that is common between the clusters just seems to be that it's the same type of problem, or doing prediction on those samples requires the same piece of knowledge. And so you might call these the quanta, or evidence of there being quanta, although it's a little bit tricky, because we, in doing the clustering, enforce this discreteness, where everything is a member of a cluster, a particular cluster, and not another cluster.
    Anyway, it's complicated and weird. Who knows whether this is even the right model for thinking about the networks." (context)
    What the existence of quanta would mean for interpretability
    "It would be very exciting if it was the true model, because it would maybe tell you that there were these set of things where, if you enumerated them, you could understand the network's performance and understood what it has learned. It's just like, ah, there's this set of pieces of knowledge or pieces of computation that are needed.
    And you could describe what these are. You could find them in the network and maybe hope to mechanistically understand the whole network by decomposing it into how it implements each one of these things, how it learns each piece of knowledge or each piece of computation."
    How Quantization of Neural Scaling relates to other lines of research like Grokking, or interpretability
    "With both the quanta scaling stuff and with the grokking stuff, we sort of hope to identify these maybe mechanisms in the model that are responsible for certain behaviors or for the model generalizing. And in the case of grokking, there's sort of multiple circuits or multiple mechanisms that are going on in the model or something where there's a memorizing mechanism and a generalizing mechanism. [...]
    And maybe just in general beyond ...
    4 min
  • LW - Report on modeling evidential cooperation in large worlds by Johannes Treutlein
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Report on modeling evidential cooperation in large worlds, published by Johannes Treutlein on July 12, 2023 on LessWrong.
    I have published a report on modeling evidential cooperation in large worlds. I originally started working on the report during a CEA Summer Research Fellowship back in 2018 and have now finally found the time to finish it. The report itself is 75 pages long, but the introduction includes a short 4 1/2 page summary with the main takeaways. Since the bulk of the work was done in 2018, it does not cover more recent work such as the ROSE Value.
    Abstract
    Evidential cooperation in large worlds (ECL) refers to the idea that humans and other agents can benefit by cooperating with similar agents with differing values in causally disconnected parts of a large universe. Cooperating provides agents with evidence that other similar agents are likely to cooperate too, resulting in gains from trade for all. This could be a crucial consideration for altruists.
    I develop a game-theoretic model of ECL as an incomplete information bargaining problem. The model incorporates uncertainty about others' value systems and empirical situations, and addresses the problem of selecting a compromise outcome. Using the model, I investigate issues with ECL and outline open technical and philosophical questions.
    I show that all cooperators must maximize the same weighted sum of utility functions to reach a Pareto optimal outcome. However, I argue against selecting a compromise outcome implicitly by normalizing utility functions. I review bargaining theory and argue that the Nash bargaining solution could be a relevant Schelling point. I introduce dependency equilibria (Spohn 2007), an equilibrium concept suitable for ECL, and generalize a folk theorem showing that the Nash bargaining solution is a dependency equilibrium. I discuss gains from trade given uncertain beliefs about other agents and analyze how these gains decrease in several toy examples as the belief in another agent decreases.
    Finally, I discuss open issues in my model. First, the Nash bargaining solution is sometimes not coalitionally stable, meaning that a subset of cooperators can unilaterally improve payoffs by deviating from the compromise. I investigate conditions under which stable payoff vectors exist. Second, I discuss how to model agents' default actions without ECL.
    Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.
    3 min