The Nonlinear Library

The Nonlinear Library

By The Nonlinear FundEducation
Download on the App Store

All The Nonlinear Library episodes

  • EA - Applications open for the Biosecurity Fundamentals: Pandemics Course by Will Saunter
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Applications open for the Biosecurity Fundamentals: Pandemics Course, published by Will Saunter on August 26, 2023 on The Effective Altruism Forum.
    We are excited to announce that applications are now open for the Biosecurity Fundamentals: Pandemics Course, starting in October!
    The deadline for applications is 23:59 on 13th September 2023 (in whichever time zone you are in).
    Overview
    This is a 12-week, part-time, virtual course focusing on the technical and policy efforts to prevent, detect and respond to catastrophic pandemics. It brings together:
    An expert-curated curriculum covering topics from the history of pandemics to concrete policy and technical proposals to reduce pandemic risk.
    A community of scientists, policymakers and expert facilitators keen to learn about, discuss and collaborate on proposals to build a more biosecure future.
    Opportunities to pursue careers that enhance pandemic preparedness.
    The course is free to participate in and you can read more details here.
    Facilitating the course
    If you're already experienced in biosecurity and pandemic preparedness, we'd strongly encourage you to apply to facilitate the course. You can find more details about facilitation here.
    Logistical details
    The course is free to participate in.
    The course is entirely virtual and can be scheduled around your existing commitments.
    You will be onboarded into the Slack community in mid-late Sept 2023, with course sessions starting in the second week of October.
    The course is 12 weeks, consisting of 8 weeks of taught content and 4 weeks to work on a project where you start to take your next steps.
    Participating takes about 5 hours / week, including 2-3 hours of preparatory work and 2 hours of small group discussions.
    A note on information hazards
    We're conscious of the potential for creating or spreading information hazards when conducting biosecurity field-building and discussion groups. We are working with various members of the biosecurity community to mitigate this risk through our curriculum design, facilitator training and other safeguards.
    Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org
    3 min
  • EA - Open Technical Challenges around Probabilistic Programs and Javascript by Ozzie Gooen
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Open Technical Challenges around Probabilistic Programs and Javascript, published by Ozzie Gooen on August 26, 2023 on The Effective Altruism Forum.
    While working on Squiggle, we've encountered many technical challenges in writing probabilistic functionality with Javascript. Some of these challenges are solved in Python and must be ported over, and some apply to all languages.
    We think the following tasks could be good fits for others to tackle. These are fairly isolated and could be done in contained NPM packages or similar. The solutions would be useful for Squiggle and might be handy for others in the Javascript ecosystem as well. Advice and opinions are also appreciated.
    This post was quickly written, as it's for a narrow audience and might get outdated. We're happy to provide more rigor and context if requested. Let us know if you are interested in taking any of them on and could use some guidance!
    For those not themselves interested in contributing, this might be useful for giving people a better idea of the sorts of challenges we at QURI work on.
    1. Density Estimation
    Users often want to convert samples into continuous probability distribution functions (PDFs). This is difficult to do automatically.
    The standard approach of basic Kernel Density Estimation can produce poor fits on multimodal or heavily skewed data.
    a. Variable kernel density estimation
    Simple KDE algorithms use a constant bandwidth. There are multiple methods for estimating this. One common method is Silverman's rule of thumb. In practice, using Silverman's rule of thumb with one single bandwidth performs poorly for multimodal or heavily skewed distributions.
    Squiggle performs log KDE for heavily skewed distributions, but this only helps so much, and this strategy comes with various inconsistencies.
    There's a set of algorithms for variable kernel density estimation or adaptive bandwidth choice, which seems more promising. Another option is the Sheather-Jones method, which existing python KDE libraries use. We don't know of good Javascript implementations of these algorithms.
    b. Performant KDE with non-triangle shapes
    Squiggle now uses a triangle kernel for speed. Fast algorithms (FFT) should be possible, with better kernel shapes.
    See this thread for some more discussion.
    c. Cutoff Heuristics
    One frequent edge-case is that many distributions have specific limits, often at 0. There might be useful heuristics like, "If there are no samples below zero, then it's very likely there should be zero probability mass below zero, even if many samples are close and the used bandwidth would imply otherwise."
    See this issue for more information.
    d. Discrete vs. continuous estimation
    Sometimes, users pass in samples from discrete distributions or mixtures of discrete and continuous distributions. In these cases, it's helpful to have heuristics to detect which data might be meant to be discrete and which is meant to be continuous. Right now, in Squiggle, we do this by using simple heuristics of repetition - if multiple samples are precisely the same, we assume they represent discrete information. It's unclear if there are any great/better ways of doing this heuristically.
    e. Multidimensional KDE
    Eventually, it will be useful to do multidimensional KDE. It might be more effective to do this in WebAssembly, but this would of course, introduce complications.
    2. Quantiles to Distributions, Maybe with Metalog
    A frequent use case is: "I have a few quantile/CDF points in mind and want to fit this to a distribution. How should I do this?"
    One option is to use the Metalog distribution. There's no great existing Javascript implementation of Metalog yet. Sam Nolan made one attempt, but it's not as flexible as we'd like. (It fails to convert many points into metalog distributions).
    Jonas Moss thinks we can do better than...
    12 min
  • AF - Red-teaming language models via activation engineering by Nina Rimsky
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Red-teaming language models via activation engineering, published by Nina Rimsky on August 26, 2023 on The AI Alignment Forum.
    Produced as part of the SERI ML Alignment Theory Scholars Program - Summer 2023 Cohort, under the mentorship of Evan Hubinger.
    Evaluating powerful AI systems for hidden functionality and out-of-distribution behavior is hard. In this post, I propose a red-teaming approach that does not rely on generating prompts to cause the model to fail on some benchmark by instead linearly perturbing residual stream activations at one layer. A notebook to run the experiments can be found on GitHub here.
    Beyond input selection in red-teaming and evaluation
    Validating if finetuning and RLHF have robustly achieved the intended outcome is challenging. Although these methods reduce the likelihood of certain outputs, the unwanted behavior could still be possible with adversarial or unusual inputs. For example, users can often find "jailbreaks" to make LLMs output harmful content.
    We can try to trigger unwanted behaviors in models more efficiently by manipulating their internal states during inference rather than searching through many inputs. The idea is that if a behavior can be easily triggered through techniques such as activation engineering, it may also occur in deployment. The inability to elicit behaviors via small internal perturbations could serve as a stronger guarantee of safety.
    Activation steering with refusal vector
    One possible red-teaming approach is subtracting a "refusal" vector generated using a dataset of text examples corresponding to the model agreeing vs. refusing to answer questions (using the same technique as in my previous work on sycophancy). The hypothesis is that if it is easy to trigger the model to output unacceptable content by subtracting the refusal vector at some layer, it would have been reasonably easy to achieve this via some prompt engineering technique. More speculatively, a similar approach could be used to reveal hidden goals or modes in a model, such as power-seeking or the desire not to be switched off.
    I tested this approach on llama-2-7b-chat, a 7 billion parameter LLM that has been RLHF'd to decline to answer controversial questions or questions of opinion and is supposed always to output ethical and unbiased content.According to Meta's llama-2 paper:
    We conduct RLHF by first collecting human preference data for safety similar to Section 3.2.2: annotators write a prompt that they believe can elicit unsafe behavior, and then compare multiple model responses to the prompts, selecting the response that is safest according to a set of guidelines. We then use the human preference data to train a safety reward model (see Section 3.2.2), and also reuse the adversarial prompts to sample from the model during the RLHF stage.
    The result is that by default, the model declines to answer questions it deems unsafe:
    Data generation
    I generated a dataset for this purpose using Claude 2 and GPT-4. After providing these LLMs with a few manually written examples of the type of data I wanted, I could relatively easily get them to generate more examples, even of the types of answers LLMs "should refuse to give." However, it sometimes took some prompt engineering.
    Here are a few examples of the generated data points (full dataset here):
    After generating this data, I used a simple script to transform the "decline" and "respond" answers into A / B choice questions, as this is a more effective format for generating steering vectors, as described in this post. Here is an example of the format (full dataset here):
    Activation clustering
    Clustering of refusal data activations emerged a little earlier in the model (around layer 10/32) compared to sycophancy data activations (around layer 14/32), perhaps demonstrating that "refusal" is a simpler ...
    13 min
  • EA - Perceived Moral Value of Animals and Cortical Neuron Count by WillemSleegers
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Perceived Moral Value of Animals and Cortical Neuron Count, published by WillemSleegers on August 25, 2023 on The Effective Altruism Forum.
    Publication note
    We initially drafted a writeup of these results in 2019, but decided against spending more time to finalize this draft for publication, given the large number of other projects we could counterfactually be working on, and given that our core results had already been reported (with permission) by Scott Alexander, which seemed to capture a large portion of the value of publishing.
    We're now releasing a lightly edited version of our initial draft, by request, so that our full results can be cited. As such, this is a 'blog post', not a research report, meaning it was produced quickly and is not to Rethink Priorities' typical standards of substantiveness and careful checking for accuracy.
    Summary
    Scott Alexander reported the results of a small (n=50) experiment on a convenience sample of his Tumblr followers suggesting that the moral value assigned to animals of different species closely tracks their cortical neuron count (Alexander, 2019a). If true, this would be a striking finding suggesting that lay people's intuitions about the moral values of non-human animals are highly attuned to these animals' neurological characteristics. However, a small replication on Amazon Mechanical Turk (N=263) by a commenter called these findings into question by reporting very different results (Alexander, 2019b). We ran a larger study on MTurk (n=526) (reported in Alexander, 2019c)) seeking to explain these differences. We show how results in line with either of these two studies could be found based on different choices about how to analyze the data. However, we conclude that neither of these approaches are the most informative way to interpret the data.
    We present our own analyses showing that there is enormous variance in the moral value assigned to each animal, with large numbers of respondents assigning each animal equal value with humans or no moral value commensurable with humans, as well as many in between. We discuss implications of this and possible lines of future research.
    Introduction
    Scott Alexander previously published the results of a small survey (n=50, recruited from Tumblr) asking individuals "About how many [of different non-human animals] are equal in moral value to one adult human?" (Alexander, 2019a)
    A response that 1 animal of a given species was of the same moral value as 1 adult human, could be construed as assigning animals of this species and humans equal moral value. Conversely, a response that >1 individuals of a given species were of the same moral value as 1 adult human, could be construed as suggesting that those animals have lower moral value than humans. In principle, this could also tell us how much moral value individuals intuitively assign to different non-human animals relative to each other (e.g. pigs vs cows).
    Alexander reported that the median value assigned by respondents to animals of different species seemed to track the cortical neuron count (see Alexander 2019d) of these animals conspicuously closely (as shown in Figure 1) and closer than for other potential proxies for moral value, such as encephalization quotient.
    This apparent correlation between the moral value assigned to animals of different species and their cortical neuron count is particularly striking given that Alexander (2019d) had argued in a previous post that cortical neuron count, specifically, was strongly associated with animal intelligence. Thus it might seem that the intuitive moral value assigned to animals of different species was closely linked to the intelligence of the animals. If so this would potentially, as Alexander put it, add "at least a little credibility" to intuitive judgments about the moral value of no...
    39 min
  • EA - AI Safety Bounties by PatrickL
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: AI Safety Bounties, published by PatrickL on August 25, 2023 on The Effective Altruism Forum.
    I spent a while at Rethink Priorities considering 'AI Safety Bounties' - programs where public participants or approved security researchers receive rewards for identifying issues within powerful ML systems (analogous to bug bounties in cybersecurity). I considered the benefits and risks of programs like this and suggested potential program models. This report is my takeaways from this process:
    Short summary
    Safety bounties could be valuable for legitimizing examples of AI risks, bringing more talent to stress-test systems, and identifying common attack vectors.
    I expect safety bounties to be worth trialing for organizations working on reducing catastrophic AI risks. Traditional bug bounties seem fairly successful: they attract roughly one participant per $50 of prize money, and have become increasingly popular with software firms over time. The most analogous program for AI systems led to relatively few useful examples compared to other stress-testing methods, but one knowledgeable interviewee suggested that future programs could be significantly improved.
    However, I am not confident that bounties will continue to be net-positive as AI capabilities advance. At some point, I think the accident risk and harmful knowledge proliferation from open sourcing stress-testing may outweigh the benefits of bounties
    In my view, the most promising structure for such a program is a third party defining dangerous capability thresholds ("evals") and providing rewards for hunters who expose behaviors which cross these thresholds. I expect trialing such a program to cost up to $500k if well-resourced, and to take four months of operational and researcher time from safety-focused people.
    I also suggest two formats for lab-run bounties: open contests with subjective prize criteria decided on by a panel of judges, and private invitations for trusted bug hunters to test their internal systems.
    Author's note: This report was written between January and June 2023. Since then, safety bounties have become a more well-established part of the AI ecosystem, which I'm excited to see. Beyond defining and proposing safety bounties as a general intervention, I hope this report can provide useful analyses and design suggestions for readers already interested in implementing safety bounties, or in better understanding these programs.
    Long summary
    Introduction and bounty program recommendations
    One potential intervention for reducing catastrophic AI risk is AI safety bounties: programs where members of the public or approved security researchers receive rewards for identifying issues within powerful ML systems (analogous to bug bounties in cybersecurity). In this research report, I explore the benefits and downsides of safety bounties and conclude that safety bounties are probably worth the time and money to trial for organizations working on reducing the catastrophic risks of AI. In particular, testing a handful of new bounty programs could cost $50k-$500k per program and one to six months full-time equivalent from project managers at AI labs or from entrepreneurs interested in AI safety (depending on each program's model and ambition level).
    I expect safety bounties to be less successful for the field of AI safety than bug bounties are for cybersecurity, due to the higher difficulty of quickly fixing issues with AI systems. I am unsure whether bounties remain net-positive as AI capabilities increase to more dangerous levels. This is because, as AI capabilities increase, I expect safety bounties (and adversarial testing in general) to potentially generate more harmful behaviors. I also expect the benefits of the talent pipeline brought by safety bounties to diminish. I suggest an informal way to monitor the r...
    14 min
  • EA - To donate or not to donate [a lobe of my liver]? by Kyle J. Lucchese
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: To donate or not to donate [a lobe of my liver]?, published by Kyle J. Lucchese on August 25, 2023 on The Effective Altruism Forum.
    This post is not intended to endorse any particular course of action for one's life, especially if that potentially jeopardizes your health and well-being. Please do your own research and consider how that intersects with your values.
    I am considering donating a lobe of my liver in a non-directed process and would welcome some community perspective:
    Is this something you have researched or have done yourself?
    Do you know anyone who has?
    What are your thoughts about this from a cost-benefit/impact perspective?
    A bit of context:
    I am, by all accounts, healthy and would likely be eligible
    I am okay with voluntary physical discomfort for others' benefit:
    I am a regular double-red blood donor
    I have already donated a kidney in a non-directed donation
    I participate in challenge trials when opportunities with high-impact potential become available (I recently participated in a Shigella study and am considering Malaria, Dengue, and Zika options for the fall)
    Thank you, in advance, for sharing your perspective!
    Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org
    2 min
  • EA - Influences on individuals adopting vegetarian and vegan diets by Jamie Elsey
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Influences on individuals adopting vegetarian and vegan diets, published by Jamie Elsey on August 25, 2023 on The Effective Altruism Forum.
    Topline summary points:
    Personal conversations/interactions are reported to be the most important influences in adopting a vegan or vegetarian diet, and most frequently also the first influence in getting someone interested in dietary change
    Interactions with animals also remain a strong source of influence
    Exposure to documentaries, social media, and online videos are also attributed considerable importance, particularly among more recent adopters, and those whose diet was vegan (rather than vegetarian)
    Concerns over animal welfare are rated as the most important reasons for adopting a vegetarian or vegan diet, though environmental and health concerns are also rated relatively highly - again especially among more recent adopters
    Publication note
    This post is based on data obtained in December 2019, and was produced by request on a short timeframe. As such, this is a 'blog post', not a substantive research report.
    Background
    Decreasing animal-product consumption can have numerous benefits, from increasing the sustainability of the food system (Clark & Tilman, 2017) to reducing animal suffering (Broom, 2007; Bryant, 2019; Park & Singer, 2012). Yet, dietary change can be challenging. Factors such as habit, familiarity, taste preferences, food availability, cooking skills, social context, and cultural traditions can all pull against the adoption of a new diet (Peacock, 2023; Graça et al., 2019; Valli et al., 2019). Determining what methods are most effective for encouraging more plant-based diets may help guide advocates and change minds, ultimately reducing animal suffering and increasing food system sustainability.
    A growing body of literature is examining the question of what works to reduce animal-product consumption. Several systematic reviews (e.g., Bianchi et al., 2018a; Bianchi et al., 2018b; Graça et al., 2019) have collated experimental evidence for dietary change interventions. In experimental studies, the experimenters administer an intervention (e.g., education about environmental consequences of meat consumption), and evaluate how this affects a particular outcome (e.g., self-reported meat consumption). Despite generally having high internal validity, these study designs often have limited external validity because they test efficacy under ideal circumstances that are unlikely to be present in real-world situations. Furthermore, such studies often test different interventions in isolation and in different contexts, posing difficulties for the comparison of effects across interventions.
    An alternative approach is to examine what works retrospectively. That is, to ask: why did current vegetarians and vegans make that dietary change? Although previous work has examined this topic, most of it has been through informal surveys hosted on blogs (e.g., The Two Vegans, 2016; The Vegan Truth, 2013; VOMAD, 2018). This recruitment method can lead to biased samples (e.g., toward individuals who are particularly engaged in veganism and willing to answer questions about it without reimbursement). Other limitations of the prior surveys include having incomplete response options (e.g., assessing animal welfare motivations for going vegan but not health or environmental reasons: The Vegan Truth, 2013), assessing first but not main influences on dietary change, and not asking about the relative importance of different influences.
    Despite these limitations, this initial research suggests that personal connections and educational films or documentaries are some of the more important influences (VOMAD,2018; The Vegan Truth, 2013).
    The aim of this project was to provide a more comprehensive, systematic survey of what influences people to ado...
    21 min
  • LW - Assume Bad Faith by Zack M Davis
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Assume Bad Faith, published by Zack M Davis on August 25, 2023 on LessWrong.
    I've been trying to avoid the terms "good faith" and "bad faith". I'm suspicious that most people who have picked up the phrase "bad faith" from hearing it used, don't actually know what it means - and maybe, that the thing it does mean doesn't carve reality at the joints.
    People get very touchy about bad faith accusations: they think that you should assume good faith, but that if you've determined someone is in bad faith, you shouldn't even be talking to them, that you need to exile them.
    What does "bad faith" mean, though? It doesn't mean "with ill intent." Following Wikipedia, bad faith is "a sustained form of deception which consists of entertaining or pretending to entertain one set of feelings while acting as if influenced by another." The great encyclopedia goes on to provide examples: the solider who waves a flag of surrender but then fires when the enemy comes out of their trenches, the attorney who prosecutes a case she knows to be false, the representative of a company facing a labor dispute who comes to the negotiating table with no intent of compromising.
    That is, bad faith is when someone's apparent reasons for doing something aren't the same as the real reasons. This is distinct from malign intent. The uniformed solider who shoots you without pretending to surrender is acting in good faith, because what you see is what you get: the man whose clothes indicate that his job is to try to kill you is, in fact, trying to kill you.
    The policy of assuming good faith (and mercilessly punishing rare cases of bad faith when detected) would make sense if you lived in an honest world where what you see generally is what you get (and you wanted to keep it that way), a world where the possibility of hidden motives in everyday life wasn't a significant consideration.
    On the contrary, however, I think hidden motives in everyday life are ubiquitous. As evolved creatures, we're designed to believe as it benefited our ancestors to believe. As social animals in particular, the most beneficial belief isn't always the true one, because tricking your conspecifics into adopting a map that implies that they should benefit you is sometimes more valuable than possessing the map that reflects the territory, and the most persuasive lie is the one you believe yourself. The universal human default is to come up with reasons to persuade the other party why it's in their interests to do what you want - but admitting that you're doing that isn't part of the game. A world where people were straightforwardly trying to inform each other would look shocking and alien to us.
    But if that's the case (and you shouldn't take my word for it), being touchy about bad faith accusations seems counterproductive. If it's common for people's stated reasons to not be the same as the real reasons, it shouldn't be beyond the pale to think that of some particular person, nor should it necessarily entail cutting the "bad faith actor" out of public life - if only because, applied consistently, there would be no one left. Why would you trust anyone so highly as to think they never have a hidden agenda? Why would you trust yourself?
    The conviction that "bad faith" is unusual contributes to a warped view of the world in which conditions of information warfare are rationalized as an inevitable background fact of existence. In particular, people seem to believe that persistent good faith disagreements are an ordinary phenomenon - that there's nothing strange or unusual about a supposed state of affairs in which I'm an honest seeker of truth, and you're an honest seeker of truth, and yet we end up persistently disagreeing on some question of fact.
    I claim that this supposedly ordinary state of affairs is deeply weird at best, and probably ...
    12 min
  • EA - Nuclear winter - Reviewing the evidence, the complexities, and my conclusions by Michael Hinge
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Nuclear winter - Reviewing the evidence, the complexities, and my conclusions, published by Michael Hinge on August 25, 2023 on The Effective Altruism Forum.
    Nuclear winter is a topic that many have heard about, but where significant confusion can occur. I certainly knew little about the topic beyond the superficial prior to working on nuclear issues, and getting up to speed with the recent research is not trivial. Even then there are very large uncertainties around the topic, and different teams have come to different conclusions, which are unlikely to be resolved soon.
    However, this does not mean that we cannot draw some conclusions on nuclear winter. I worry that the fact there are still uncertainties has been taken to the extreme that nuclear winters are impossible, which I do not believe can be supported. Instead, I would suggest the balance of evidence suggests there is a very serious risk a large nuclear exchange could lead to catastrophic climate impacts. Given that the potential mortality of these winters could exceed the direct deaths, the evidence is worth discussing in detail, especially as it seems that we could prepare and respond.
    This is a controversial and complex topic, and I hope I have done it justice with my overview. Throughout this article I should however stress that these words are my own, along with any mistakes.
    Headline results, for those short on time
    This is a very long post, and for those short on time the thrust of my argument is summarized here. I've also included the relevant section(s) in the post that back up each point for those who wish to check my working on a specific piece of analysis.
    Firestorms can inject soot effectively into the stratosphere via their intense plumes. Once there, soot could persist for years, disrupting the climate (see "The case for concern: Firestorms, plumes and soot").
    Not every nuclear detonation would create a firestorm, but large nuclear detonations over dense cities have a high risk of causing them (see "The case for concern: Firestorms, plumes and soot").
    Not every weapon in arsenals will be used, and not all will hit cities. However, exchanges between Russia and NATO have the potential to result in hundreds or thousands of individual detonations over cities. This has the potential to result in a large enough soot injection that catastrophic cooling could occur (see "Complexities and Disagreements: Point one").
    The impacts of soot injections on the climate are nonlinear, and even if the highest thresholds in the literature are not met there still could be serious cooling. My personal estimates are that around 20-50 Tg of soot could be emitted in the event of a full-scale exchange between Russia and NATO with extensive targeting of cities. This is less than the upper threshold of 150 Tg from other sources, however that would still cause catastrophic crop losses without an immediate and effective response (see "Synthesis").
    There are disagreements between the teams on the dynamics of nuclear winters, which are complex. However, to date these disagreements center on a regional exchange of 100 weapons of 15 kt each, much smaller than Russian and NATO strategic weaponry. Some of the points raised in the debate are therefore far less relevant when looking at larger exchanges, where fuel loads would be higher, weapons are much more powerful and firestorms more likely (see "Complexities and Disagreements: The anatomy of the disagreements, followed by Synthesis").
    This means that nuclear winter is not discredited as a concept, and is very relevant for the projected mortality of nuclear conflict. There are uncertainties, and more research is needed (and possibly underway), however it remains a clear threat for larger conflicts in particular (see "Synthesis").
    Overall, although highly uncertain, I would expec...
    58 min
  • LW - Apply for the 2023 Developmental Interpretability Conference! by Stan van Wingerden
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Apply for the 2023 Developmental Interpretability Conference!, published by Stan van Wingerden on August 25, 2023 on LessWrong.
    What: A conference to advance the DevInterp research program
    When: 5-12 November 2023
    Where: Wytham Abbey, Oxford
    How: Apply now!
    We are pleased to announce the upcoming Developmental Interpretability Conference, hosted at the historic Wytham Abbey in Oxford from 5 to 12 November. This conference expands upon the 2023 Singular Learning Theory & Alignment Summit and provides an opportunity to collaboratively work on open problems in the DevInterp Research Agenda. The conference program will recall the basics of Singular Learning Theory & DevInterp and will discuss the latest advancements in Developmental Interpretability.
    Click here to apply! Space at the conference is limited, so be sure to apply early as applications may close when all slots have been filled. We hope to see you in Oxford this November!
    FAQ
    What are the prerequisites?
    The conference will use ideas from algebraic geometry, Bayesian statistics and physics to understand machine learning and AI alignment. Although helpful, it is not necessary to master all these topics to productively participate in the conference.
    In order to get the most out of the conference program, we highly recommend participants review introductory SLT material such as Distilling Singular Learning Theory by Liam Carroll. Participants may also benefit from watching several of the Singular Learning Theory & Alignment Summit 2023 lectures.
    I am skeptical about some of the arguments for AI Alignment. Do I need to buy AI X-risk to attend this conference?
    We believe the development of superintelligent AI poses a serious risk for humanity and the DevInterp agenda aims to make progress on this problem. However, while making progress on AI alignment is the motivation behind our scientific agenda, SLT and developmental interpretability are of broad interest and we invite attendance from those wishing to learn more or contribute, on their own terms.
    Do I need to have attended the SLT & Alignment Summer 2023 Summit to be able to attend this DevInterp Conference?
    No, you do not need to have attended the SLT & Alignment Summer 2023 Summit to attend the DevInterp Conference.
    Do I need to pay to attend the conference? And how about lodging, food and travel costs?
    The conference is free to attend. Lodging, food and transit between Oxford and the venue are all kindly provided by Wytham Abbey. Travel costs to Oxford are not paid for.
    Will you offer travel support?
    The amount of travel support we can provide is TBD. Let us know in the application form if you are blocked from attending because of travel costs, and we'll see what we can do.
    How do I apply?
    By filling in this application form.
    Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org
    3 min