The Nonlinear Library

The Nonlinear Library

By The Nonlinear FundEducation
Download on the App Store

All The Nonlinear Library episodes

  • AF - TinyStories: Small Language Models That Still Speak Coherent English by Ulisse Mini
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: TinyStories: Small Language Models That Still Speak Coherent English, published by Ulisse Mini on May 28, 2023 on The AI Alignment Forum.
    Abstract
    Language models (LMs) are powerful tools for natural language processing, but they often struggle to produce coherent and fluent text when they are small. Models with around 125M parameters such as GPT-Neo (small) or GPT-2 (small) can rarely generate coherent and consistent English text beyond a few words even after extensive training. This raises the question of whether the emergence of the ability to produce coherent English text only occurs at larger scales (with hundreds of millions of parameters or more) and complex architectures (with many layers of global attention). In this work, we introduce TinyStories, a synthetic dataset of short stories that only contain words that a typical 3 to 4-year-olds usually understand, generated by GPT-3.5 and GPT-4.
    We show that TinyStories can be used to train and evaluate LMs that are much smaller than the state-of-the-art models (below 10 million total parameters), or have much simpler architectures (with only one transformer block), yet still produce fluent and consistent stories with several paragraphs that are diverse and have almost perfect grammar, and demonstrate reasoning capabilities. We also introduce a new paradigm for the evaluation of language models: We suggest a framework which uses GPT-4 to grade the content generated by these models as if those were stories written by students and graded by a (human) teacher. This new paradigm overcomes the flaws of standard benchmarks which often requires the model's output to be very structures, and moreover provides a multidimensional score for the model, providing scores for different capabilities such as grammar, creativity and consistency.
    We hope that TinyStories can facilitate the development, analysis and research of LMs, especially for low-resource or specialized domains, and shed light on the emergence of language capabilities in LMs.
    Implications
    Interpretability
    One part that isn't mentioned in the abstract but is interesting:
    We show that the trained SLMs appear to be substantially more interpretable than larger ones. When models have a small number of neurons and/or a small number of layers, we observe that both attention heads and MLP neurons have a meaningful function: Attention heads produce very clear attention patterns, with a clear separation between local and semantic heads, and MLP neurons typically activated on tokens that have a clear common role in the sentence. We visualize and analyze the attention and activation maps of the models, and show how they relate to the generation process and the story content.
    The difference between highly activating tokens for a neuron is striking, here's the tiny model:
    ...and here's GPT2-XL:
    Capabilities
    Again from the introduction (emphasis mine)
    However, it is currently not clear whether the inability of SLMs to produce coherent text is a result of the intrinsic complexity of natural language, or of the excessive breadth and diversity of the corpora used for training. When we train a model on Wikipedia, for example, we are not only teaching it how to speak English, but also how to encode and retrieve an immense amount of facts and concepts from various domains and disciplines. Could it be that SLMs are overwhelmed by the amount and variety of information they have to process and store, and that this hinders their ability to learn the core mechanisms and principles of language? This raises the question of whether we can design a dataset that preserves the essential elements of natural language, such as grammar, vocabulary, facts, and reasoning, but that is much smaller and more refined in terms of its breadth and diversity.
    If this is true, there could be ways to drastica...
    4 min
  • AF - Language Agents Reduce the Risk of Existential Catastrophe by Cameron Domenico Kirk-Giannini
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Language Agents Reduce the Risk of Existential Catastrophe, published by Cameron Domenico Kirk-Giannini on May 28, 2023 on The AI Alignment Forum.
    This post was written by Simon Goldstein, associate professor at the Dianoia Institute of Philosophy at ACU, and Cameron Domenico Kirk-Giannini, assistant professor at Rutgers University, for submission to the Open Philanthropy AI Worldviews Contest. Both authors are currently Philosophy Fellows at the Center for AI Safety.
    Abstract: Recent advances in natural language processing have given rise to a new kind of AI architecture: the language agent. By repeatedly calling an LLM to perform a variety of cognitive tasks, language agents are able to function autonomously to pursue goals specified in natural language and stored in a human-readable format. Because of their architecture, language agents exhibit behavior that is predictable according to the laws of folk psychology: they have desires and beliefs, and then make and update plans to pursue their desires given their beliefs. We argue that the rise of language agents significantly reduces the probability of an existential catastrophe due to loss of control over an AGI. This is because the probability of such an existential catastrophe is proportional to the difficulty of aligning AGI systems, and language agents significantly reduce that difficulty.
    In particular, language agents help to resolve three important issues related to aligning AIs: reward misspecification, goal misgeneralization, and uninterpretability.
    1. Misalignment and Existential Catastrophe
    There is a significant chance that artificial general intelligence will be developed in the not-so-distant future — by 2070, for example. How likely is it that the advent of AGI will lead to an existential catastrophe for humanity? Here it is worth distinguishing between two possibilities: an existential catastrophe could result from humans losing control over an AGI system (call this a misalignment catastrophe), or an existential catastrophe could result from humans using an AGI system deliberately to bring that catastrophe about (call this a malicious actor catastrophe). In what follows, we are interested in assessing the probability of a misalignment catastrophe rather than a malicious actor catastrophe.
    Carlsmith (2021) helpfully structures discussion of the probability of a misalignment catastrophe around six propositions. Since we are interested in the probability of a misalignment catastrophe conditional on the development of AGI, we focus our attention on the final four of these propositions, which we summarize as follows:
    1. Of the following two options, the first will be much more difficult:
    a. Build AGI systems with an acceptably low probability of engaging in power-seeking behavior.
    b. Build AGI systems that perform similarly but do not have an acceptably low probability of engaging in power-seeking behavior.
    2. Some AGI systems will be exposed to inputs which cause them to engage in power-seeking behavior.
    3. This power-seeking will scale to the point of permanently disempowering humanity.
    4. This disempowerment will constitute an existential catastrophe.
    Carlsmith assigns a probability of .4 to (1) conditional on the rise of AGI, a probability of .65 to (2) conditional on (1) and the rise of AGI, a probability of .4 to (3) conditional on (1), (2), and the rise of AGI, and a probability of .95 to (4) conditional on (1-3) and the rise of AGI. This translates into a probability of approximately .1 (10%) for a misalignment catastrophe conditional on the rise of AGI.
    We believe that the development of language agents ought to significantly decrease assessments of these probabilities. In particular, we suggest that the development of language agents reduces the probability of (1) conditional on the rise of AGI...
    49 min
  • EA - Drawing attention to invasive Lymantria dispar dispar spongy moth outbreaks as an important, neglected issue in wild animal welfare by Meghan Barrett
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Drawing attention to invasive Lymantria dispar dispar spongy moth outbreaks as an important, neglected issue in wild animal welfare, published by Meghan Barrett on May 28, 2023 on The Effective Altruism Forum.
    This post contains only the summary of a longer research post, written by Meghan Barrett and Hannah McKay. The full post can be found at the above link on Rethink Priorities website.
    Summary
    One aim of wild animal welfare research is to identify situations where large numbers of wild animals are managed by humans in ways that have significant welfare impacts. If the number of individuals is large and the welfare impacts significant, the issue is important. As humans are managing these animals, it is possible the welfare impacts could be moderated to reduce their suffering. The massive scale of invasive (e.g., non-native) Lymantria dispar dispar (spongy moth) outbreaks represents an unappreciated wild animal welfare issue, and thus deserves further attention from a welfare (not simply an invasive species-control) perspective.
    The spongy moth is not endemic to North America. The species experiences localized three year-long outbreaks of half a billion or more caterpillars/km2 every 10-15 years in regions where they are well established (including their native range). Spongy moths currently occupy at least 860,000 km2 in North America, only ¼ of their possible range (though most of the occupied area is not experiencing outbreak conditions, most of the time). L. dispar continues to spread slowly to new areas each year despite multi-million dollar efforts to stop expansion. Assuming spongy moth caterpillars are sentient, methods for actively controlling them during outbreaks cause substantial suffering. The aerial spray (Btk) ruptures the stomach, causing the insect to die from either starvation or sepsis over two to seven days. However, because outbreaks are so large, most caterpillars are not actively targeted for control, and ‘natural forces’ are allowed to reduce the outbreak.
    The most prominent natural forces to rein in an outbreak are starvation and disease. The accidentally introduced fungus, Entomophaga (meaning “insect eater”) maimaiga, digests caterpillars’ insides before pushing through the exoskeleton to release spores, usually within a week. LdNPV virus is also common in the spongy moth population, but only causes high levels of mortality during outbreaks when larvae are stressed from extreme competition. A symptom of severe LdNPV infection is “larval melting,” or the liquefaction of the insect’s internal organs.
    The scale of spongy moth outbreaks is tremendous, though notably these outbreaks are not necessarily higher-density than numbers of other insect species (e.g., 740 million to 6.2 billion individual wireworms/km2; Smithsonian, n.d.). However, spongy moths are one of the best tracked non-native insects (Grayson & Johnson, 2018; e.g., Stop the Spread program), providing us with better data for analyzing the scale of the welfare issue (both in terms of caterpillar density within outbreaks, and the total area affected by outbreaks). In addition, there is potential for significant range expansion by spongy moths that would increase the scope of the welfare concern, and there appears to be extreme suffering1 induced by both active and natural outbreak control. As a result, spongy moth welfare during outbreaks could be an issue of concern for animal welfare advocates.
    Further research could improve spongy moth welfare by: 1) identifying the most promising long-term interventions for preventing/reducing the occurrence of outbreaks behind the invasion front, 2) contributing to halting the spread of spongy moths into new areas, and 3) identifying the highest-welfare outbreak management strategies where outbreaks do occur.
    Thanks for listening. To help us ...
    4 min
  • LW - D&D.Sci 5E: Return of the League of Defenders by aphyer
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: D&D.Sci 5E: Return of the League of Defenders, published by aphyer on May 26, 2023 on LessWrong.
    This is an entry in the 'Dungeons & Data Science' series, a set of puzzles where players are given a dataset to analyze and an objective to pursue using information from that dataset.
    Note: this is a sequel to the original 'League of Defenders of the Storm' scenario, but uses a different ruleset. If you want to play that one first you can, but it's not necessary to have played that one in order to play this.
    STORY (skippable)
    You've been feeling pretty good about your past successes as an esports advisor on Cloud Liquid Gaming, using your Data Science skills to help them optimize their strategies against rival teams. But recently, you've gotten a very attractive offer from a North American team.
    The one constant in the esports scene is that the US and European teams invariably lose to Korean and Chinese ones. In the recent Mongolian Summer Invitational, no Western team ever beat an Asian one.
    The attempts of Western teams to hire away top Asian players have not helped. Recently, however, the 'Silver Shielders' team had the bright idea of hiring away the support staff rather than the players themselves. And that's where you come in.
    The sequel to the critically acclaimed 'League of Defenders of the Storm' is being released soon. While the full ruleset isn't available, your new employer does have a dataset of results of beta plays of the game, and is hoping for advice. In the upcoming release-day tournament, they're going to send a team of three of their best players to play against a foreign team - and they're hoping that with your help this might not be North America's 185th consecutive loss on the international stage.
    If you can employ your Data Science skills again, perhaps you can reverse the decline of NA as a region. (And get paid a lot. That's good too.)
    DATA & OBJECTIVES
    You need to select a team for your employers to play. You should choose three of the following 15 characters:
    Amazon Archer
    Bludgeon Bandit
    Captain Chakram
    Daring Duelist
    Flamethrower Felon
    Granite Golem
    Hammer Hurler
    Jaunty Javelineer
    Lamellar Legionary
    Matchlock Marauder
    Professor Pyro
    Rugged Ranger
    Silent Samurai
    Thunder Tyrant
    Wily Wizard
    Your goal is to maximize your team's winrate against an opposing team, which your employers believe will be playing the following three characters:
    Flamethrower Felon
    Rugged Ranger
    Silent Samurai
    (Note that you are allowed to select characters your opponents are also playing, but you are not allowed to select the same character more than once).
    To help you with this, you have a dataset of plays of the game. Each entry is a game that was played, the three characters on each team, and which team won.
    BONUS PVP OBJECTIVE
    You may also submit a PVP team. I recommend sending it as a PM to me, but if you don't mind other people seeing it you can just put it in your answer. The PVP team with the best overall record (sum of performances against all other submitted teams) will in theory win the right to specify the theme of an upcoming D&D.Sci scenario. In practice, I still owe the last winner abstractapplic their scenario first, and while I expect to finish that soon I've thought that for at least the past nine months, so maybe don't count on it too much.
    I don't want the existence of a PVP objective to incentivize people too strongly against posting findings in the chat, so as an effort to reduce the risk of your findings being used against you: if multiple people submit the same PVP team, I will break the tie in favor of whoever submits it earlier.
    I'll aim to post the ruleset and results on June 5th (giving one week and both weekends for players). If you find yourself wanting extra time, comment below and I can push this deadline back.
    As usual, working together is ...
    5 min
  • LW - You now can enable reacts on your own posts! (and other react info) by Ruby
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: You now can enable reacts on your own posts! (and other react info), published by Ruby on May 28, 2023 on LessWrong.
    LessWrong is experimenting with the addition of reacts to the site, as per the recent experimental Open Thread. We are now progressing to the next stage of the experiment: trying out reacts in actual discussion threads.
    The dev/moderator team will be proactively looking for posts to enable to react voting on (with author permission), but also any user can enable it themselves to help us experiment:
    When creating or editing a post, expand the "Options" section at the bottom and change the Voting system to Names-attached reactions
    The admins will also be on the lookout for good posts to enable reacts on (with author permission).
    Iterating on the react palette
    We're continuing to think about what reacts should be available. Thanks to everyone who's weighed in so far.
    I just spent time today and yesterday adding a range of new reacts and sorting them, with a focus on reacts that express LessWrong-style conversational moves. Hover-overs tooltips have a longer description that you can view on the live thing (reacts are enabled on this post).
    We'll be tracking which ones get used, but please comment here with further feedback and requests. In other places, it'd be cool if people said things like "wish I could react with X" for various values of X.
    Different palette views
    Raemon made it so you can change how you view the palette. See the options at the top there:
    The options are:
    List
    Mixed List and Grid
    Grid of Icons (no text)
    Grid with Names
    Long-term I expect we'll have just one view, but we've made several available so people can give feedback on which they prefer.
    Currently live reacts are getting tweaked and might shift slightly
    Reacts are experimental! We're continuing to add new reacts but also edit and delete existing ones. This means that some reacts will be deprecated (and might disappear) and also that some will have their meaning tweaked. E.g. we might change the "Roll to disbelieve" react to just be "Skeptical", and things like that. So during this experimental phase, be warned that people might have clicked a react with a slightly adjacent meaning. We'll try to avoid doing this too much.
    Reacts aren't ready for mobile yet
    It's a large design challenge to make reacts work on mobile as well as desktop, so we'll do that if the experiment seems worth turning into a permanent site feature. Till then, you'll have the most luck with reacts on desktop.
    Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.
    3 min
  • EA - Has Russia’s Invasion of Ukraine Changed Your Mind? by JoelMcGuire
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Has Russia’s Invasion of Ukraine Changed Your Mind?, published by JoelMcGuire on May 27, 2023 on The Effective Altruism Forum.
    [This post was written in a purely personal capacity, etc.]
    I recently had several long conversations with a friend about whether my regular doom-scrolling regarding the Ukraine war had sharpened my understanding of the world or mostly been a waste of time.
    Unfortunately, it seems more of the latter. When my mind has changed, it's been slight, and it’s unclear what actions my new views justify. Personally, this means I should probably go back to thinking about happiness and RCTs.
    I set out what I think are some relevant questions Russia's invasion of Ukraine could change your mind about and provide some sloppy commentary, but I'm interested to know what other EAs and rationalists think about this issue.
    High-level questions
    Likelihood of great power conflict
    It seems like the Metaculus forecasting community is now more worried about great power conflict than it was before the war. I assume the invasion of Ukraine is a causal factor. But I feel oddly reassured about this, like the world was ruled by drunks who sobered up when the knives came out, reminded that knives are sharp and bodily fluids are precious.
    After the invasion, the prospect of a Russia-USA War shifted from a 5-15% to a 25% chance before 2050. I hadn’t known about this forecast, but I would have assumed the opposite. Before the war, Russia viewed the US as a waning power, losing in Afghanistan, not-winning in Syria, Libya and Venezuela, riven by internecine strife and paralyzed by self-doubt. Meanwhile, Russia’s confidence in its comeback rose with each cost-effective success in Crimea, Syria, and Kazakhstan.
    Now Russia knows how hollow its military was. And it knows the USA knows. And it knows that NATO hand-me-downs are emptying its once vast stockpiles of tanks and APCs. I assume it won’t recover the depth of its armour stocks in the near term (it doesn’t have the USSR’s state capacity or industrial base). The USA also doesn’t need to fight Russia. If Ukraine is doing this well, then Ukraine + Poland + Baltics would probably do just fine. I’d put this more around 6.5%.
    I think a Russian war with a European state has probably increased simply based on Russia’s revealed willingness to go to war, in conjunction with forecasters predicting a good chance (20%-24%) that the US and China will go to war over Taiwan. Russia may find such a conflict an opportunity to attempt to occupy a square mile of uninhabited Lithuanian forest to create a safe zone for ethnic Russian speakers and puncture the myth of NATO’s 5th article.
    Will there be a 'World War Three' before 2050? | Metaculus
    The predicted probability to this question shifted by around 10%, from the 10-15% range to 20-25% after the war began. I assume this is mostly driven by Russia-NATO-initiated conflict. China-India conflict predictions have decreased from 30% pre-war to 17% before 2035 most recently. And China-US war predictions have stayed constant (20% before 2035). So the rise must stem from the increase in the likelihood of a Russia-US war or by other major powers between 2035 and 2050. I don’t think I agree with the community here, as I explained previously.
    Will China get involved in the Russo-Ukrainian conflict by 2024?
    China hasn’t involved itself in the Ukraine war yet. And the prospects for its involvement seem like they should dim over time — surely it would have acted or given more hints that it was considering doing so by now?
    This makes me more confused about whether China committed to a military confrontation with the West. If it has, and China believed it had more military-industrial capacity than the West (which is what I’d believe if I was China), then now is the perfect opportunity to drain Western stocks ...
    10 min
  • LW - Self-administered EMDR without a therapist is very useful for a lot of things! by Anton Rodenhauser
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Self-administered EMDR without a therapist is very useful for a lot of things!, published by Anton Rodenhauser on May 25, 2023 on LessWrong.
    EMDR (Eye Movement Desensitization and Reprocessing) therapy is a structured therapy that encourages the patient to focus briefly on a traumatic memory while simultaneously experiencing bilateral stimulation (typically eye movements, but also tones or taps), which is associated with a reduction in the vividness and emotion associated with the traumatic memories.
    EMDR is usually done with a therapist. However, you can also just do self-administered EMDR on your own - as often and whenever you want without any costs! Most people don't seem to know this great "do it on your own" option exists - I didn't. So my main goal with this post is to just make you aware of the fact that: "Hey, there's this great therapeutic tool called EMDR, and you can just do it!". And bearing some important caveats in mind, I highly recommend it. I've been doing emotional work like extended meditation retreats, Internal Family Systems (IFS), Ideal Parent Figures (IPF), Focusing, etc., for a long time, but self-administered EMDR has actually been one of the most helpful techniques of them all to me!
    Also, I've found EMDR helpful for a much broader set of problems than the official EMDR protocol implies. You can use it for anything "trauma" in the broadest sense of the word - any unhelpful emotional schema; any strongly negatively charged emotion, belief, or memory that is kind of stuck in unconsciousness.
    EMDR also doesn't have to be this complicated thing at all. I don't think you need to know more than there is in this blog post. You can totally 80/20 this, i.e. get 80% of the benefits from just 20% of the knowledge/effort.
    How to do self-administered EMDR
    Read here how to do EMDR with a therapist:
    For self-administered EMDR, you simply do the same, just without the therapist! So to summarize the most important steps of the official protocol, you:
    Identify:
    A specific scene or picture that best represents the traumatic event you’re targeting with this treatment. This is your “target.”
    A negative or unhelpful belief that is associated with the traumatic event. For example, “I am helpless,” “I am worthless”, or “I fucked up and therefore I’m a bad person.”
    A more wholesome alternative positive belief you would rather believe instead. This belief should reflect what is appropriate in the present. It should also feel true, at least to your rational current more wise self. For example, perhaps you survived an assault, with the subsequent negative belief: “I’m in danger.” You might replace it with the positive belief: “I’m safe now.”
    Focus on your target (the traumatic memory/event/whatever) while you subject yourself to bilateral stimulation. E.g. you look at a dot moving from left to right on a screen. Pay careful attention to all the negative beliefs and disturbing emotions and bodily sensations that arise. Take note of all your reactions to the processing - good, bad, or neutral, and any new insights, associations, or emotions you experience. During this process, you “digest” or process the feelings, images, and beliefs that occur in relation to the trauma. This step is called "desensitization." You do this until the target is no longer emotionally disturbing.
    Once you've desensitized your target, you can begin the "installation step." Again subject yourself to bilateral stimulation while you focus on your target. But now you also focus on your positive belief and try to override the original negative belief with it. To do this, it might help to alternate between the two beliefs in your mind, but put more emphasis on the positive belief. Do this until the new belief is "installed" and you no longer believe the negative old belief.
    Here's what you can do fo...
    17 min
  • LW - Should Rational Animations invite viewers to read content on LessWrong? by Writer
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Should Rational Animations invite viewers to read content on LessWrong?, published by Writer on May 27, 2023 on LessWrong.
    When I introduced Rational Animations, I wrote:
    What I won't do is aggressively advertise LessWrong and the EA Forum. If the channel succeeds, I will organize fundraisers for EA charities. If I adapt an article for YT, I will link it in the description or just credit the author. If I use quotes from an author on LW or the EA Forum, I will probably credit them on-screen. But I will never say: "Come on LW! Plenty of cool people there!" especially if the channel becomes big. Otherwise, "plenty of cool people" becomes Reddit pretty fast.
    If the channel becomes big, I will also refrain from posting direct links to LW and the EA Forum. Remind me if I ever forget. And let me know if these rules are not conservative enough.
    In my most recent communications with Open Phil, we discussed the fact that a YouTube video aimed at educating on a particular topic would be more effective if viewers had an easy way to fall into an "intellectual rabbit hole" to learn more. So a potential way to increase Rational Animations' impact is to increase the number of calls to action to read the sources of our videos. LessWrong and the EA Forum are good candidates for people to go learn more. Suppose we adapt an article from the sequences. One option is to link to readthesequences.com, another is to link to the version of the article hosted on LessWrong. It seems to me that linking to the version hosted on LessWrong would dramatically increase the chance of people falling into an "intellectual rabbit hole" and learning a lot more and eventually start to contribute.Here are what I think are the main cons and pros of explicitly inviting people to read stuff on LessWrong:
    Cons:
    If lots of people join at once we may get a brief period of lower-quality posts and comments until moderation catches up.
    If we can't onboard people fast enough, the quality of what's in the site will become a lot lower over time, and people producing excellent content will be driven away.
    Pro:
    If people read LessWrong, they will engage with important topics such as reducing existential risk from AI. Eventually, some of them might be able to contribute in important ways, such as by doing alignment research.
    I'm more optimistic now about trying to invite people than I was in 2021, mainly thanks to recent adjustments in moderation policy and solutions such as the Rejected Content Section, which, in my understanding, are meant at least in part for dealing with the large influx of new users resulting from the increased publicity around AGI x-risk. I think it likely (~ 70%) that such moderation norms are strong enough filters that they would select for good contributions and for users that can contribute in positive ways.
    Among the cons, I think number 2 is worse and more likely to happen without a strong and deliberate moderation effort. That said, it's probably relatively easy and safe to run experiments.
    I don't think many people would make a new account as a result of an isolated call to action to read an article hosted on LessWrong. I have high confidence that the number of new accounts would be between 100 and 1000 per million views, given the three conditions in this market. You should treat the market as providing a per-video upper bound, since the conditions describe some very aggressive publicity.
    Market conditions:
    In a video, Rational Animations invites viewers to read a LessWrong article or sequence. It's an explicit invitation made in the narration of the video rather than, e.g., only as in-video text.
    The article/sequence is linked in three places: at the end of the video (e.g., in place or near the Patreon link), at the top of the video description, and in the pinned comment.
    The video accrues...
    5 min
  • EA - Do you think decreasing the consumption of animals is good/bad? Think again? by Vasco Grilo
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Do you think decreasing the consumption of animals is good/bad? Think again?, published by Vasco Grilo on May 27, 2023 on The Effective Altruism Forum.
    Question
    Do you think decreasing the consumption of animals is good/bad? For which groups of farmed animals?
    Context
    I stopped eating animals 4 years ago mostly to decrease the suffering of farmed animals. I am glad I did that based on the information I had at the time. However, I am no longer confident that decreasing the consumption of animals is good/bad. It has many effects:
    Decreasing the number of factory-farmed animals.
    I believe this would be good for chickens, since I expect them to have negative lives. I estimated the lives of broilers in conventional and reformed scenarios are, per unit time, 2.58 and 0.574 times as bad as human lives are good (see 2nd table). However, these numbers are not resilient:
    On the one hand, if I consider disabling pain is 10 (instead of 100) times as bad as hurtful pain, the lives of broilers in conventional and reformed scenarios would be, per unit time, 2.73 % and 26.2 % as good as human lives. Nevertheless, disabling pain being only 10 times as bad as hurtful pain seems quite implausible if one thinks being alive is as good as hurtful pain is bad.
    On the other hand, I may be overestimating broilers’ pleasurable experiences.
    I guess the same applies to other species, but I honestly do not know. Figuring out whether farmed shrimps and prawns have good/bad lives seems especially important, since they are arguably the driver for the welfare of farmed animals.
    Decreasing the production of animal feed, and therefore reducing crop area, which tends to:
    Increase the population of wild animals, which I do not know whether it is good or bad. I think the welfare of terrestrial wild animals is driven by that of terrestrial arthropods, but I am very uncertain about whether they have good or bad lives. I recommend checking this preprint from Heather Browning and Walter Weit for an overview of the welfare status of wild animals.
    Decrease the resilience against food shocks. As I wrote here:
    The smaller the population of (farmed) animals, the less animal feed could be directed to humans to mitigate the food shocks caused by the lower temperature, light and humidity during abrupt sunlight reduction scenarios (ASRS), which can be a nuclear winter, volcanic winter, or impact winter.
    Because producing calories from animals is much less efficient than from plants, decreasing the number of animals results in a smaller area of crops.
    So the agricultural system would be less oversized (i.e. it would have a smaller safety margin), and scaling up food production to counter the lower yields during an ASRS would be harder.
    To maximise calorie supply, farmed animals should stop being fed and quickly be culled after the onset of an ASRS. This would decrease the starvation of humans and farmed animals, but these would tend to experience more severe pain for a faster slaughtering rate.
    As a side note, increasing food waste would also increase resilience against food shocks, as long as it can be promptly cut down. One can even argue humanity should increase (easily reducible) food waste instead of the population of farmed animals. However, I suspect the latter is more tractable.
    Increase biodiversity, which arguably increases existential risk due to ecosystem collapse (see Kareiva 2018).
    Decreasing greenhouse gas emissions, and therefore decreasing global warming.
    I have little idea whether this is good or bad.
    Firstly, it is quite unclear whether climate change is good or bad for wild animals.
    Secondly, although more global warming makes climate change worse for humans, I believe it mitigates the food shocks caused by ASRSs. Accounting for both of these effects, I estimated the optimal median global warming i...
    12 min
  • AF - Hands-On Experience Is Not Magic by Thane Ruthenis
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Hands-On Experience Is Not Magic, published by Thane Ruthenis on May 27, 2023 on The AI Alignment Forum.
    Here are some views, oftentimes held in a cluster:
    You can't make strong predictions about what superintelligent AGIs will be like. We've never seen anything like this before. We can't know that they'll FOOM, that they'll have alien values, that they'll kill everyone. You can speculate, but making strong predictions about them? That can't be invalid.
    You can't figure out how to align an AGI without having an AGI on-hand. Iterative design is the only approach to design that works in practice. Aligning AGI right on the first try isn't simply hard, it's impossible, so racing to build an AGI to experiment with is the correct approach for aligning it.
    An AGI cannot invent nanotechnology/brain-hacking/robotics/[insert speculative technology] just from the data already available to humanity, then use its newfound understanding to build nanofactories/take over the world/whatever on the first try. It'll have to engage in extensive, iterative experimentation first, and there'll be many opportunities to notice what it's doing and stop it.
    More broadly, you can't genuinely generalize out of distribution. The sharp left turn is a fantasy — you can't improve without the policy gradient, and unless there's someone holding your hand and teaching you, you can only figure it out by trial-and-error. Thus, there wouldn't be genuine sharp AGI discontinuities.
    There's something special about training by SGD, and the "inscrutable" algorithms produced this way. They're a specific kind of "connectivist" algorithms made up of an inchoate mess of specialized heuristics. This is why interpretability is difficult — it involves translating these special algorithms into a more high-level form — and indeed, it's why AIs may be inherently uninterpretable!
    You can probably see the common theme here. It holds that learning by practical experience (henceforth LPE) is the only process by which a certain kind of cognitive algorithms can be generated. LPE is the only way to become proficient in some domains, and the current AI paradigm works because it implements this kind of learning, and it only works inasmuch as it implements this kind of learning.
    All in all, it's not totally impossible. I myself had suggested that some capabilities may only be implementable via one algorithm and one algorithm only.
    But I think this is false, in this case. And perhaps, when put this way, it already looks false to you as well.
    If not, let's dig into the why.
    A Toy Formal Model
    What is a "heuristic", fundamentally speaking? It's a recorded statistical correlation — the knowledge that if you're operating in some environment E with the intent to achieve some goal G, taking the action A is likely to lead to achieving that goal.
    As a toy formality, we can say that it's a structure of the following form:
    The question is: what information is necessary for computing h? Clearly you need to know E and G — the structure of the environment and what you're trying to do there. But is there anything else?
    The LPE view says yes: you also need a set of "training scenarios" S={EA1, ..., EAn}, where the results of taking various actions Ai on the environment are shown. Not because you need to learn the environment's structure — we're already assuming it's known. No, you need them because... because...
    Perhaps I'm failing the ITT here, but I think the argument just breaks down at this step, in a way that can't be patched. It seems clear, to me, that E itself is entirely sufficient to compute h, essentially by definition. If heuristics are statistical correlations, it should be sufficient to know the statistical model of the environment to generate one!
    Toy-formally, P(h|ES)=P(h|E). Once the environment's structure is known, you gain no...
    9 min