The Nonlinear Library

The Nonlinear Library

By The Nonlinear FundEducation
Download on the App Store

All The Nonlinear Library episodes

  • LW - Intelligence Officials Say U.S. Has Retrieved Craft of Non-Human Origin by lc
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Intelligence Officials Say U.S. Has Retrieved Craft of Non-Human Origin, published by lc on June 6, 2023 on LessWrong.
    Epistemic status: Big if true/I am clearly an idiot for even posting this.
    Some apparently real journalists have been approached by (& approached) several intelligence officials, some tasked specifically with investigating UFOs, who claim that the DoD has had evidence of alien intervention for a while in the form of partial & mostly-whole fragments of alien aircraft. A followup article where the publication outlines how the editors verified this persons' and others' claims and affiliations is here, and a part 2 is expected tomorrow.
    For some reason - very possibly because it's complete nonsense, or because they haven't had time to independently verify - the story has only been picked up by NYMag so far. The consensus among the people I've been reviewing this article with, is that it's either a complete hoax (i.e., the entire thing nearly top to bottom is some deliberate deception) or there's a non-negligible (>5%) chance aliens are here. I would love for someone who has a good understanding of the material to give an explanation (including possibly on priors, just thinking clearly about the content of the article) of why my friend group should discount this out of hand.
    Thus far I have been unconvinced by most stories of why we should to-the-point-of-not-caring-about-UFO-sightings-expect Aliens have to be big and obvious and tile the universe with fun, as opposed to operating some sort of noninterventionist monitored lightcone.
    Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.
    2 min
  • LW - One implementation of regulatory GPU restrictions by porby
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: One implementation of regulatory GPU restrictions, published by porby on June 4, 2023 on LessWrong.
    Some observations:
    ML-relevant hardware supply is bottlenecked at several points.
    One company, NVIDIA, is currently responsible for most purchasable hardware.
    NVIDIA already implements driver licensing to force data center customers to buy into the more expensive product line.
    NVIDIA would likely not oppose even onerous regulation on the use of its ML hardware if it gives them more of a competitive moat.
    Where can you stick a regulatory lever to prevent improper use of GPUs at scale?
    Here's one option!
    Align greed with oversight
    Buff up driver licensing with some form of hardware authentication so that only appropriately signed drivers can run. (NVIDIA may do this already; it would make sense!)
    Modify ML-focused products to brick themselves if they do not receive a periodic green light signal from a regulatory source with proper authentication.
    Require chain of custody for ML-focused products. If an auditable chain of custody cannot be produced for a particular piece of hardware, stop sending a green light.
    Use both targeted and randomized audits to confirm that hardware is being used for the stated purpose. Audits are not primarily automatic: regulators should have on-demand access to the physical hardware and verifiable detailed information on how the system is being used.
    If audits are blocked or refused by a product owner, stop sending a green light.
    While not a primary form of security, the hardware/driver could self-report on some kinds of usage patterns. It does not seem realistic to collect invasive data, but the combination of chain of custody and high-level samples of hardware utilization over time could help corroborate client claims or flag installations for audits.
    I imagine NVIDIA would be happy to offer Certified ML-MOAT-2023-B Compliant hardware and drivers. Given the opportunity, they may even push regulators to implement progressively more difficult forms of verification to make it more expensive for smaller hardware providers to break into the market.
    Between the cost of the hardware involved and the chain of custody information, it is unlikely that many small businesses would suffer excessive burdens as a result of the regulation. A company with 16 H100's is far less of a risk than a company with 140,000 H100's; you can probably skip the audit unless you have some reason to suspect the ostensibly small installations are attempting to exploit a loophole.
    Clouds
    GPU cloud service providers (CSPs) obscure chain of custody and complicate audits. Getting sufficient regulatory oversight here will likely have the most consumer-visible impacts.
    One option would be to require any compute requests above a certain threshold (either in terms of simultaneous compute or integrated compute over time) to meet KYC-style requirements.
    Regulators could lean on the CSP to extract information from large-scale customers, and those large-scale customers could be directly required to report or submit to audits as if they owned the hardware.
    Given the profit margins for GPU CSPs, I suspect most major AI labs to prefer building out their own infrastructure or making special deals (like OpenAI and Microsoft). The main value of having requirements on CSPs would be to plug an obvious loophole that bad actors might otherwise try to abuse because it's still cheaper than their other options for getting access to tons of compute.
    It is unlikely that any small research group would ever actually encounter the regulatory system. That helps keep regulatory overheads low and focuses the effort on the more dangerous massive deployments.
    Threat model exclusions
    This doesn't do anything against an extremely capable hostile actor in the limit. The hardware and software are in the hands of t...
    10 min
  • LW - How to Think About Activation Patching by Neel Nanda
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: How to Think About Activation Patching, published by Neel Nanda on June 4, 2023 on LessWrong.
    This is an excerpt from my post on attribution patching, that I think is of more general interest, around how to think about the technique of activation patching in mechanistic interpretability, and what it can and cannot teach us. You don't need to know what attribution patching is to read this, check out this section for an overview of activation patching
    What Is the Point of Patching?
    The conceptual framework I use when thinking about patching is to think of a model as an enormous mass of different circuits. On any given input, many circuits will be used. Further, while some circuits are likely very common and important (eg words after full stops start with capitals, or "this text is in English"), likely many are very rare and niche and will not matter on the vast majority of inputs (eg "this is token X in the Etsy terms and conditions footer" - a real thing that GPT-2 Small has neurons for!). For example, on IOI, beyond the actual IOI circuit, there is likely circuits to:
    Detect multi-token words (eg names) and assemble them into a single concept
    Detect that this is English text
    Detect that this is natural language text
    Encode unigram frequencies (under which John is 2-3x as likely as Mary!)
    Copying all names present in the context - if a name is present in the text it's way more likely to occur next than an unseen name!
    That this is the second clause of a sentence
    That Indirect Object Identification is the right grammatical structure to use
    That the next word should be a name
    As a mech interp researcher, this is really annoying! I can get traction on a circuit in isolation, and there's a range of tools with ablations, direct logit attribution, etc to unpick what's going on. And hopefully any given circuit will be clean and sparse, such that I can ignore most of a model's parameters and most activation dimensions, and focus on what's actually going on. But when any given input triggers many different circuits, it's really hard to know what's happening.
    The core point of patching is to solve this. In IOI, most of the circuits will fire the same way regardless of which name is the indirect object/repeated subject. So by formulating a clean and corrupted input that are as close as possible except for the key detail of this name, we can control for as many of the shared circuits as possible. Then, by patching in activations from one run to another, we will not affect the many shared circuits, but will let us isolate out the circuit we care about. Taking the logit difference (ie difference between the log prob of the correct and incorrect answer) also helps achieve this, by controlling for the circuits that decide whether to output a name at all.
    Importantly, patching should be robust to some of our conceptual frameworks being wrong, eg a model not having a linear representation, circuits mattering according to paths that go through every single layer, etc. Though it's much less informative when a circuit is diffuse across many heads and neurons than when it's sparse.
    Whenever reasoning about patching, it's valuable to keep this in mind! And to be clear to yourself about what exactly is the scope of your investigation - which circuits do you want to identify, how well have you controlled for the irrelevant circuits, what aspects of the circuits you do care about have you accidentally controlled for, etc. And even if you want to identify several circuits, it's generally best to try to have several types of clean and corrupted inputs, so that you can isolate out each circuit one at a time (eg in the Python docstring circuit).
    Patching Clean to Corrupted vs Corrupted to Clean
    One surprisingly subtle detail is whether you patch a clean activation into the corrupted run, or vice v...
    32 min
  • EA - Cause area report: Antimicrobial Resistance by Akhil
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Cause area report: Antimicrobial Resistance, published by Akhil on June 6, 2023 on The Effective Altruism Forum.
    This post is a summary of some of my work as a field strategy consultant at Schmidt Futures' Act 2 program, where I spoke with over a hundred experts and did a deep dive into antimicrobial resistance to find impactful investment opportunities within the cause area. The full report can be accessed here.
    AMR is a global health priority
    Antimicrobials, the medicines we use to fight infections, have played a foundational role in improving the length and quality of human life since penicillin and other antimicrobials were first developed in the early and mid 20th century.
    Antimicrobial resistance, or AMR, occurs when bacteria, viruses, fungi, and parasites evolve resistance to antimicrobials. As a result, antimicrobial medicine such as antibiotics and antifungals become ineffective and unable to fight infections in the body.
    AMR is responsible for millions of deaths each year, more than HIV or malaria (ARC 2022). The AMR Visualisation Tool, produced by Oxford University and IHME, visualises IHME data which finds that 1.27 million deaths per year are attributable to bacterial resistance and 4.95 million deaths per year are associated with bacterial resistance, as shown below.
    Figure 1: Composition of global bacterial infection related deaths, from AMR Visualisation Tool
    This burden does not include that of non-bacterial infections, such as fungi or pathogens, which might increase this burden by several factors. For instance, every year, there are 150 million cases of severe fungal infections, which result in 1.7 million deaths annually (Kainz et al 2020). Unlike for bacterial infections, we do not have good estimates of how many of those are associated or attributable to resistance.
    Concerningly, AMR is escalating at an increasing rate (USA data, Swiss data, Mhondoro et al 2019, Indian Council of Medical Research 2021). One prominent report estimates that AMR will result in 10 million deaths every year by 2050 (Jim O’Neill report, 2014).
    Even more concerningly, we may be at a critical juncture, where if we do not drastically change our current trajectory, we could run out of effective antimicrobials. This would mean that our ability to perform surgery, give cancer patients chemotherapy, or manage chronic diseases like cystic fibrosis and asthma, all of which hinge on the effectiveness of antimicrobials, would be significantly impacted. The very foundations of modern medicine could be threatened; the WHO has warned that we could return to a pre-antibiotic age, which would result in the average human life expectancy going down from 70 to 50 (WHO, 2015).
    Beyond the health effects, there is a profound economic cost to AMR – for patients, healthcare systems and the economy. In the USA, the CDC estimates that the cost of AMR is $55 billion every year (Dadgostar 2019). Studies show that as a result of AMR, the annual global GDP could decrease by 1% and there would be a 5-7% loss in low and middle income countries by 2050 (Dadgostar 2019). In conjunction, the World Bank states that AMR might limit gains in poverty reduction, push more people into extreme poverty and have significant labour market effects.
    The importance of AMR is recognised by major governments and multilateral organisations. The WHO calls AMR one of the greatest public health threats facing humanity, the UK government lists AMR on its National Risk Register, and both GAVI and the United Nations Foundation term AMR as a ‘silent pandemic’.
    AMR is a neglected field
    Although there has been some global response to AMR, it has not been proportional to its threat to healthcare systems and the economy. Despite many governments developing National Action Plans (NAPs) in response to the WHO call for the same in ...
    10 min
  • LW - The (local) unit of intelligence is FLOPs by boazbarak
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: The (local) unit of intelligence is FLOPs, published by boazbarak on June 5, 2023 on LessWrong.
    [Crossposting again on Lesswrong and Windowsontheory, with the hope I am not overstaying my welcome in LW.]
    Wealth can be measured by dollars. This is not a perfect measurement: it’s hard to account for purchasing power and circumstances when comparing people across varying countries or time periods. However, within a particular place and time, one can measure wealth in the local currency. It still does not capture everything (e.g., future earnings, social connections). But generally, all else being roughly equal, the more dollars one has, the wealthier one is.
    How do we measure intelligence? I am not interested in measuring the intelligence of individual humans or individual animals. Nor am I looking for a universal absolute scale of intelligence on which we could rank humans, elephants, and GPT4. (Indeed, it doesn’t seem that a one-dimensional comparison can be made; for example, we seem to be more intelligent than elephants on most dimensions, but they do have an impressive memory.) Rather, I want to compare different species within the same genus or different models within the same general architecture (e.g., Transformers).
    I think it’s fair to say that the local unit of intelligence for animal species is neurons. While elephants have larger brains than humans, within the genus Homo, to a first approximation, the bigger the brain, the more intelligent the species.
    (Figure from Bolihus et al.)
    I claim that within the current architectures and training frameworks of large language models, the local unit of intelligence is FLOPs. That is, as long as we follow the current paradigm of training transformer-based architectures within best practices of scaling compute and data, the more compute resources (FLOPs) invested in training the model, the more intelligent it is. This is an imperfect measurement, but probably one that is better than trying to give models “IQ exams” that were designed for humans (and even there have dubious value). Another way to say this is that the intelligence of the model scales with the number of “load-bearing gradient steps” that have gone into training it.
    So far, it might seem like a tautology, but as I claimed in the “intelligence forklift” post, this does have some implications. In particular, current general-purpose models such as ChatGPT are built in two phases. The first phase is a pretraining phase, in which the model is trained in a Trillion or more gradient steps on the next-token prediction task. The second phase is the adaptation/fine-tuning phase, in which, whether through instruction-tuning, reinforcement learning on human feedback (RLHF) or other methods, the model is “fine tuned” using fewer than a million gradient steps to be a better instruction-following or chatting agent. In other words, more than 99.9% (maybe as much as 99.9999%) of the FLOPs / gradient steps in training the model are invested during its pretraining phase.
    (One reason that the fine-tuning phase involves much fewer gradient steps is that, while the first phase can use any static data grabbed from the Internet, the second phase requires data that was especially collected for this task and often needs human labeling as well.)
    The adaptation phase can make a huge difference in the usefulness of the model. The chatbot arena doesn’t even contain non-fine-tuned models, and we can see that smaller but well-tuned models can put up a decent fight against ones that have at least 10 times the parameters (and so roughly at least 100 times the training compute). Unlike sashimi, language models should not be consumed raw.
    However, their “intelligence” is ultimately derived from the FLOPs invested in the base models. (See also this paper on the limitations of fine-tuning to c...
    9 min
  • LW - AISafety.info "How can I help?" FAQ by steven0461
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: AISafety.info "How can I help?" FAQ, published by steven0461 on June 5, 2023 on LessWrong.
    AI existential risk has been in the news recently. A lot of people have gotten interested in the problem and some want to know what they can do to help. Additionally, other existing routes to getting advice are getting overwhelmed, like AI Safety Support, 80,000 Hours, AGI Safety Fundamentals, AI Safety Quest, etc. With this in mind, we’ve created a new FAQ as a part of Stampy’s AI Safety Info, based mostly on ideas from plex, Linda Linsefors, and Severin Seehrich. We're continuing to improve these articles and we welcome feedback.
    By starting at the root of the tree and clicking on the articles at the bottom of each article, you can navigate to the article that most applies to your situation. It branches out into the rest of AISafety.info as well.
    Or you can just look at the full list here:
    I’m convinced that AI existential safety is important and want to contribute. What can I do to help?
    I want to help out AI alignment without necessarily making major life changes. What are some simple things I can do to contribute?
    I would like to focus on AI alignment, but it might be best to prioritize improving my life situation first. What should I do?
    I want to take big steps to contribute to AI alignment (e.g. making it my career). What should I do?
    How can I do conceptual, mathematical, or philosophical work on AI alignment?
    How can I use a background in the social sciences to help with AI alignment?
    How can I do organizational or operations work around AI alignment?
    How can I work on AGI safety outreach in academia and among experts?
    How can I work on public AI safety outreach?
    How can I work on AI policy?
    I’m interested in providing significant financial support to AI alignment. How should I go about this?
    How can I work on assessing AI alignment projects and distributing grants?
    How can I work on helping AI alignment researchers be more effective, e.g. as a coach?
    What should I do with my idea for helping with AI alignment?
    What subjects should I study at university to prepare myself for alignment research?
    I’d like to do experimental work (i.e. ML, coding) for AI alignment. What should I do?
    What should I do with my machine learning research idea for AI alignment?
    Who should I talk to about my non-research AI alignment coding project idea?
    How can I work toward AI alignment as a software engineer?
    How can I do machine learning programming work to help with AI alignment?
    Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.
    3 min
  • LW - Algorithmic Improvement Is Probably Faster Than Scaling Now by johnswentworth
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Algorithmic Improvement Is Probably Faster Than Scaling Now, published by johnswentworth on June 6, 2023 on LessWrong.
    The Story as of ~4 Years Ago
    Back in 2020, a group at OpenAI ran a conceptually simple test to quantify how much AI progress was attributable to algorithmic improvements. They took ImageNet models which were state-of-the-art at various times between 2012 and 2020, and checked how much compute was needed to train each to the level of AlexNet (the state-of-the-art from 2012). Main finding: over ~7 years, the compute required fell by ~44x. In other words, algorithmic progress yielded a compute-equivalent doubling time of ~16 months (though error bars are large in both directions).
    On the compute side of things, in 2018 a group at OpenAI estimated that the compute spent on the largest training runs was growing exponentially with a doubling rate of ~3.4 months, between 2012 and 2018.
    So at the time, the rate of improvement from compute scaling was much faster than the rate of improvement from algorithmic progress. (Though algorithmic improvement was still faster than Moore's Law; the compute increases were mostly driven by spending more money.)
    ... And That Immediately Fell Apart
    As is tradition, about 5 minutes after the OpenAI group hit publish on their post estimating a training compute doubling rate of ~3-4 months, that trend completely fell apart. At the time, the largest training run was AlphaGoZero, at about a mole of flops in 2017. Six years later, Metaculus currently estimates that GPT-4 took ~10-20 moles of flops. AlphaGoZero and its brethren were high outliers for the time, and the largest models today are only ~one order of magnitude bigger.
    A more recent paper with data through late 2022 separates out the trend of the largest models, and estimates their compute doubling time to be ~10 months. (They also helpfully separate the relative importance of data growth - though they estimate that the contribution of data was relatively small compared to compute growth and algorithmic improvement.)
    On the algorithmic side of things, a more recent estimate with more recent data (paper from late 2022) and fancier analysis estimates that algorithmic progress yielded a compute-equivalent doubling time of ~9 months (again with large error bars in both directions).
    ... and that was just in vision nets. I haven't seen careful analysis of LLMs (probably because they're newer, so harder to fit a trend), but eyeballing it... Chinchilla by itself must have been a factor-of-4 compute-equivalent improvement at least. And then there's been chain-of-thought and all the other progress in prompting and fine-tuning over the past couple years. That's all algorithmic progress, strategically speaking: it's getting better results by using the same amount of compute differently.
    Qualitatively: compare the progress in prompt engineering and Chinchilla and whatnot over the past ~year to the leisurely ~2x increase in large model size predicted by recent trends. It looks to me like algorithmic progress is now considerably faster than scaling.
    Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.
    4 min
  • EA - ALTER Israel - 2023 Mid-Year Update by Davidmanheim
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: ALTER Israel - 2023 Mid-Year Update, published by Davidmanheim on June 6, 2023 on The Effective Altruism Forum.
    ALTER is an organization in Israel that works on several EA priority areas and causes. This semiannual update is intended to inform the community of what we have been doing, and provide a touchpoint for those interested in engaging with us. Since the last update at the beginning of 2023, we have made progress on a number of areas, and have ambitious ideas for future projects.
    Progress to Date
    Since its founding, ALTER has started and run a number of projects.
    Organized and managed an AI safety conference in Israel, AISIC 2022 hosted at the Technion, bringing in several international speakers including Stuart Russell, to highlight AI Safety focused on existential-risk and global-catastrophic-risk, to researchers and academics in Israel. This was successful in raising the profile of AI safety here in Israel, and in helping identify prospective collaborators and researchers.
    Support for Vanessa Kosoy’s Learning-Theoretic Safety Agenda, including an ongoing prize competition, and work to hire researchers working in the area.
    Worked with Israel’s foreign ministry, academics here in Israel, and various delegations to and organizations at the Biological Weapons Convention to find avenues to promote Israel’s participation.
    Launched our project to get the Israeli government to iodize salt, to mitigate or eliminate the current iodine deficiency that we estimate causes an expected 4-IQ point loss to the median child born in Israel today.
    Worked on mapping the current state of metagenomic sequencing usage in Israel, in order to prepare for a potential use of widespread metagenomic monitoring for detecting novel pathogens.
    Organized and hosted a closed Q&A with Eliezer Yudkowsky while he was visiting Israel, for 20 people in Israel working on or interested in contributing to AI safety. This was followed by a larger LessWrong meetup with additional attendees.
    Current and Ongoing Work
    We have a number of ongoing projects related to both biorisk and AI safety.
    Fellowship program. We have started this program to support researchers interested in developing research agendas relevant to AI safety. Ram Rahum is our inaugural funded AI safety fellow, who was found via our AI Safety conference. Since then, he has co-organized a conference in London on rebellion and disobedience in AI jointly with academics in Israel, the US, and the UK. As a fellow, he is also continuing to work with academics in Israel as well as a number of researchers at Deep Mind on understanding strategic deception and multi-agent games and dynamics for ML systems. His research home is here and monthly updates are here. Rona Tobolsky is a policy fellow, and is also working with us on policy, largely focused on biorisk and iodization.
    Support for Vanessa Kosoy’s Learning-Theoretic AI Safety Agenda. To replace the former FTX funding, we have been promised funding from an EA donor lottery to fund a researcher working on the learning-theoretic safety agenda. We are working on recruiting a new researcher, and are excited about expanding this. Relatedly, we are helping support a singular learning theory workshop.
    Biosecurity. David Manheim and Rona Tobolsky attended the Biological Weapons Convention - Ninth Review Conference, and have continued looking at ways to push for greater participation by Israel, which is not currently a member. David will also be attending a UNIDIR conference on biorisk in July. We are also continuing to explore additional pathways for Israel to contribute to global pandemic preparedness, especially around PPE and metagenomic biosurveillance.
    AI field building. Alongside other work to build AI-safety work in Israel, ALTER helped initiate a round of the AGI Safety Fundamentals 101 program...
    6 min
  • AF - Algorithmic Improvement Is Probably Faster Than Scaling Now by johnswentworth
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Algorithmic Improvement Is Probably Faster Than Scaling Now, published by johnswentworth on June 6, 2023 on The AI Alignment Forum.
    The Story as of ~4 Years Ago
    Back in 2020, a group at OpenAI ran a conceptually simple test to quantify how much AI progress was attributable to algorithmic improvements. They took ImageNet models which were state-of-the-art at various times between 2012 and 2020, and checked how much compute was needed to train each to the level of AlexNet (the state-of-the-art from 2012). Main finding: over ~7 years, the compute required fell by ~44x. In other words, algorithmic progress yielded a compute-equivalent doubling time of ~16 months (though error bars are large in both directions).
    On the compute side of things, in 2018 a group at OpenAI estimated that the compute spent on the largest training runs was growing exponentially with a doubling rate of ~3.4 months, between 2012 and 2018.
    So at the time, the rate of improvement from compute scaling was much faster than the rate of improvement from algorithmic progress. (Though algorithmic improvement was still faster than Moore's Law; the compute increases were mostly driven by spending more money.)
    ... And That Immediately Fell Apart
    As is tradition, about 5 minutes after the OpenAI group hit publish on their post estimating a training compute doubling rate of ~3-4 months, that trend completely fell apart. At the time, the largest training run was AlphaGoZero, at about a mole of flops in 2017. Six years later, Metaculus currently estimates that GPT-4 took ~10-20 moles of flops. AlphaGoZero and its brethren were high outliers for the time, and the largest models today are only ~one order of magnitude bigger.
    A more recent paper with data through late 2022 separates out the trend of the largest models, and estimates their compute doubling time to be ~10 months. (They also helpfully separate the relative importance of data growth - though they estimate that the contribution of data was relatively small compared to compute growth and algorithmic improvement.)
    On the algorithmic side of things, a more recent estimate with more recent data (paper from late 2022) and fancier analysis estimates that algorithmic progress yielded a compute-equivalent doubling time of ~9 months (again with large error bars in both directions).
    ... and that was just in vision nets. I haven't seen careful analysis of LLMs (probably because they're newer, so harder to fit a trend), but eyeballing it... Chinchilla by itself must have been a factor-of-4 compute-equivalent improvement at least. And then there's been chain-of-thought and all the other progress in prompting and fine-tuning over the past couple years. That's all algorithmic progress, strategically speaking: it's getting better results by using the same amount of compute differently.
    Qualitatively: compare the progress in prompt engineering and Chinchilla and whatnot over the past ~year to the leisurely ~2x increase in large model size predicted by recent trends. It looks to me like algorithmic progress is now considerably faster than scaling.
    Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.
    4 min
  • EA - National EA groups shouldn’t focus on city groups by DavidNash
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: National EA groups shouldn’t focus on city groups, published by DavidNash on June 5, 2023 on The Effective Altruism Forum.
    Summary
    National EA groups have a variety of strategies available to them, but many seem to focus on supporting local city groups as the main activity with less consideration of other interventions. I think this leads to neglecting more impactful activities for national groups. Potentially this is because they are following more established groups/resources where city groups are given as a default example.
    Most people interested in EA are not joining local EA groups, and most people who could get more involved in EA don’t necessarily want to do that via joining a local group first
    From EA London attendance data for 2016-18, out of ~1300 people roughly 75% attended just 1 event, and only 10% attended 4 or more which suggests that most weren't aiming to become regular members
    There is an unseen majority of people who know about EA and want to have more impact who are neglected by a city-first strategy
    EA should attract more people than those also looking for community
    Community is still important, but should be seen as additional rather than a main focus
    Community can mean a lot of different things but I’m defining community in this post as a more densely connected subset of a network based around a location
    In practice this means a community is more likely to involve social gatherings, daily/weekly in person touchpoints
    A network will involve conferences, mentorship, newsletters/social media, monthly/yearly touchpoints
    There is probably value to having some city organisers if there is a critical mass of people interested in EA and the city has strong comparative advantages
    Alternative strategies could include cause specific field building, career advising, supporting professional networks nationally, organisation incubation, translation
    The Unseen Majority
    When most people hear about EA for the first time, it’s usually via an online resource (80,000 Hours, GWWC, podcast) or word of mouth. The message they receive is that EA cares about having more impact and that EA as a movement is trying to help people have more impact.
    This can contrast to the experience of going along to a local group (which is regularly suggested as a good way to get more involved with EA), and experiencing the main message as ‘join our community’, with less focus on helping that person have impact. This could lead to people who are focused on generating a lot of impact bouncing away from EA. Anecdotally I have heard people say that they don’t find that much value from attending local group events but are still interested in EA and focus on having an impact in their career.
    For the subset of people who are looking for community, local groups can be great. But for a lot of people who do not have that preference/have other life circumstances, this isn’t what they are looking for. People already have communities they are a part of (family, friends, professional, hobbies) and often don’t have time for many more. Anecdotally from conversations with other organisers the people most likely to join are those looking for a community - students, recent graduates or people who are new to the city.
    This can be self reinforcing as the people who are likely to keep on attending meetups are the ones with spare time and lacking community. We often use neglectedness when choosing cause areas, leading to support of unseen majorities - people in poorer parts of the world, animals and future beings. But when it comes to movement building there is less thought paid to those who aren’t visible. A lot of strategies I have seen are about increasing attendance or engagement at events rather than providing value to people who may not be as interested in attending lots of events each year but still wan...
    7 min