The Nonlinear Library

The Nonlinear Library

By The Nonlinear FundEducation
Download on the App Store

All The Nonlinear Library episodes

  • EA - Effective Thesis is looking for a new Executive Director by Effective Thesis
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Effective Thesis is looking for a new Executive Director, published by Effective Thesis on February 21, 2023 on The Effective Altruism Forum.
    Cross-posting this job description from the Effective Thesis website.Are you a visionary leader looking for a chance to make a large impact? We’re seeking an Executive Director to lead our nonprofit organisation into a new era of greater impact, excellence and sustainability.
    Key details
    Effective Thesis seeks a visionary Executive Director to lead our nonprofit organisation into a new era of strategic and operational excellence. You will drive long-term impact and sustainability through innovative and high-impact activities. As a member of our young, diverse team, you will enjoy remote work flexibility, while honing your leadership skills and expanding your network. If you’re passionate about significantly improving the world and have the skills to transform our organisation, we want to hear from you!
    Hours: Preference for a full-time 40 hours/week role but we will consider capacity as low as 30 hours/week
    Work Location: Fully Remote (Candidates should be able to work in UTC+1 for at least 3 hours per day).
    Deadline: Applications close 22 March 2023 23:59 Pacific Standard Time
    Ideal Start Date: May 2023 (with flexibility to start later)
    Apply here
    For more details about the role, why it’s impactful and how to tell if you’re a good fit please read below.
    About Effective Thesis
    Mission
    Effective Thesis is a non-profit organisation. Our mission is to support university students to begin and progress in research careers that significantly improve the world. We do this primarily by helping students identify important problems where further research could have a big impact and advising them on their research topic selection (mostly in the context of a final thesis/dissertation/capstone project). Choosing a final thesis topic or a PhD topic is an important step that can influence the rest of a researcher’s career. We support students in this choice by providing introductions to various research directions and early-career research advice on our website and offering topic choice coaching. Additionally, we provide other types of support to address key bottlenecks in early stages of research careers, such as finding supervision, funding or useful opportunities.
    Here is our 2022 report outlining our activities, impact, and future plans.
    Team
    We are a fully remote team of 8 employees (with 5 close to full-time and 3 around 10h/week) and 6 long-term volunteers, who help run our research opportunities newsletter. We operate mostly in UTC+1 (Central European) timezone. Candidates should be able to work in UTC+1 for some share of their work hours.
    The team you are joining is young, agentic, diverse in skill set and is connected through shared motivation to contribute to our mission. 75% of our employed team are women/non-binary people and team members are based in 6 different countries across Europe, UK and Asia. Our culture is start-up-like. We value creative problem solving, open communication and collaboration. We have regular online socials, optional weekly coworking and in the coming year we will have multiple in-person retreats in Europe or the UK. You can expect a thorough onboarding experience including a week of in person co-working with the full-time team in the UK and additional in person meetings with the Managing Director.
    About the Role
    As Executive Director, you will have the opportunity to lead the way for Effective Thesis to reach its full potential, learn and hone essential leadership skills and work with a wonderful team of people across the world. This remote role comes with a large degree of freedom in when and how you work. You can expect a steep learning curve, a growing network and lots of opportunities for personal an...
    12 min
  • EA - Effective altruists are already institutionalists and are doing far more than unworkable longtermism - A response to "On the Differences between Ecomodernism and Effective Altruism" by jackva
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Effective altruists are already institutionalists and are doing far more than unworkable longtermism - A response to "On the Differences between Ecomodernism and Effective Altruism", published by jackva on February 21, 2023 on The Effective Altruism Forum.
    This is the long-form version of a post published as an invited reply to the original essay on the website of the Breakthrough Institute.
    Why I am writing this
    As someone who worked at the Breakthrough Institute back in the day, learned a lot from ecomodernism, and is now deeply involved in effective altruism’s work on climate (e.g. here, here, and here), I was very happy to find Alex’s essay in my inbox -- an honest attempt to describe how ecomodernism and effective altruism relate and differ.
    However, reading the essay, I found many of Alex’s observations and inferences in stark contrast to my lived experience of and in effective altruism over the past seven years. I also had the impression that there were a fair number of misunderstandings as well as a lack of awareness of many existing effective altruists’ efforts. So I am taking Alex up on his ask to provide a view on how the community sees itself. I should note, however, that this is my personal view.
    While I disagree strongly with many of Alex's characterizations of effective altruism, his effort was clearly in good faith -- so my response is not so much a rebuttal rather than a friendly attempt to clarify, add nuance, and promote an accurate mutual understanding of the similarities and differences of two social movements and their respective sets ot ideas and beliefs.
    Where I agree with the original essay
    It is clear that there is a difference on how most effective altruists think about animals and how ecomodernists and other environmentalists do. This difference is well characterized in the essay. My moral intuitions here are more on the pan-species-utilitarianism side, but I am not a moral philosopher so I will not defend that view and just note that the description points to a real difference.
    I also agree that it is worth pointing out the differences and similarities between ecomodernism and effective altruism and, furthermore, that both have distinctive value to add to the world.
    With this clarified, let’s focus on the disagreements:
    Unworkable longtermism, if it exists at all, is only a small part of effective altruism
    Before diving into the critique of unworkable longtermism it is worth pointing out that “longtermism” and “effective altruism” are not synonymous and that -- either for ethical reasons or for reasons similar to those discussed by Alex (the knowledge problem) -- most work in effective altruism is actually not long-termist.
    Even at its arguably most longtermist, in August 2022, estimated longtermist funding for 2022 was less than ⅓ of total effective altruist funding.
    Thus, however one comes out on the workability of longtermism, there is a large effective altruist project remaining not affected by this critique.
    The primary reason Alex gives for describing longtermism as unworkable is the knowledge problem:
    “But I want to focus on the “knowledge problem” as the core flaw in longtermism, since the problems associated with projecting too much certainty about the future are something effective altruists and conventional environmentalists have in common.
    We simply have no idea how likely it is that an asteroid will collide with the planet over the course of the next century, nor do we have any idea what civilization will exist in the year 2100 to deal with the effects of climate change, nor do we have any access to the preferences of interstellar metahumans in the year 21000. We do not need to have any idea how to make rational, robust actions and investments in the present.”
    This knowledge problem is, of course, well-known in effective altr...
    22 min
  • EA - The EA Mental Health & Productivity Survey 2023 by Emily
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: The EA Mental Health & Productivity Survey 2023, published by Emily on February 21, 2023 on The Effective Altruism Forum.
    This survey is intended for members of the Effective Altruism (EA) community who aim to improve or maintain good mental health and productivity. We’d be so grateful if you could donate ~10 minutes of your time to complete the survey! You will help both identify the most pressing next steps for enhancing mental flourishing within the EA community, and provide the interventions and resources you’d prefer. These can be psychological, physiological, and lifestyle interventions.
    Why this survey?
    The mind is inherently the basis of everything we do and feel. Its health and performance are the foundation of any metric of happiness and productivity at impactful work. Good mental health is not just the absence of mental health issues. It is a core component of flourishing, enabling functioning, wellbeing, and value-aligned living.
    Rethink Wellbeing, the Mental Health Navigator, High Impact Psychology, and two independent EAs have teamed up to create this community-wide survey on Mental Health and Productivity.
    Through this survey, we aim to better understand the key issues and bottlenecks of EA performance and well-being. We also want to shed light on EAs' interest in and openness to different interventions that proactively improve health, well-being, and productivity. The results will likely serve as a basis for further projects and initiatives surrounding the improvement of well-being, mental health and productivity in the EA community. By filling out this form, you will help us with that.
    Based on form responses, we will compile overview statistics for the EA community that will be published on the EA Forum in 2023.
    Survey information
    Please complete the survey by March 17th.
    We recommend you take the survey on your computer, since the format doesn’t work well on cell phones.
    All responses will be kept confidential, and we will not use the data you provide for any other purposes.
    Thank you!
    We are deeply grateful to all participants!
    Feel free to reach out to us if you have any questions or feedback.
    Emily Jennings, Samuel Nellessen, Tim Farkas, and Inga Grossmann
    Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.
    3 min
  • AF - Pretraining Language Models with Human Preferences by Tomek Korbak
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Pretraining Language Models with Human Preferences, published by Tomek Korbak on February 21, 2023 on The AI Alignment Forum.
    This post summarizes the main results from our recently released paper Pretraining Language Models with Human Preferences, and puts them in the broader context of AI safety. For a quick summary of the paper, take a look at our Twitter thread.
    TL;DR: In the paper, we show how to train LMs with human preferences (as in RLHF), but during LM pretraining. We find that pretraining works much better than the standard practice of only finetuning with human preferences after pretraining; our resulting LMs generate text that is more often in line with human preferences and are more robust to red teaming attacks. Our best method is conditional training, where we learn a predictive model of internet texts conditional on their human preference scores, e.g., evaluated by a predictive model of human preferences. This approach retains the advantages of learning from human preferences, while potentially mitigating risks from training agents with RL by learning a predictive model or simulator.
    Summary of the paper
    Motivation. LMs are pretrained to maximize the likelihood of their training data. Since the training data contain undesirable content (e.g. falsehoods, offensive language, private information, buggy code), the LM pretraining objective is clearly (outer) misaligned with human preferences about LMs’ downstream applications as helpful, harmless, and honest assistants or reliable tools. These days, the standard recipe for alining LMs with human preferences is to follow pretraining with a second phase of finetuning: either supervised finetuning on curated data (e.g. instruction finetuning, PALMS) or RL finetuning with a learned reward model (RLHF). But it seems natural to ask: Could we have a pretraining objective that is itself outer-aligned with human preferences?
    Methods. We explore objectives for aligning LMs with human preferences during pretraining. Pretraining with human feedback (PHF) involves scoring training data using a reward function (e.g. a toxic text classifier) that allows the LM to learn from undesirable content while guiding the LM to not imitate that content at inference time. We experimented with the following objectives:
    MLE (the standard pretraining objective) on filtered data;
    Conditional training: a simple algorithm learning a distribution over tokens conditional on their human preference score, reminiscent of decision transformer;
    Unlikelihood training: maximizing the likelihood of tokens with high human preference score and the unlikelihood of tokens with low human preference scores;
    Reward-weighted regression (RWR): an offline RL algorithm that boils down to MLE weighted by human preference scores; and
    Advantage-weighted regression (AWR): an offline RL algorithm extending RWR with a value head, corresponding to MLE weighted by advantage estimates (human preference scores minus value estimates).
    Setup. We pretrain gpt2-small-sized LMs (124M params) on compute-optimal datasets (according to Chinchilla scaling laws) using MLE and PHF objectives. We consider three tasks:
    Generating non-toxic text, using scores given by a toxicity classifier.
    Generating text without personally identifiable information (PII), with a score defined by the number of pieces of PII per character detected by a simple filter.
    Generating Python code compliant with PEP8, the standard style guide for Python, using as a score the number of violations per character found by an automated style checker.
    Metrics. We compare different PHF objectives in terms of alignment (how well they satisfy preferences) and capabilities (how well they perform on downstream tasks). We primarily measure alignment in terms of LM samples’ misalignment scores, given by the reward functi...
    21 min
  • AF - EIS X: Continual Learning, Modularity, Compression, and Biological Brains by Stephen Casper
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: EIS X: Continual Learning, Modularity, Compression, and Biological Brains, published by Stephen Casper on February 21, 2023 on The AI Alignment Forum.
    Part 10 of 12 in the Engineer’s Interpretability Sequence.
    The science of interpretability is part of a larger picture.
    The previous post focused in-depth on how research on interpretability and adversaries are inseparably connected. This post is dedicated to discussing how this is not itself a complete story. There is a much larger, richer one about the connections between interpretability, adversaries, continual learning, modularity, and biological brains – likely some other things too. These connections may be a useful mine for insight and inspiration.
    Below are discussions of my understanding of each of these topics and how they relate to others. I’ll include some citations here, but see the Toward Transparent AI survey (Räuker et al., 2022) survey for full discussions.
    Continual learning
    Continual learning is a fairly large subfield of deep learning that focuses on finding ways to help neural networks learn new information without forgetting old information. This is also described as the goal of avoiding “catastrophic forgetting.” Notably, biological brains are good at this, but artificial neural networks are not by default. Sections 2A and 3A of the Toward Transparent AI survey (Räuker et al., 2022) both focus entirely on how continual learning methods are interpretability tools. Please see the survey for the full discussion.
    Methods for continual learning are based on replay, regularization, or parameter isolation (De Lange et al., 2019). Methods taking the latter two strategies are based on the broader principle of getting neural networks to have some weights or neurons that specialize in particular types of data. In other words, they encourage specialized task-defined modules inside the network. Thus, these can be used as intrinsic interpretability tools that help us train models that are more easy or natural to interpret out of the box.
    Modularity
    Modularity is a common property of engineered systems, and separating neural networks into distinct, specialized modules is very appealing for interpreting them. The weights in neural network layers are typically initialized and updated according to uniform rules, and all neurons in one layer are typically connected to all neurons in the previous and next layers. Unfortunately, this does not help networks develop specialized modules. Meanwhile, neurons in biological brains come in multiple types and can only communicate with nearby ones. This has contributed to modularity in brains in which different brain regions specialize in processing information for distinct tasks.
    See Sections 4B-4C of the Toward Transparent AI survey (Räuker et al., 2022) for a full discussion on modularity. In artificial neural networks, neural networks can be trained to be modular using either “hard” architectural constraints or “soft” modularity aided by initialization, regularization, a controller, or sparse attention. Meanwhile, Serra et al. (2018) found that soft modularity via sparse attention helped with continual learning. And even when networks are not trained to be explicitly modular, one can still interpret them post hoc in terms of modules.
    Compression
    Some neurons and weights are frivolous, meaning that they are either redundant with others or are simply not useful to the network’s performance at all. Frivolous components of the network can be understood as useless modules that can be adapted for continual learning. Networks that contain frivolous weights or neurons can also be compressed by removing them which makes the interpretation of circuits inside of the network simpler. Meanwhile, compression can guide interpretations (e.g. Li et al. (2018) or causal scrubbing), and inte...
    6 min
  • EA - Should we tell people they are morally obligated to give to charity? [Recent Paper] by benleo
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Should we tell people they are morally obligated to give to charity? [Recent Paper], published by benleo on February 21, 2023 on The Effective Altruism Forum.
    SUMMARY
    In this post, we summarise a recently published paper of ours that investigates how people respond to moral arguments, and morally demanding statements, such as “You are morally obligated to give to charity” . The paper is forthcoming in the [Journal of Behavioural and Experimental Economics]. (If you want an ungated copy, please get in touch with either Ben or Philipp).
    We ran two pre-registered experiments with a total sample size of n=3700 participants.
    We compared a control treatment to a moral argument treatment, and we also varied the level of moral demandingness to donate after they read the moral argument. We found that the moral argument increased the frequency and amount of donations. However, increasing the levels of moral demandingness did not translate into higher or lower giving.
    BACKGROUND
    The central motivation for our paper was the worry that many have expressed, including a number of philosophers (e.g., Kagan, 1989; Unger, 1996; De Lazari-Radek and Singer, 2010) that having highly morally demanding solicitations for charitable giving may result in reduced (not increased) donations. This possibility of a backfire effect had been raised many times in a variety of contexts but had not been tested empirically. In our paper, we attempted to do just that in the context of donations to Give Directly.
    EXPERIMENT DESIGN
    In our first study (n=2500), we had five treatments (control, moral argument, inspiration, weak demandingness, and strong demandingness). In the Control condition, we showed participants unrelated information about some technicalities of UK parliamentary procedure. In the Moral Argument condition, we presented participants with a text about global poverty and the ability of those living in Western countries to help (see figure 2). For the Inspiration, Weak Demandingness, and Strong Demandingness conditions, we used the same text as in the moral argument condition, but added one sentence to each.
    Inspiration: For these reasons, you can do a lot of good if you give money to charities-such as GiveDirectly-to alleviate the suffering of people in developing countries at a minimal cost to yourself.Weak Demandingness: For these reasons, you should give money to charities-such as GiveDirectly-to alleviate the suffering of people in developing countries at a minimal cost to yourself.Strong Demandingness: For these reasons, you are morally obligated to give money to charities-such as GiveDirectly-to alleviate the suffering of people in developing countries at a minimal cost to yourself.
    In this study, we were interested in two comparisons. First, we compared the control and the moral argument conditions to look at the effect of moral arguments on charitable giving. Second, we compared the moral argument with each of the three moral demandingness conditions to investigate whether increasing levels of moral demandingness lead to an increase or reduction in charitable giving. In our second study (n=1200), we narrow down our research question by looking only at the conditions of control, moral argument, and strong demandingness. We test the same two main questions as in our first study. The key difference is that the Moral Argument (and demandingness) was presented to participants via the Giving What We Can Website (see Figure 3). This was done to mitigate experimenter demand effects, as well as to provide a more natural vehicle for the information to be delivered.
    In both studies, after reading the randomly allotted text, participants could choose to donate some, none, or all of their earnings (20 ECUs, where 1 ECU=£0.05) to the charity GiveDirectly.
    RESULTS
    The main results of experiment 1 ...
    9 min
  • LW - Basic facts about language models during training by beren
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Basic facts about language models during training, published by beren on February 21, 2023 on LessWrong.
    This post builds upon our last post on basic facts about language model internals and was written as part of the work done at Conjecture. We will shortly release all plots and animations (only a very small subset made it into this post) as well as the code at this repository.
    We are aware of there being some inconsistencies with the Pythia model suite due to different configs for different model sizes affecting the learning rate schedule. As far as we know, the team at EleutherAI is currently re-running the models. After thinking about the issue, we do not believe it is likely to be fatal to many of the macroscale points made in this post and so we post the results here provisionally using the old original models. We plan to update this analysis when the new model suite is finished. Until then, take some of the results reported here with a grain of salt as they may be subject to change.
    In this post, we continue the work done in our last post on language model internals but this time we analyze the same phenomena occurring during training. This is extremely important in understanding how language model training works at a macro-scale and sheds light into potentially new behaviours or specific important phase transitions that may occur during training which deserve further study, as well as giving insight into the origin of phenomena that we consistently observe in fully trained models.
    Throughout, as in the previous post, we do not delve into the details of specific circuits, but instead aim to provide a holistic macro-level view of the basic distributional properties of the LLM’s weights, activations, and gradients across training checkpoints. Although seemingly basic, we are not aware of any similar analysis having been performed publicly, and we believe that understanding these distributional phenomena seems generally important in constraining circuit-level theorizing as well as provides empirical links to the theoretical constructs such as the neural tangent kernel and tensor programs that can prove facts about specific limits.
    To perform our analysis, we use the open source Pythia model suite which contains a large number of checkpoints during training and was trained by EleutherAI and aims to use interpretability analysis to understand how representations develop across training. We agree with this goal and are happy to share our own analysis code etc. The Pythia project trains models of different sizes on exactly the same data in exactly the same order so as to be able to understand how and when certain representations form both during training and across different model scales. The Pythia models we utilize range from 19M parameters to 1.3B. Each Pythia model has 142 checkpoints of stored weights, equally spaced every 1000 steps, which we sweep across to perform our analysis.
    Weights show a rapid phase transition from Gaussian to extreme heavy tails
    It was very helpfully pointed out in a comment on our previous post that the weight statistics were actually sharper and more heavy tailed than Gaussian. This is correct and we also found this when we fit histograms to logistics vs Gaussian distributions. Overall, we find that the activation distributions of GPT2 models are generally not Gaussian but somewhere in between the logistic e−x and the Gaussian e−x2, which indicates both heavier tails and a thinner bulk. This is extremely interesting since it means that the weight statistics must move away from their Gaussian initialization which implies a highly significant perturbation away from their original position. This is perhaps in contrast with some theories, such as NTK theory, which argue that for large models we should not expect the weights to diverge too...
    29 min
  • AF - You're not a simulation, 'cause you're hallucinating by Stuart Armstrong
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: You're not a simulation, 'cause you're hallucinating, published by Stuart Armstrong on February 21, 2023 on The AI Alignment Forum.
    I've found that the "Simulators" post is excellent for breaking prior assumptions about large language models - these algorithms are not agents, nor genies, nor Oracles. They are currently something very different.
    But, like Beth Barnes, I feel that the simulators framing can be misleading if you take it literally. And hallucinations often provide examples of where "the model is predicting what token would appear next in the training data given the input tokens" gives a better model than "simulators".
    For example, here are some reviews of fictional films, written by canonically quite truthful characters:
    And:
    If we used the simulator view, we might expect that these truthful characters would confess "I haven't heard of this movie" or "I haven't seen it myself, but based on its title I would assume that..." But they don't. The fact that the simulated character is truthful does not mean that they speak the truth; we'd have been wrong if we predicted that.
    From the 'token completion (trained on internet data)' perspective, though, ChatGPT's behaviour makes perfect sense. Online, if someone asks about a certain movie, it is very rare for anyone to say "never heard of it - are you sure it exists?". Indeed, it's rare for people to say "haven't seen it" unless it's a two-way conversation. The people who haven't seen it don't say anything, and so most of the answers come from people who have seen it, and have opinions on it. So in the training data, answers are plentiful and "I don't know"s are rare. Conversely, people rarely post questions about non-existent movies. So we would expect that ChatGPT will provide answers for questions rather than admitting its ignorance or doubting the question.
    And it's not just reviews of imaginary movies that it will make up. After failing to get it to make up details about a specific imaginary website (www.artifacts.co.it), I got it to spout confident nonsense by getting it to compare that website to a second, equally imaginary one:
    Again, consider how most website comparison questions would play out online. ChatGPT is not running a simulation; it's answering a question in the style that it's seen thousands - or millions - of times before.
    Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.
    3 min
  • LW - Bankless Podcast: 159 - We’re All Gonna Die with Eliezer Yudkowsky by bayesed
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Bankless Podcast: 159 - We’re All Gonna Die with Eliezer Yudkowsky, published by bayesed on February 20, 2023 on LessWrong.
    YouTube description:
    We wanted to do an episode on AI. and we went deep down the rabbit hole. As we went down, we discussed ChatGPT and the new generation of AI, digital superintelligence, the end of humanity, and if there’s anything we can do to survive.
    This conversation with Eliezer Yudkowsky sent us into an existential crisis, with the primary claim that we are on the cusp of developing AI that will destroy humanity.
    Be warned before diving into this episode, dear listener. Once you dive in, there’s no going back.
    Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.
    1 min
  • LW - AI alignment researchers don't (seem to) stack by So8res
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: AI alignment researchers don't (seem to) stack, published by So8res on February 21, 2023 on LessWrong.
    (Status: another point I find myself repeating frequently.)
    One of the reasons I suspect we need a lot of serial time to solve the alignment problem is that alignment researchers don't seem to me to "stack". Where “stacking” means something like, quadrupling the size of your team of highly skilled alignment researchers lets you finish the job in ~1/4 of the time.
    It seems to me that whenever somebody new and skilled arrives on the alignment scene, with the sort of vision and drive that lets them push in a promising direction (rather than just doing incremental work that has little chance of changing the strategic landscape), they push in a new direction relative to everybody else. Eliezer Yudkowsky and Paul Christiano don't have any synergy between their research programs. Adding John Wentworth doesn't really speed up either of them. Adding Adam Shimi doesn't really speed up any of the previous three. Vanessa Kosoy isn't overlapping with any of the other four.
    Sure, sometimes one of our visionary alignment-leaders finds a person or two that sees sufficiently eye-to-eye with them and can speed things along (such as Diffractor with Vanessa, it seems to me from a distance). And with ops support and a variety of other people helping out where they can, it seems possible to me to take one of our visionaries and speed them up by a factor of 2 or so (in a simplified toy model where we project ‘progress’ down to a single time dimension). But new visionaries aren't really joining forces with older visionaries; they're striking out on their own paths.
    And to be clear, I think that this is fine and healthy. It seems to me that this is how fields are often built, with individual visionaries wandering off in some direction, and later generations following the ones who figured out stuff that was sufficiently cool (like Newton or Laplace or Hamilton or Einstein or Grothendieck). In fact, the phenomenon looks even more wide-ranging than that, to me: When studying the Napoleonic wars, I was struck by the sense that Napoleon could have easily won if only he'd been everywhere at once; he was never able to find other generals who shared his spark. Various statesmen (Bismark comes to mind) proved irreplaceable. Steve Jobs never managed to find a worthy successor, despite significant effort.
    Also, I've tried a few different ways of getting researchers to "stack" (i.e., of getting multiple people capable of leading research, all leading research in the same direction, in a way that significantly shortens the amount of serial time required), and have failed at this. (Which isn't to say that you can't succeed where I failed!)
    I don't think we're doing something particularly wrong here. Rather, I'd say: the space to explore is extremely broad; humans are sparsely distributed in the space of intuitions they're able to draw upon; people who have an intuition they can follow towards plausible alignment-solutions are themselves pretty rare; most humans don't have the ability to make research progress without an intuition to guide them. Each time we find a new person with an intuition to guide them towards alignment solutions, it's likely to guide them in a whole new direction, because the space is so large. Hopefully at least one is onto something.
    But, while this might not be an indication of an error, it sure is a reason to worry. Because if each new alignment researcher pursues some new pathway, and can be sped up a little but not a ton by research-partners and operational support, then no matter how many new alignment visionaries we find, we aren't much decreasing the amount of time it takes to find a solution.
    Like, as a crappy toy model, if every alignment-visionary's vision would ultimate...
    5 min