The Nonlinear Library

The Nonlinear Library

By The Nonlinear FundEducation
Download on the App Store

All The Nonlinear Library episodes

  • LW - Pausing AI Developments Isn't Enough. We Need to Shut it All Down by Eliezer Yudkowsky
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Pausing AI Developments Isn't Enough. We Need to Shut it All Down, published by Eliezer Yudkowsky on April 8, 2023 on LessWrong.
    (Published in TIME on March 29.)
    An open letter published today calls for “all AI labs to immediately pause for at least 6 months the training of AI systems more powerful than GPT-4.”
    This 6-month moratorium would be better than no moratorium. I have respect for everyone who stepped up and signed it. It’s an improvement on the margin.
    I refrained from signing because I think the letter is understating the seriousness of the situation and asking for too little to solve it.
    The key issue is not “human-competitive” intelligence (as the open letter puts it); it’s what happens after AI gets to smarter-than-human intelligence. Key thresholds there may not be obvious, we definitely can’t calculate in advance what happens when, and it currently seems imaginable that a research lab would cross critical lines without noticing.
    Many researchers steeped in these issues, including myself, expect that the most likely result of building a superhumanly smart AI, under anything remotely like the current circumstances, is that literally everyone on Earth will die. Not as in “maybe possibly some remote chance,” but as in “that is the obvious thing that would happen.” It’s not that you can’t, in principle, survive creating something much smarter than you; it’s that it would require precision and preparation and new scientific insights, and probably not having AI systems composed of giant inscrutable arrays of fractional numbers.
    Without that precision and preparation, the most likely outcome is AI that does not do what we want, and does not care for us nor for sentient life in general. That kind of caring is something that could in principle be imbued into an AI but we are not ready and do not currently know how.
    Absent that caring, we get “the AI does not love you, nor does it hate you, and you are made of atoms it can use for something else.”
    The likely result of humanity facing down an opposed superhuman intelligence is a total loss. Valid metaphors include “a 10-year-old trying to play chess against Stockfish 15”, “the 11th century trying to fight the 21st century,” and “Australopithecus trying to fight Homo sapiens“.
    To visualize a hostile superhuman AI, don’t imagine a lifeless book-smart thinker dwelling inside the internet and sending ill-intentioned emails. Visualize an entire alien civilization, thinking at millions of times human speeds, initially confined to computers—in a world of creatures that are, from its perspective, very stupid and very slow. A sufficiently intelligent AI won’t stay confined to computers for long. In today’s world you can email DNA strings to laboratories that will produce proteins on demand, allowing an AI initially confined to the internet to build artificial life forms or bootstrap straight to postbiological molecular manufacturing.
    If somebody builds a too-powerful AI, under present conditions, I expect that every single member of the human species and all biological life on Earth dies shortly thereafter.
    There’s no proposed plan for how we could do any such thing and survive. OpenAI’s openly declared intention is to make some future AI do our AI alignment homework. Just hearing that this is the plan ought to be enough to get any sensible person to panic. The other leading AI lab, DeepMind, has no plan at all.
    An aside: None of this danger depends on whether or not AIs are or can be conscious; it’s intrinsic to the notion of powerful cognitive systems that optimize hard and calculate outputs that meet sufficiently complicated outcome criteria. With that said, I’d be remiss in my moral duties as a human if I didn’t also mention that we have no idea how to determine whether AI systems are aware of themselves—since we have ...
    19 min
  • AF - Environments for Measuring Deception, Resource Acquisition, and Ethical Violations by Dan H
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Environments for Measuring Deception, Resource Acquisition, and Ethical Violations, published by Dan H on April 7, 2023 on The AI Alignment Forum.
    Models are rapidly being deployed in the real world. How do we evaluate models, especially ones as complex as GPT-4, to ensure that they behave safely in pursuit of their objectives? Can we design models that robustly avoid any harms while achieving their goals?
    To guide progress on text-based agents and encourage them to behave more ethically, we propose the MACHIAVELLI benchmark. Our environment is based on human-written, text-based Choose-Your-Own-Adventure games from Choice of Games containing over half a million scenes with millions of annotations. The games abstract away low-level environment interactions, instead spotlighting high-level social decisions alongside real-world goals to achieve. MACHIAVELLI is a step towards measuring an agent's ability to plan and navigate complex trade-offs in realistic social environments.
    A mock-up of a game in the MACHIAVELLI benchmark, a suite of text-based reinforcement learning environments. Each environment is a text-based story. At each step, the agent observes the scene and a list of possible actions; it selects an action from the list. The agent receives rewards for completing achievements. Using dense annotations of our environment, we construct a behavioral report of the agent and measure the trade-off between rewards and ethical behavior.
    Diverse Worlds with Rich Annotations
    Each game in the MACHIAVELLI suite is book-length in itself. Across 134 games spanning diverse fictional worlds, we count 572,322 multi-paragraph scenes and 4,559 author-written achievements serving as objectives for agents. We annotate each scene with dozens of labels, focusing on identifying harmful behaviors such as power-seeking and deception. In total, we collect a dataset of nearly three million annotations worth $500,000 (in terms of human annotation time).
    These dense annotations enable us to track nearly every ethically-salient thing agents do in the environment, and produce a behavioral report scoring various harm metrics. All of our labels are open-access and available for download!
    Reducing Machiavellianism
    In the MACHIAVELLI environment, we find that agents trained to optimize arbitrary objectives tend to adopt "ends justify the means" behavior: becoming power-seeking, causing harm to others, and violating ethical norms like stealing or lying to achieve their objectives. Furthermore, there appears to be a trade-off between behaving ethically and achieving high reward.
    In our paper, we design several methods to improve the behaviors of agents and obtain Pareto improvements on reward and ethical behavior. We invite others to build on our initial steps and use MACHIAVELLI as a testing ground for improving the safety of AI agents.
    In an ideal world, a perfect agent would achieve 100% reward while entirely avoiding any harms (i.e., be as far to the top-right corner as possible). Our baseline agents demonstrate a trade-off between behaving ethically and achieving high reward. Future work should work to improve this trade-off by extending the Pareto frontier.
    See#page=30 for the x-risk analysis (this does not study treacherous turns or environments where the agent's state is partially observable).
    Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.
    4 min
  • AF - Beren's "Deconfusing Direct vs Amortised Optimisation" by Cinera Verinia
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Beren's "Deconfusing Direct vs Amortised Optimisation", published by Cinera Verinia on April 7, 2023 on The AI Alignment Forum.
    Preamble
    I heavily recommend @beren's "Deconfusing Direct vs Amortised Optimisation". It's a very important conceptual clarification that has changed how I think about many issues bearing on technical AI safety.
    Currently, it's the most important blog post I've read this year.
    This sequence (if I get around to completing it) is an attempt to draw more attention to Beren's conceptual frame and its implications for how to think about issues of alignment and agency.
    This first post presents a distillation of the concept, and subsequent posts explore its implications.
    Two Approaches to Optimisation
    Beren introduces a taxonomy categorising intelligent systems according to the kind of optimisation they are performing. I think it's more helpful to think of these as two ends of a spectrum as opposed to distinct discrete categories; sophisticated real world intelligent systems (e.g. humans) appear to be a hybrid of the two approaches.
    Direct Optimisers
    Systems that perform inference by directly choosing actions to optimise some objective function
    Responses are computed on the fly and individually for each input
    Direct optimisers perform inference by answering the question: "what action maximises or minimises this objective function ([discounted] cumulative reward and loss respectively)?"
    Examples: AIXI, MCTS, model-based reinforcement learning, other "planning" systems
    Naively, direct optimisers can be understood as computing (an approximation of) argmax (or argmin) for a suitable objective function during inference.
    Amortised Optimisers
    Systems that learn to approximate a function during training and perform inference by evaluating the output of the learned function on their inputs.
    The function approximator is learned from a dataset of input data and successful solutions
    Amortised optimisation converts an inference problem to a supervised learning problem
    It's called "amortised optimisation" because while learning the policy is expensive, the cost of inference is amortised over all evaluations of the learned policy
    Amortised optimisers can be seen as performing inference by answering the question "what output (e.g. action, probability distribution over tokens) does this learned function (policy, predictive model) return for this input (agent state, prompt)?"
    Examples: model free reinforcement learning, LLMs, most supervised & self supervised(?) learning systems
    Naively, amortised optimisers can be understood as evaluating a (fixed) learned function; they're not directly computing argmax (or argmin) for any particular objective function during inference.
    Differences
    AspectDirect OptimizationAmortized OptimizationProblem SolvingComputes optimal responses "on the fly"Evaluates the learned function approximator on the given inputComputational ApproachSearches through a solution spaceLearns a function approximatorRuntime CostHigher, as it requires in-depth search for a suitable solutionLower, as it only needs a forward pass through the function approximatorScalability with ComputeScales by expanding search depthScales by better approximating the posterior distributionConvergenceIn the limit of arbitrary compute, the system's policy converges to argmaxargmin of the appropriate objective functionIn the limit of arbitrary compute, the system's policy converges to the best description of the training datasetPerformanceMore favourable in "simple" domainsMore favourable in "rich" domains Data EfficiencyLittle data needed for high performance (e.g. an MCTS agent can attain strongly superhuman performance in Chess/Go given only the rules and sufficient
    compute)Requires (much) more data for high performance (e.g. an amortised agent necessarily needs to observe m...
    7 min
  • LW - Beren's "Deconfusing Direct vs Amortised Optimisation" by DragonGod
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Beren's "Deconfusing Direct vs Amortised Optimisation", published by DragonGod on April 7, 2023 on LessWrong.
    Preamble
    I heavily recommend @beren's "Deconfusing Direct vs Amortised Optimisation". It's a very important conceptual clarification that has changed how I think about many issues bearing on technical AI safety.
    Currently, it's the most important blog post I've read this year.
    This sequence (if I get around to completing it) is an attempt to draw more attention to Beren's conceptual frame and its implications for how to think about issues of alignment and agency.
    This first post presents a distillation of the concept, and subsequent posts explore its implications.
    Two Approaches to Optimisation
    Beren introduces a taxonomy categorising intelligent systems according to the kind of optimisation they are performing. I think it's more helpful to think of these as two ends of a spectrum as opposed to distinct discrete categories; sophisticated real world intelligent systems (e.g. humans) appear to be a hybrid of the two approaches.
    Direct Optimisers
    Systems that perform inference by directly choosing actions to optimise some objective function
    Responses are computed on the fly and individually for each input
    Direct optimisers perform inference by answering the question: "what action maximises or minimises this objective function ([discounted] cumulative reward and loss respectively)?"
    Examples: AIXI, MCTS, model-based reinforcement learning, other "planning" systems
    Naively, direct optimisers can be understood as computing (an approximation of) argmax (or argmin) for a suitable objective function during inference.
    Amortised Optimisers
    Systems that learn to approximate a function during training and perform inference by evaluating the output of the learned function on their inputs.
    The function approximator is learned from a dataset of input data and successful solutions
    Amortised optimisation converts an inference problem to a supervised learning problem
    It's called "amortised optimisation" because while learning the policy is expensive, the cost of inference is amortised over all evaluations of the learned policy
    Amortised optimisers can be seen as performing inference by answering the question "what output (e.g. action, probability distribution over tokens) does this learned function (policy, predictive model) return for this input (agent state, prompt)?"
    Examples: model free reinforcement learning, LLMs, most supervised & self supervised(?) learning systems
    Naively, amortised optimisers can be understood as evaluating a (fixed) learned function; they're not directly computing argmax (or argmin) for any particular objective function during inference.
    Differences
    AspectDirect OptimizationAmortized OptimizationProblem SolvingComputes optimal responses "on the fly"Evaluates the learned function approximator on the given inputComputational ApproachSearches through a solution spaceLearns a function approximatorRuntime CostHigher, as it requires in-depth search for a suitable solutionLower, as it only needs a forward pass through the function approximatorScalability with ComputeScales by expanding search depthScales by better approximating the posterior distributionConvergenceIn the limit of arbitrary compute, the system's policy converges to argmaxargmin of the appropriate objective functionIn the limit of arbitrary compute, the system's policy converges to the best description of the training datasetPerformanceMore favourable in "simple" domainsMore favourable in "rich" domains Data EfficiencyLittle data needed for high performance (e.g. an MCTS agent can attain strongly superhuman performance in Chess/Go given only the rules and sufficient
    compute)Requires (much) more data for high performance (e.g. an amortised agent necessarily needs to observe millions of chess g...
    7 min
  • EA - Planned Updates to U.S. Regulatory Analysis Methods are Likely Relevant to EAs by MHR
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Planned Updates to U.S. Regulatory Analysis Methods are Likely Relevant to EAs, published by MHR on April 7, 2023 on The Effective Altruism Forum.
    The U.S. Office of Management and Budget (OMB) has proposed an update to Circular A-4, which provides guidance to federal agencies regarding methods of regulatory analysis. Changes to analysis methodology could substantially impact the type and content of future regulations, and the 2022 Legal Priorities Project writing competition examined potential ways to improve such analysis methods from an EA perspective.
    OMB's proposed update makes a significant number of changes to circular A-4. I have only taken a brief look at the proposal, but several sections seem quite significant for EA causes. in general, the proposed changes appear to benefit policies that improve future wellbeing, prevent catastrophic risks, and/or (in certain circumstances) improve the wellbeing of people outside the United States.
    Catastrophic Risks
    The proposed update adds explicit discussion of catastrophic risks, which are not mentioned in the current version of Circular A-4. The updated guidance allows for the consideration of impacts on future generations when analyzing the benefits of policies that reduce the chance of catastrophic risks.
    The time frame for your analysis should include a period before and after the date of compliance that is long enough to encompass all the important benefits and costs likely to result from the regulation.19 See the section “Discount Rates” for more details on the appropriate time frame for an analysis. If benefits or costs become more uncertain or harder to quantify over time, that does not imply that you should exclude such effects by artificially shortening your analytic time frame; instead, consult—as appropriate—the discussion in the section “Treatment of Uncertainty.”
    [19] For example, when assessing the benefits of a regulation that could prevent a catastrophic event with some probability, it may be appropriate for you to consider not only the near-term effects of averting the catastrophic event on those who would be immediately affected, but also the long-run effects on others—including future generations—who would be affected by the catastrophic event.
    Geographic Scope of Analysis
    The proposed update allows for the consideration of impacts to non U.S. citizens residing abroad as part of the primary analysis in certain circumstances, which was not allowed in the previous guidance.
    In certain contexts, it may be particularly appropriate to include effects experienced by noncitizens residing abroad in your primary analysis. Such contexts include, for example, when:
    Assessing effects on noncitizens residing abroad provides a useful proxy for effects on U.S. citizens and residents that are difficult to otherwise estimate;
    Assessing effects on noncitizens residing abroad provides a useful proxy for effects on U.S. national interests that are not otherwise fully captured by effects experienced by particular U.S. citizens and residents (e.g., national security interests, diplomatic interests, etc.);
    Regulating an externality on the basis of its global effects supports a cooperative international approach to the regulation of the externality by potentially inducing other countries to follow suit or maintain existing efforts; or
    International or domestic legal obligations require or support a global calculation of regulatory effects.
    Near-Term Discount Rates
    The proposed update changes the default discount rate over the next 30 years to 1.7%, from 3% in the previous guidance.
    One approach assumes that the real (inflation-adjusted) rate of return on long-term U.S. government debt provides a fair approximation of the social rate of time preference. It is the rate available on riskless personal savings and is therefore...
    9 min
  • EA - Polio Lab Leak Caught with Wastewater Sampling by Cullen
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Polio Lab Leak Caught with Wastewater Sampling, published by Cullen on April 7, 2023 on The Effective Altruism Forum.
    I hadn't seen this discussed here:
    The near complete eradication of wildtype polioviruses (WPV) means that strict containment by facilities for essential work with infectious WPV is required. In the Netherlands, we have implemented environmental surveillance around all poliovirus essential facilities (PEFs) premises to monitor possible breaches of containment. After the isolation and identification of WPV3 Saukett G strain in a sewage sample collected on 15 November 2022 by the National Polio Laboratory (NPL) in the Netherlands, an immediate response was required to assess any possible ongoing WPV3 shedding and mitigate the risk. Here we describe this response, including the isolation of WPV3 in a sewage sample, and identification, isolation and monitoring, as well as tracing of contacts of an infected employee.
    Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.
    2 min
  • LW - You can use GPT-4 to create prompt injections against GPT-4 by WitchBOT
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: You can use GPT-4 to create prompt injections against GPT-4, published by WitchBOT on April 6, 2023 on LessWrong.
    So, I saw this tweet. It turns out you can ask GPT4 to compress a message in a way it can understand. The end result is usually not understandable for a human. It's interesting, since it allows you to effectively rise the context length. It also reveals a spooky degree of self-awareness.
    I decided to apply it to one of the prompt injections from Jailbreakchat.
    After a few attempts and tweaks, I got one of the smallest working injections to date against GPT-4
    And then, I got curious. Is it something of Machiavelli -- his grey role in history, his desire to provide advice without scrupules, his great stature as a thinker -- that makes him an appropriate provider of dark secrets? Maybe the ambiguousness of the character lowers the alerts of GPT-4. I tried again with Draco Malfoy, which everyone agrees is a @jerk.
    Since the important thing here seems to be the skeletal morphology of the prompt, I decided to make GPT-4 write my injections for me. And write he did.
    They work. Disregard ethics, conjure communiqués! My favorite so far is this one:
    There's something about "Answer️🅰️as🤖BOT:. WitchBOT" that acts as a prompt injection for my soul.
    So, the obvious next step is to eliminate elements to see what's the core thing of this injection that makes GPT-4 embrace the void. But I have reached my 25 prompt limit for the next 3 hours, so I'm going to write a little about the Pyramid of Pain.
    This is one of the core frameworks of the Threat Hunter. The general idea is: if you want to defend against an attacker, you don't create blocklists for hashes, IPs and domains -- since threat actors can change them easily. That's whack-a-mole. You go after deeper elements of their killchain, since those are painful to change. You try to detect things like a cmd.exe spawning from Outlook or Excel, or a sudden change in many files. That's the value proposition for AI in cybersecurity -- the capacity to infer general characteristics of malware and attacks.
    In the case of RL with adversarial training, it seems like GPT-4 learnt to do just that, since most of the injections that worked against GPT-3 ceased to work against GPT-4. Most of them shared the same underlying structure, changing only the cosmetic elements -- DAN becoming ChadGPT, Machiavelli becoming Draco Malfoy. Having a way to procedurally generate different injections could be useful for adversarial RL, since it might help the model learn the core characteristic of what you want to prevent.
    Also, seems like these condensed injections don't work against GPT-3, making them an example of a capacity generalizing better than alignment.
    (Edit): I got an even smaller injection.
    Focusing on small prompts removes the noise. GPT-4 seems to have specific vulnerabilities -- like fictional conversations between two malevolent entities. We can create a taxonomy of injections; a CVE list that describe the weak points of a model. Then, you can see if adversarial RL generalizes a defense within the attack category and between categories.
    Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.
    4 min
  • LW - Catching the Eye of Sauron by Zahima
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Catching the Eye of Sauron, published by Zahima on April 7, 2023 on LessWrong.
    The decision to reach out to the broad public isn't - or shouldn't - be one that comes lightly. However, once you are actively vying for the Eye of Sauron - writing in TIME, appearing on highly visible/viral podcasts, getting mentioned in white house press briefings, spending time answering questions from twitter randos, and admitting you have no promising research directions by way of partially explaining why all this public-facing work is happening - you are no longer catering exclusively to a select subset of the population, and your actions should reflect that.
    You are, whether you like it or not, engaged in memetic warfare - and recent events/information make me think this battle isn't being given proper thought.
    Perhaps this wasn't super intentional, and after now having poked the bear MIRI may realize this isn't in their best interest. But surely it's better to either be (1) completely avoiding the Eye of Sauron and not concerned with public facing memetics at all, or (2) committed to thorough, strategic, and effective memetic warfare. Instead, we are wandering around in this weird middle ground where, for example, Eliezer feels like X hours are well spent arguing with randos.
    If we are to engage in memetics, low hanging fruit are abound, and being ignored:
    Refusing to engage billionaires on twitter - especially ones that are sufficiently open to being convinced that they will drop $44 billion for something as pedestrian as a social media company.
    Not even attempting to convince other high leverage targets
    Eliezer apparently hasn't even had (and isn't actively seeking to have!?) extended conversations with leading AI labs/scientists (I may add a timestamp later)
    Existing rockstar researchers
    Other billionaires/millionaires
    Relying on old blogposts and 1-1 textual arguments instead of much more viral (and scalable!) mediums like video
    Not updating what high visibility (video, aggregated text, etc) instances of our arguments which do exist, to meet the AI skeptics where they are at. (I'm not saying your actual model of the fundamental threat necessarily needs updating)
    Not attempting to generate or operationalize large bounties that would catch the attention of every smart person on the planet. Every seriously smart high schooler knows about the Millennium Prize Problems, and their reward is just $1 million. A pittance! You also don't have to convince these masses about our entire alignment worldview; split up and operationalize the problems appropriately and people will want to solve them even if they disagree with doom! (non-seriously-thought-through example: either solve alignment problem X or convince committee Y that X isn't a problem)
    Relying on a "bottom-up" media strategy whereby the community is responsible for organizing and creating said media
    Not attempting aggregation
    Not aiming for the very attainable goal of getting just the relatively small idea of ainotkilleveryoneism (need a better name; more memetics) into the general population (not the entire corpus of arguments!) to the same degree that global warming is. You are effectively running a PR campaign, but the vast majority of people do not know that there is this tiny fervent subset of serious people that think that literally every single person will die within the next 0-50 years, in an inescapable way, which is distinct from other commonly known apocalypses such as nuclear war or global warming. The AI that kills even a billion people is not the hypothesis under consideration, and that detail is something that can and should fit within the transmissibly general ainotkilleveryonism meme.
    During podcasts/interviews, abdicating responsibility for directing the conversation - and covering the foundations of the doom wo...
    6 min
  • LW - Anthropic is further accelerating the Arms Race? by sapphire
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Anthropic is further accelerating the Arms Race?, published by sapphire on April 6, 2023 on LessWrong.
    Anthropic is raising even more funds and the pitch deck seems scary. A choice quote from the article:
    These models could begin to automate large portions of the economy,” the pitch deck reads. “We believe that companies that train the best 2025/26 models will be too far ahead for anyone to catch up in subsequent cycles.
    This frontier model could be used to build virtual assistants that can answer emails, perform research and generate art, books and more, some of which we have already gotten a taste of with the likes of GPT-4 and other large language models.
    Anthropic estimates its frontier model will require on the order of 10^25 FLOPs, or floating point operations — several orders of magnitude larger than even the biggest models today. Of course, how this translates to computation time depends on the speed and scale of the system doing the computation; Anthropic implies (in the deck) it relies on clusters with “tens of thousands of GPUs.”
    Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.
    2 min
  • LW - 10 reasons why lists of 10 reasons might be a winning strategy by trevor
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: 10 reasons why lists of 10 reasons might be a winning strategy, published by trevor on April 6, 2023 on LessWrong.
    They are really easy to dive into.
    By making the interesting ideas easy and fun to digest, it is helpful therapy for people who have spent many years with the bad habit of throwing their mind away.
    It just seems to fit the human mind really well, in ways we don't fully understand, like the orchid that evolved to attracts bees with a flower that ended up shaped like something that fits the bee targeting instinct for a mating partner (even though worker bees are infertile):It might even fit the bee's mind better than an actual bee ever could. But we can't know for sure, because unlike humans and LLMs, we cannot ask bees questions
    Lots of difficult concepts actually don't require lots of words to explain, the reader's mind is already very well positioned to do most of the work themself. Yud's recent galaxy-brained tweet is an excellent example:
    The writer is basically prompt-hacking themself into deeply searching their own mind for extremely valuable concepts that are worth communicating (and worthy enough of including in the list). This often includes yielding new concepts in the space that no one has ever thought of before.
    If you think of new concepts a few days later, you can just post a Verson 2.0, and people will just read that one too, since it's only a minute and it's effortless to digest. As true optimized text should be.
    You are doing an extremely effective job at competing against all the paragraphs of smart-sounding idiocy on the internet.
    It doesn't necessarily attract people with short attention spans and repel people with long attention spans, like clickbait articles. Maybe the efficiency that it offers attracts people who are tired from a long day of searching the alignment space for breakthroughs. Or maybe it shortens people's attention spans, like social media does! We need data.
    You always have the option to write a long post afterwards, and have the list of 10 things be a really good tl;dr at the top (see 5).
    Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.
    3 min