The Nonlinear Library

The Nonlinear Library

By The Nonlinear FundEducation
Download on the App Store

All The Nonlinear Library episodes

  • EA - Pausing AI Developments Isn't Enough. We Need to Shut it All Down by EliezerYudkowsky
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Pausing AI Developments Isn't Enough. We Need to Shut it All Down, published by EliezerYudkowsky on April 9, 2023 on The Effective Altruism Forum.
    Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.
    1 min
  • EA - EA & “The correct response to uncertainty is not half-speed” by Lizka
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: EA & “The correct response to uncertainty is not half-speed”, published by Lizka on April 9, 2023 on The Effective Altruism Forum.
    TL;DR: When we're unsure about what to do, we sometimes naively take the "average" of the obvious options — despite the fact that a different strategy is often better. For example, if you're not sure if you're in the right job, continuing to do your job as before but with less energy ("going half-speed") is probably not the best approach. Note, however, that sometimes speed itself is the problem, in which case "half-speed" can be totally reasonable — I discuss this and some other considerations below.
    I've referenced this phenomenon in some conversations recently, so I'm sharing a relevant post from 2016 — The correct response to uncertainty is not half-speed — and sketching out some examples I've seen.
    The correct response to uncertainty is not half-speed
    The central example in the post is a time when the author was driving along a long stretch of road and started wondering if she’d passed her hotel. So she continued at half-speed, trying to decide if she should keep going or turn around. After a while, she realized:
    If the hotel was ahead of me, I'd get there fastest if I kept going 60mph. And if the hotel was behind me, I'd get there fastest by heading at 60 miles per hour in the other direction. And if I wasn't going to turn around yet -- if my best bet given the uncertainty was to check N more miles of highway first, before I turned around -- then, again, I'd get there fastest by choosing a value of N, speeding along at 60 miles per hour until my odometer said I'd gone N miles, and then turning around and heading at 60 miles per hour in the opposite direction.
    Either way, fullspeed was best. My mind had been naively averaging two courses of action -- the thought was something like: "maybe I should go forward, and maybe I should go backward. So, since I'm uncertain, I should go forward at half-speed!" But averages don't actually work that way.[1]
    [...] [From a comment] Often a person should hedge bets in some fashion, or should take some action under uncertainty that is different from the action one would take if one were certain of model 1 or of model 2. The point is that "hedging" or "acting under uncertainty" in this way is different in many particulars from the sort of "kind of working" that people often end up accidentally doing, from a naiver sort of average. Often it e.g. involves running info-gathering tests at full speed, one after another.
    Opinions expressed here are mine, not my employer’s, not the Forum’s, etc. I wrote this fast, so it’s definitely not an exhaustive list of examples or considerations and is probably wrong in important places.
    Assorted links that seem related to the post
    False compromises & the fallacy of the middle / the argument to moderation / the middle ground fallacy (example link - I don’t know what the right term here is, or if there’s an excellent explanation)
    Dive in and the explanation of “split and commit” here
    Where I’ve seen the “half-speed” phenomenon recently
    I think that I’ve seen multiple instances of each of these in the past few months. I’m not sure that all of these directly stem from the phenomenon described above — there might be better descriptions for what’s going on — but they seem quite related.
    Jobs. Someone is unsure if their role is a good fit (or if it's the most impactful option, etc.) for them. So they continue working in it, but put less energy into it.
    What you might do instead: set aside time to evaluate your options and fit (and switch jobs based on that), consider setting up some tests, see if you can change or improve things in your current job (talk to your manager, etc.), decide that it’s a bad time to think about this and that you'll re-evaluate at a set time (schedu...
    8 min
  • LW - A decade of lurking, a month of posting by Max H
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: A decade of lurking, a month of posting, published by Max H on April 9, 2023 on LessWrong.
    This post is a look back on my first month or so as an active contributor on LessWrong, after lurking for over a decade. My experience so far has been overwhelmingly positive, and one purpose of this post is to encourage other lurkers to do the same.
    The reason I decided to start posting, in a nutshell:
    For the last 10 years or so, I've been following Eliezer's public writing and nodding along in silent agreement with just about everything he says.
    I mostly didn't feel like I had much to contribute to the discussion, at least not enough to overcome the activation energy required to post, which for me seems to be pretty high.
    However, over the last few years and especially the last few months, I've grown increasingly alarmed and disappointed by the number of highly-upvoted and well-received posts on AI, alignment, and the nature of intelligent systems, which seem fundamentally confused about certain things. I think (what I perceive as) these misunderstandings and confusion are especially prominent in posts which reject all or part of the Yudkowskian view of intelligence and alignment.
    I notice Eliezer's own views seem to be on the outs with some fraction of prominent posters these days. One hypothesis for this is that Eliezer is actually wrong about a lot of things, and that people are right to treat his ideas with skepticism.
    Reading posts and comments from both Eliezer and his skeptics though, I find this hypothesis unconvincing. Eliezer may sometimes be wrong about important things, but his critics don't seem to be making a very strong case.(I realize the paragraphs above are potentially controversial. My intent is not to be inflammatory or to attack anyone. My goal in this post is simply to be direct about my own beliefs, without getting too much into the weeds about why I hold them.)
    My first few posts and comments have been an attempt to articulate my own understanding of some concepts in AI and alignment which I perceive as widely misunderstood. My goal is to build a foundation from which to poke and prod at some of the Eliezer-skeptical ideas, to see if I have a knack for explaining where others have failed. Or, alternatively, to see if I am the one missing something fundamental, which becomes apparent through more active engagement.
    Overview of my recent posts
    This section is an overview of my posts so far, ranked by which ones I think are the most worth reading.
    Most of my posts assume some background familiarity, if not agreement with, Yudkowskian ideas about AI and alignment. This makes them less accessible as "101 explanations", but allows me to wade a bit deeper into the weeds without getting bogged down in long introductions.
    Steering systems
    My longest and most recent post, and the one that I am most proud of.
    As of publishing this piece, it has gotten a handful of strong and weak upvotes, and zero downvotes. I'm not sure if this indicates it dropped off the front page before it could get more engagement, or if it was simply not interesting enough per-word for most people in its target audience to read to the end and vote on it.
    The main intuition I wanted to convey in this post is how powerful systems might be constructed in the near future, by composing "non-agentic" foundation models in relatively simple ways. And further, that there are ways this leads to extreme danger / failure even before we get to the point of having to worry about even more powerful systems reflecting, deceiving, power-seeking, or exhibiting other more exotic examples of POUDA.I'll highlight one quote from this piece, which I think is a nice distillation of a key insight for making accurate predictions about how the immediate future of LLMs is likely to play out:
    Training GPT-4 was the w...
    9 min
  • AF - Expanding the domain of discourse reveals structure already there but hidden by Tsvi Benson-Tilsen
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Expanding the domain of discourse reveals structure already there but hidden, published by Tsvi Benson-Tilsen on April 9, 2023 on The AI Alignment Forum.
    [Metadata: crossposted from. First completed May 15, 2022. This essay is more like research notes than exposition, so context may be missing, the use of terms may change across essays, and the text might be revised later; only the versions at tsvibt.blogspot.com are definitely up to date.]
    To understand a complex system like a mind or an ecosystem, we have to understand a tangled web of objects, features, processes, relations, correlations, clusters, constraints, causes, and so on. It helps to find underlying explanations and generating process, and to find deep reasons that explain why many relationships are the way they are. To find those underlying explanations, it helps to know relationships between relationships: which relationships cause, explain, or constrain other relationships. When staring at an assembly of relationships and asking which relationships explain other relationships, it can seem like we're at a loss for where to go; sometimes there are clearly relationships, but there's no clear way to extract any further order, because there's no basis on which to say that one relationship explains another--the relationships are just there, and that's it.
    By way of example/analogy, suppose we look at a list of facts like [3×4=12, 12/4=3, 4=12/3, 12=4×3], and ask, which facts are more foundational to which other facts? Which facts explain, ground, cause, or constitute the essence of, which other facts? Is 12/4=3 because 12=4×3, or vice versa? Is the evenness of 12 due to the evenness of 4? Or due to the evenness of 6? What if there's no answer, and all we can say is that all these facts stand together as related by derivations and proofs, with no priority given to any of them over any other?
    Another example: take a smooth closed curve C in the plane. These conditions are equivalent:
    There is a point p such that all points on C are equidistant from p.
    C has constant curvature.
    (If C bounds a disc) C bounds the maximum area that's possible to bound with a curve of the same length as C.
    The group of isometries of the plane that maps C to itself is a nontrivial connected compact topological group.
    These conditions all define a circle in the plane. If we're asked "Why does a locus of equidistant points have constant curvature?", we can answer with a proof. But if we're asked, "Is a circle constant curvature because it maximizes area, or does it maximize area because it has constant curvature?", we might have nothing to say. If we ask "Why does a wheel roll?", should we be satisfied with the explanation: "To roll, it's necessary that it admits a nontrivial connected compact group of isometries; it admits these isometries because it has constant curvature; it has constant curvature because the points on its rim are equidistant from the axle."? What if instead the explanation ended with "...because its rim bounds the maximum area boundable by any rim of the same length."? What if we meant to ask why the rolling keeps the axle level, not why the rolling is a continuous motion?
    In some cases, deriving the presence of traits in a species from other present traits might be like this: the traits all imply each other, so the implications can be understood, but none of the traits underlie the other traits. This situation is not so implausible. For one thing, features of species that we name might be related to each other less like two objects relate to each other (causal dynamics, interaction) and more like logical properties (implication, equivalence). E.g. the feature "totally consumes its prey" pretty much logically implies "causes its prey to die". For another thing, many systems (especially biological and mental ones) h...
    10 min
  • EA - SERI MATS - Summer 2023 Cohort by Aris Richardson
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: SERI MATS - Summer 2023 Cohort, published by Aris Richardson on April 8, 2023 on The Effective Altruism Forum.
    Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.
    1 min
  • LW - Agentized LLMs will change the alignment landscape by Seth Herd
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Agentized LLMs will change the alignment landscape, published by Seth Herd on April 9, 2023 on LessWrong.
    Epistemic status: head spinning, suddenly unsure of everything in alignment. And unsure of these predictions.
    I'm following the suggestions in 10 reasons why lists of 10 reasons might be a winning strategy in order to get this out quickly (reason 10 will blow your mind!). I'm hoping to prompt some discussion, rather than try to do the definitive writeup on this topic when this technique was introduced so recently.
    Ten reasons why agentized LLMs will change the alignment landscape:
    Agentized LLMs like Auto-GPT and Baby AGI may fan the sparks of AGI in GPT-4 into a fire. These techniques use an LLM as a central cognitive engine, within a recursive loop of breaking a task goal into subtasks, working on those subtasks (including calling other software), and using the LLM to prioritize subtasks and decide when they're adequatly well done. They recursively check whether they're making progress on their top-level goal.
    While it remains to be seen what these systems can actually accomplish, I think it's very likely that they will dramatically enhance the effective intelligence of the core LLM. I think this type of recursivity and breaking problems into separate cognitive tasks is central to human intelligence. This technique adds several key aspects of human cognition; executive function; reflective, recursive thought; and episodic memory for tasks, despite using non-brainlike implementations. To be fair, the existing implementations seem pretty limited and error-prone. But they were implemented in days. So this is a prediction of near-future progress, not a report on amazing new capabilities.
    This approach appears to be easier than I'd thought. I've been expecting this type of self-prompting to imitate the advantages of human thought, but I didn't expect the cognitive capacities of GPT-4 to make it so easy to do useful multi-step thinking and planning. The ease of initial implementation (something like 3 days, with all of the code also written by GPT-4 for baby AGI) implies that improvements may also be easier than we would have guessed.
    Integration with HuggingGPT and similar approaches can provide these cognitive loops with more cognitive capacities. This integration was also easier than I'd have guessed, with GPT-4 learning from a handful (e.g., 40) of examples how to use other software tools. Those tools will include both sensory capacities, with vision models and other sensory models of various types, and the equivalent of a variety of output capabilities.
    Integration of recursive LLM self-improvement like "Reflexion" can utilize these cognitive loops to make the core model better at a variety of tasks.
    Easily agentized LLMs is terrible news for capabilities. I think we'll have an internet full of LLM-bots "thinking" up and doing stuff within a year.
    This is absolutely bone-chilling for the urgency of the alignment and coordination problems. Some clever chucklehead already created ChaosGPT, an instance of Auto-GPT given the goal to destroy humanity and create chaos. You are literally reading the thoughts of something thinking about how to kill you. It's too stupid to get very far, but it will get smarter with every LLM improvement, and every improvement to the recursive self-prompting wrapper programs. This gave me my very first visceral fear of AGI destroying us. I recommend it, unless you're already plenty viscerally freaked out.
    Watching agents think is going to shift public opinion. We should be ready for more AI scares and changing public beliefs. I have no idea how this is going to play out in the political sphere, but we need to figure this out to have a shot at successful alignment, because
    We will be in a multilateral AGI world. Anyone can spawn a dumb AGI...
    6 min
  • LW - The benevolence of the butcher by dr s
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: The benevolence of the butcher, published by dr s on April 8, 2023 on LessWrong.
    A few days ago I published this post on the risks of powerful transformative AGI (by which I meant AGI that takes off fast and pretty much rules the world in no time), even if aligned. Among the comments there was one by Paul Christiano which I think was very interesting, but also focused on a different scenario, one of slower take off in which AGI stays with us as a regular part of our economy for a bit longer. This post is an elaboration of the answer I gave there, because it brought to my mind a different kind of risk.
    It's a common in rebuttals to pessimism about AGI to compare it to other past technologies, and how eventually they all ended up boosting productivity and thus raising the average human welfare in the long run (though I would also suggest that we do not completely ignore the short run: after all, we're most likely to live through it. I don't just want the destination to be nice, I want the trip to be reasonably safe!). I worry however that carrying this way of thinking to AGI might be a case of a critical error of extrapolation - applying knowledge that worked in a certain domain to a different domain in which some very critical assumptions on which that knowledge relied aren't true any more.
    Specifically, when one thinks of any technology developed during or after the industrial revolution, one thinks of a capitalist, free-market economy. In such an economy, there are people who mostly own the capital (the land, the factories, and any other productive infrastructure) and there are people who mostly work for the former, putting the capital to use so it can actually produce wealth. The capital acts as a force multiplier which makes the labour of a single human be worth tens, hundreds, thousands of times what it would have been in a pre-industrial era; but ultimately, it is still a multiplier. A thousand times zero is zero: the worker is still an essential ingredient.
    The question of how this surplus in productivity is to be split fairly between interests that reward the owner of the capital for the risk they take and salary for the worker who actually put in the labour has been a... somewhat contentious issue throughout the last two centuries, but all capitalist economies exist in some kind of equilibrium that is satisfactory enough to at least not let the social fabric straight up unravel itself. Mostly, both groups need each other, and not just that; workers being also consumers, their participation in the economy is vital to make the huge gains of industrial productivity even worth anything at all, and them having higher living standards (including, crucially, good literacy and education) is essential to their ability to actually contribute to systems that have been growing more and more cognitively complex by the year.
    These forces are an essential part of what propelled our society to its current degree of general prosperity through the 19th, 20th and now 21st century.
    But this miracle is not born of disinterested generosity. Rather, it has been achieved through a lot of strife, and is an equilibrium between different forms of self-interest. The whole sleight of (invisible) hand with which free market capitalism makes people richer is that making other people well off is the best way to make yourself very well off. To quote Adam Smith himself, "it is not from the benevolence of the butcher, the brewer, or the baker that we expect our dinner, but from their regard to their own self-interest". In AI language, one can think of a capitalist society as a sort of collective superintelligence whose terminal goal is everyone's selfish interest to personally live better, roughly weighed by how much capital (and thus steering power over the economy) they control, but structur...
    10 min
  • LW - Guidelines for productive discussions by ambigram
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Guidelines for productive discussions, published by ambigram on April 8, 2023 on LessWrong.
    I enjoyed reading Duncan_Sabien's post, Basics of Rationalist Discourse, as well as the response posts and comments. The guidelines probably wouldn't have made sense to my beginner-self though, so I wanted to try creating a list that is more targeted at beginners. In that sense, this is more of a complement to the other pieces than an alternate take.
    I am not really sure if I've captured everything important, and I don't really like the "Other notes" section because it feels randomly put together. What I'm most curious about is whether taking this approach can help address some of the criticisms of the original post. I've added some of my thinking behind this piece to the appendix.
    On a site like Less Wrong, we want to promote discussions and conversations that help members improve at rationality. We want participants and the audience to walk away from discussions with a clearer, more accurate understanding of the world, one which enables them to act more effectively.
    Here are some guidelines for more fruitful conversations:
    1) Seek to be Less Wrong instead of More Right
    It feels good to be right, to show that we are correct and others are wrong. However, this can be counter-productive. It encourages us to find ways to prove we are right instead of figuring out whether or not our beliefs are right in the first place. This makes it much less likely for us to arrive at the truth.
    Let's instead focus on how we can be Less Wrong:
    When you hear something that doesn't make sense, get curious instead of dismissive. After all, we each experience a very small slice of reality. If someone has a seemingly nonsensical opinion, maybe it's because we're missing out a perspective.
    Steelman: When someone presents a criticism that you believe is flawed, see if you can improve the criticism and use it to identify a gap in your thinking.
    Ask yourself, if you were wrong, how would you find out? If nothing can change your mind, further discussion would be pointless.
    2) First, check your understanding
    Communication is hard. Words mean different things to different people, and we add our own interpretations to things we hear. Make sure you are addressing what the other person is saying, and not just talking past each other. Sometimes that can mean having to do some background reading.
    Paraphrase to check if you've understood the points. If you think that there is something being implied, make it explicit.
    Check if what you are reading or hearing is as you expect, and take notice when you are confused
    Ask clarifying questions if there's anything that confuses you, or if there's something that seems obviously wrong. There's usually reasons why people believe the things they believe, even for beliefs that seem nonsensical to you. Or maybe they just didn't express themselves very well, or maybe you misunderstood!
    3) Say what you mean and mean what you say
    It's usually okay to be imprecise in our daily conversations, but when we hold ourselves to higher standards, we get to practice thinking more clearly.
    Say what you mean:
    Sarcasm or exaggeration can work well in face-to-face conversations or where people have a shared context. It doesn't work as well online, where it's mostly text and the people come from different cultures. It can also make communication harder, e.g. if "everyone knows X" can be used an exaggeration, how do you say it if you mean the statement literally?
    Imprecise statements like "A little bit more" works fine if you are talking about a glass of water at dinner but not if you are measuring out medicine for a patient. Being precise is a skill that needs to be learned and practiced, and is important for clear thinking. Let's try to be more precise than we might in our daily conversation...
    9 min
  • LW - The surprising parameter efficiency of vision models by beren
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: The surprising parameter efficiency of vision models, published by beren on April 8, 2023 on LessWrong.
    Crossposted from my personal blog.
    Epistemic status: This is a short post meant to highlight something I do not yet understand and therefore a potential issue with my models. I would also be interested to hear if anybody else has a good model of this.
    Why do vision (and audio) models work so well despite being so small? State of the art models like stable diffusion and midjourney work exceptionally well, generating near-photorealistic art and images and give users a fair degree of controllability over their generations. I would estimate with a fair degree of confidence that the capabilities of these models probably surpass the mental imagery abilities of almost all humans (they definitely surpass mine and a number of people I have talked to). However, these models are also super small in terms of parameters. The original stable diffusion is only 890M parameters.
    In terms of dataset size, image models are at a rough equality with humans. The stable diffusion dataset is 2 billion images. Assuming that you see 10 images per second every second you are awake and that you are awake 18 hours a day, you can observe 230 million images per year and so get the same data input as stable diffusion after 10 years. Of course, the images you see are much more redundant and we made some highly aggressive assumptions but after a human lifetime being in the same OOM as a SOTA image model is not insane. On the other hand, the hundreds of billions to trillions of tokens fed to LLMs is orders of magnitude beyond what humans could ever experience.
    A similar surprising smallness occurs in audio models. OpenAI's Whisper can do almost flawless audio transcription (including multilingual translation!) with just 1.6B parameters.
    Let's contrast this to the brain. Previously, I estimated that we should expect the visual cortex to have on the order of 100B parameters, if not more. The auditory cortex should be of roughly the same order of magnitude, but slightly smaller than the visual cortex. That is two orders of magnitude larger than state of the art DL models in these modalities.
    This contrasts with state of the art language models which appear to be approximately equal to the brain in parameter count and abilities. Small (1-10B) language models are clearly inferior to the brain at producing valid text and completions as well as standard question answering and factual recall tasks. Human parity in factual knowledge is reached somewhere between GPT-2 and GPT-3. Human language abilities are still not entirely surpassed with GPT-3 (175B parameters) or GPT-4 (presumably significantly larger). This puts large language models within approximately the same order of magnitude as the human linguistic cortex.
    What could be the reasons for this discrepancy? Off the top of my head I can think of a number which are below (and ranked by rough intuitive plausibility), and it would be interesting to try to investigate these further. Also, if anybody has ideas or evidence either way please send me a message.
    1.) The visual cortex vs image models is not a fair comparison. The brain does lots of stuff image generation models can't do such as parse and render very complex visual scenes, deals with saccades and having two eyes, and, crucially, handle video data and moving stimuli. We haven't fully cracked video yet and it is plausible that to do so existing vision models require an OOM or two more of scale.
    2.) There are specific inefficiencies in the brain's processing of images that image models skip which do not apply to language models. One very obvious example of this is convolutions. While CNNs have convolutional filters which are applied to all tiles of the image individually, the brain cannot do this an...
    8 min
  • EA - GPTs are Predictors, not Imitators by EliezerYudkowsky
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: GPTs are Predictors, not Imitators, published by EliezerYudkowsky on April 8, 2023 on The Effective Altruism Forum.
    Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.
    1 min