The Nonlinear Library

The Nonlinear Library

By The Nonlinear FundEducation
Download on the App Store

All The Nonlinear Library episodes

  • LW - AI x-risk, approximately ordered by embarassment by Alex Lawsen
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: AI x-risk, approximately ordered by embarassment, published by Alex Lawsen on April 12, 2023 on LessWrong.
    Advanced AI systems could lead to existential risks via several different pathways, some of which may not fit neatly into traditional risk forecasts. Many previous forecasts, for example the well known report by Joe Carlsmith, decompose a failure story into a conjunction of different claims, and in doing so risk missing some important dangers. ‘Safety’ and ‘Alignment’ are both now used by labs to refer to things which seem far enough from existential risk reduction that using the term ‘AI notkilleveryoneism’ instead is becoming increasingly popular among AI researchers who are particularly focused on existential risk.
    This post presents a series of scenarios that we must avoid, ranked by how embarrassing it would be if we failed to prevent them. Embarrassment here is clearly subjective, and somewhat unserious given the stakes, but I think it gestures reasonably well at a cluster of ideas which are important, and often missed by the kind of analysis which proceeds via weighing the incentives of multiple actors:
    Sometimes, easy problems still don’t get solved on the first try.
    An idea being obvious to nearly everyone does not mean nobody will miss it.
    When one person making a mistake is sufficient for a catastrophe, the relevant question is not whether the mistake will be obvious on average, but instead whether it will be obvious to everyone with the capacity to make it.
    The scenarios below are neither mutually exclusive nor collectively exhaustive, though I’m trying to cover the majority of scenarios which are directly tackled by making AI more likely to try to do what we want (and not do what we don’t). I’ve decided to include some kinds of misuse risk, despite this more typically being separated from misalignment risk, because in the current foundation model paradigm there is a clear way in which the developers of such models can directly reduce misuse risk via alignment research.
    Many of the risks below interact with each other in ways which are difficult to fully decompose, but my guess is that useful research directions will map relatively well onto reducing risk in at least one of the concrete scenarios below. I think people working on alignment might well want to have some scenario in mind for exactly what they are trying to prevent, and that this decomposition might also prove somewhat useful for risk modelling. I expect that some criticism of the sort of decomposition below, especially on LessWrong, will be along the lines of ‘it isn’t dignified to work on easy problems, ignoring the hard problems that you know will appear later, and then dying anyway when the hard problems show up’.
    I have some sympathy with this, though also a fairly big part of me that wants to respond with: ‘I dunno man, backing yourself to tightrope walk across the grand canyon having never practiced does seem like undignified suicide, but I still think it’d be even more embarrassing if you didn't secure one of the ends of the tightrope properly and died as soon as you took one step because checking your knots rather than staring at the drop seemed too like flinching away from the grimness of reality’. Ultimately though, this post isn’t asking people to solve the problems in order, it’s just trying to lay out which problems might emerge in a way that might help some people work out what they are trying to do. How worried people will feel by different scenarios will vary a bunch, and that's kind of the point.
    In a world where this piece turns out to be really valuable, my guess is that it's because it allows people to notice where they disagree, either with each other or with older versions of their own takes.
    Not all of the scenarios described below necessarily lead to complete huma...
    31 min
  • LW - LW Account Restricted: OK for me, but not sure about LessWrong by amelia
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: LW Account Restricted: OK for me, but not sure about LessWrong, published by amelia on April 12, 2023 on LessWrong.
    My LessWrong account has been restricted.
    I’m alright with this. In fact, the restriction has turned out to be advantageous to me for the following reasons:
    1-It inspired me to finally set up a Medium account and express my views on that platform. (/@ameliajones3.14)
    2-On the Medium platform, all my writing can be in one place. This even includes my short stories and poetry. And when it comes to writing on philosophical physics, I don’t have to worry that physics-oriented platforms will reject the writing for being too philosophical, while LW might reject it for not being philosophical enough. The writing can just exist for whatever it is worth in its own right.
    3-The Medium platform will presumably have a larger and more diverse audience.
    4-There is some comic potential in having my account restricted. I printed out the message about the restriction, and have it displayed on my desk. I don't know exactly why it's amusing. Maybe it has to do with the fact that I thought my writing was a healthy part of the dialog, when in actuality it must have really been annoying some people.
    Yet, while the restriction does not necessarily harm me personally, I do have some concerns for the future of LW. I’ll include these concerns in responses to the message about the restriction. The message is in italic. My thoughts are in bold.
    Hey amelia,
    We're generally tightening our moderation standards lately (see LW Team is adjusting moderation policy). This may feel like a significant shift in what the de-facto moderation standards. We're leaning towards the default expectation being more like new users can periodically submit comments or posts for approval,
    The “for approval” part concerns me a little. Does this mean any new user’s post will go through a moderator, and the moderator can decide to hide it from the rest of the LW audience, simply because they disagree with it? It seems this will stifle new ideas.
    with gradually escalating privileges of how often they can comment that depends on being actively upvoted.
    The upvoting part also concerns me. I think some readers use the karma vote based on whether they agree or disagree with the writer (and not whether the writer’s point was unusual, unique or well-argued). We know that groundbreaking ideas are often initially unpopular or deemed as "radical," so I worry that the need for popularity might limit the voicing of such ideas.
    You've been posting fairly frequently,
    I made 7 posts, including drafts, in 59 days. If you add in comments on my posts, others’ posts, and replies to comments, that brings us to 24. I’m thinking that, for the good of LW, you might want to have a consistent standard, and publicize it, because some newbies might not realize what is meant by “frequent.”
    [EDIT, 20230412 21:48, All but one of my comments were regarding my own posts. Somehow I feel obligated to reply w/my own comments to most comments on my posts, even if it's just to say "Thanks for your feedback."
    So if you don't count the comments on my own posts, then I've done 7 posts and 1 comment in 59 days. I can understand though if this is too much. You just might want to let future newbies know this.]
    and my subjective impression
    I won’t say anything to this, other than that it seems problematic.
    as well as voting patterns suggest most people aren't finding your comments sufficiently helpful.
    Yeah, my comments received a net karma of +34, with 11 agree votes, and 1 disagree vote. However, my posts did receive a net -4 karma.
    Generally, if I have an idea that I think everyone will agree with, or an idea that can be very easily argued, I don’t bother posting it--because such ideas don’t move the frontiers of philosophy forward. (I don’t reall...
    7 min
  • AF - AI x-risk, approximately ordered by embarrassment by Alex Lawsen
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: AI x-risk, approximately ordered by embarrassment, published by Alex Lawsen on April 12, 2023 on The AI Alignment Forum.
    Advanced AI systems could lead to existential risks via several different pathways, some of which may not fit neatly into traditional risk forecasts. Many previous forecasts, for example the well known report by Joe Carlsmith, decompose a failure story into a conjunction of different claims, and in doing so risk missing some important dangers. ‘Safety’ and ‘Alignment’ are both now used by labs to refer to things which seem far enough from existential risk reduction that using the term ‘AI notkilleveryoneism’ instead is becoming increasingly popular among AI researchers who are particularly focused on existential risk.
    This post presents a series of scenarios that we must avoid, ranked by how embarrassing it would be if we failed to prevent them. Embarrassment here is clearly subjective, and somewhat unserious given the stakes, but I think it gestures reasonably well at a cluster of ideas which are important, and often missed by the kind of analysis which proceeds via weighing the incentives of multiple actors:
    Sometimes, easy problems still don’t get solved on the first try.
    An idea being obvious to nearly everyone does not mean nobody will miss it.
    When one person making a mistake is sufficient for a catastrophe, the relevant question is not whether the mistake will be obvious on average, but instead whether it will be obvious to everyone with the capacity to make it.
    The scenarios below are neither mutually exclusive nor collectively exhaustive, though I’m trying to cover the majority of scenarios which are directly tackled by making AI more likely to try to do what we want (and not do what we don’t). I’ve decided to include some kinds of misuse risk, despite this more typically being separated from misalignment risk, because in the current foundation model paradigm there is a clear way in which the developers of such models can directly reduce misuse risk via alignment research.
    Many of the risks below interact with each other in ways which are difficult to fully decompose, but my guess is that useful research directions will map relatively well onto reducing risk in at least one of the concrete scenarios below. I think people working on alignment might well want to have some scenario in mind for exactly what they are trying to prevent, and that this decomposition might also prove somewhat useful for risk modelling. I expect that some criticism of the sort of decomposition below, especially on LessWrong, will be along the lines of ‘it isn’t dignified to work on easy problems, ignoring the hard problems that you know will appear later, and then dying anyway when the hard problems show up’.
    I have some sympathy with this, though also a fairly big part of me that wants to respond with: ‘I dunno man, backing yourself to tightrope walk across the grand canyon having never practiced does seem like undignified suicide, but I still think it’d be even more embarrassing if you didn't secure one of the ends of the tightrope properly and died as soon as you took one step because checking your knots rather than staring at the drop seemed too like flinching away from the grimness of reality’. Ultimately though, this post isn’t asking people to solve the problems in order, it’s just trying to lay out which problems might emerge in a way that might help some people work out what they are trying to do. How worried people will feel by different scenarios will vary a bunch, and that's kind of the point.
    In a world where this piece turns out to be really valuable, my guess is that it's because it allows people to notice where they disagree, either with each other or with older versions of their own takes.
    Not all of the scenarios described below necessarily lead to...
    31 min
  • LW - The UBI dystopia: a glimpse into the future via present-day abuses by Solenoid Entity
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: The UBI dystopia: a glimpse into the future via present-day abuses, published by Solenoid Entity on April 12, 2023 on LessWrong.
    Disclaimer: This does risk being a little culture-war-adjacent. This is more political than I normally see on LessWrong, but the spirit of it is intended to be less "Smash Capitalism" or "Become Ungovernable" and more "look at these things that are happening in Australia."
    Inspired by: Basic Income, Not Basic Jobs: Against Hijacking Utopia, SSC Gives A Graduation Speech.
    Summary
    Some bad things are happening today to people who depend on the government for money. This suggests that similar bad things could happen in the future if more people depend on the government for money.
    If you're an advocate for UBI, or it's a linchpin in your plan for how we'll live well post-AI, it's important to consider two connected worries carefully:
    Creating the scheme as truly universal (and keeping it that way) may be politically untenable in a democracy.
    In many likely worlds, some people will depend on this non-universal UBI for survival, leaving them vulnerable to coercive control and arbitrary punishment by the government (lawfully), government officials (unlawfully), even corrupt bureaucrats.
    Worries along the lines of "UBI could make us all serfs" and various techno-futuristic dystopian visions are already common enough. There's also a growing part of journalism/civil society/activism concerned with an industry that "farms the unemployed" — billing the government for services it ostensibly provides to poor people, while in fact spending their time on coercive control and a moralistic form of discipline. A "digital poorhouse" per Virginia Eubanks.
    I think the average LW reader has also probably heard worries about Government Issued Digital Currency (GIDC), which is certainly part of the concern here. Others have expressed worry about payment processors and banks being politically manipulated.
    But I think many people here are less familiar with arguments about problems within the social welfare system that already exist today. These problems are suggestive to me that, absent a change in the culture of these institutions, a UBI might lead to serious abuses.
    After listing some reasons it might be hard to actually make an actual universal policy, or 'make it an inalienable human right', etc., I then briefly survey some of the abuses of welfare recipients that have recently occurred in Australia, due to the ability of the government, public servants, and employees of private companies to 'turn off the tap' on their money.
    Two main problems
    1. You probably can't make it universal, and you probably can't keep it that way.
    The most compact way to pump this intuition is to imagine a caricature of a political debate, where a outrage-stoking populist politician is tearing shreds off a nervously stammering, principled, liberal, establishment centrist candidate.
    Are you seriously proposing to give access to our hard earned tax dollars to migrants? Refugees? Oh, you're not? How can we trust you on this? So you'll demand government-issued photo-ID and proof of citizenship for people signing up? Ok, what about convicted terrorists? What about people (accused of) traveling abroad to join terrorist organizations (accused by members of the government or security agencies without a trial)? What about 'terrorist sympathizers?'We won't be giving it to convicted felons, though, right? Oh we ARE? But not while they're in prison, right? And obviously child sex offenders are barred from the UBI for life, right?What about people who refuse to get vaccinated or vaccinate their children? What about people who take part in unpopular protests?What about draft-dodgers? Taxpayers would be paying for them to sit on their behinds while everyone else who gets conscripted does their duty a...
    12 min
  • AF - AXRP Episode 20 - ‘Reform’ AI Alignment with Scott Aaronson by DanielFilan
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: AXRP Episode 20 - ‘Reform’ AI Alignment with Scott Aaronson, published by DanielFilan on April 12, 2023 on The AI Alignment Forum.
    Google Podcasts link
    How should we scientifically think about the impact of AI on human civilization, and whether or not it will doom us all? In this episode, I speak with Scott Aaronson about his views on how to make progress in AI alignment, as well as his work on watermarking the output of language models, and how he moved from a background in quantum complexity theory to working on AI.
    Topics we discuss:
    ‘Reform’ AI alignment
    Epistemology of AI risk
    Immediate problems and existential risk
    Aligning deceitful AI
    Stories of AI doom
    Language models
    Democratic governance of AI
    What would change Scott’s mind
    Watermarking language model outputs
    Watermark key secrecy and backdoor insertion
    Scott’s transition to AI research
    Theoretical computer science and AI alignment
    AI alignment and formalizing philosophy
    How Scott finds AI research
    Following Scott’s research
    Daniel Filan: Hello, everyone. In this episode, I’ll be speaking with Scott Aaronson. Scott is a professor of computer science at UT Austin and he’s currently spending a year as a visiting scientist at OpenAI working on the theoretical foundations of AI safety. We’ll be talking about his view of the field, as well as the work he’s doing at OpenAI. For links to what we’re discussing, you can just check the description of this episode and you can read the transcript at axrp.net. Scott, welcome to AXRP.
    Scott Aaronson: Thank you. Good to be here.
    ‘Reform’ AI alignment
    Epistemology of AI risk
    Daniel Filan: So you recently wrote this blog post about something you called reform AI alignment: basically your take on AI alignment that’s somewhat different from what you see as a traditional view or something. Can you tell me a little bit about, do you see AI causing or being involved in a really important way in existential risk anytime soon, and if so, how?
    Scott Aaronson: Well, I guess it depends what you mean by soon. I am not a very good prognosticator. I feel like even in quantum computing theory, which is this tiny little part of the intellectual world where I’ve spent 25 years of my life, I can’t predict very well what’s going to be discovered a few years from now in that, and if I can’t even do that, then how much less can I predict what impacts AI is going to have on human civilization over the next century? Of course, I can try to play the Bayesian game, and I even will occasionally accept bets if I feel really strongly about something, but I’m also kind of a wuss.
    I’m a little bit risk-averse, and I like to tell people whenever they ask me ‘how soon will AI take over the world?’, or before that, it was more often, ‘how soon will we have a fault-tolerant quantum computer?’. They don’t want all the considerations and explanations that I can offer, they just want a number, and I like to tell them, “Look, if I were good at that kind of thing, I wouldn’t be a professor, would I? I would be an investor and I would be a multi-billionaire.” So I feel like probably, there are some people in the world who can just consistently see what is coming in decades and get it right. There are hedge funds that are consistently successful (not many), but I feel like the way that science has made progress for hundreds of years has not been to try to prognosticate the whole shape of the future.
    It’s been to look a little bit ahead, look at the problems that we can see right now that could actually be solved, and rather than predicting 10 steps ahead the future, you just try to create the next step ahead of the future and try to steer it in what looks like a good direction, and I feel like that is what I try to do as a scientist.
    And I’ve known the rationalist community, the AI risk community since. maybe no...
    1 hr 36 min
  • LW - Scaffolded LLMs as natural language computers by beren
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Scaffolded LLMs as natural language computers, published by beren on April 12, 2023 on LessWrong.
    Crossposted from my personal blog.
    Recently, LLM-based agents have been all the rage -- with projects like AutoGPT showing how easy it is to wrap an LLM in a simple agentic loop and prompt it to achieve real-world tasks. More generally, we can think about the class of 'scaffolded' LLM systems -- which wrap a programmatic scaffold around an LLM core and chain together a number of individual LLM calls to achieve some larger and more complex task than can be accomplished in a single prompt. The idea of scaffolded LLMs is not new, however with GPT4, we have potentially reached a threshold of reliability and instruction following capacity from the base LLM that agents and similar approaches have become viable at scale. What is missing, and urgent, however, is an understanding of the larger picture. Scaffolded LLMs are not just cool toys but actually the substrate of a new type of general-purpose natural language computer.
    Take a look at, for instance, the 'generative agent' architecture from a recent paper. The core of the architecture is an LLM that receives instructions and executes natural language tasks. There is a set of prompt templates that specify these tasks and the data for the LLM to operate on. There is a memory that stores a much larger context than can be fed to the LLM, and which can be read to and written from by the compute unit. In short, what has been built looks awfully like this:
    What we have essentially done here is reinvented the von-Neumann architecture and, what is more, we have reinvented the general purpose computer. This convergent evolution is not surprising -- the von-Neumann architecture is a very natural abstraction for designing computers. However, if what we have built is a computer, it is a very special sort of computer. Like a digital computer, it is fully general, but what it operates on is not bits, but text. We have a natural language computer which operates on units of natural language text to produce other, more processed, natural language texts. Like a digital computer, our natural language (NL) computer is theoretically fully general -- the operations of a Turing machine can be written as natural language -- and extremely useful: many systems in the real world, including humans, prefer to operate in natural language.
    Many tasks cannot be specified easily and precisely in computer code but can be described in a sentence or two of natural language.
    Armed with this analogy, let's push it as far as we can go and see where the implications take us.
    First, let's clarify the mappings between scaffolded LLM components and the hardware architecture of a digital computer. The LLM itself is clearly equivalent to the CPU. It is where the fundamental 'computation' in the system occurs. However, unlike the CPU, the units upon which it operates are tokens in the context window, not bits in registers. If the natural type signature of a CPU is bits -> bits, the natural type of the natural language processing unit (NLPU) is strings -> strings. The prompt and 'context' is directly equivalent to the RAM. This is the easily accessible memory that can be rapidly operated on by the CPU. Thirdly, there is the memory. In digital computers, there are explicit memory banks or 'disk' which have slow access memory. This is directly equivalent to the vector database memory of scaffolded LLMs.
    The heuristics we currently use (such as vector search over embeddings) for when to retrieve specific memory is equivalent to the memory controller firmware in digital computers which handles accesses for specific memory from the CPU. Finally, it is also necessary for the CPU to interact with the external world. In digital computers, this occurs through 'drivers' or special ...
    22 min
  • EA - Apply to >30 AI safety funders in one application with the Nonlinear Network by Drew Spartz
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Apply to >30 AI safety funders in one application with the Nonlinear Network, published by Drew Spartz on April 12, 2023 on The Effective Altruism Forum.
    Nonlinear spoke to dozens of earn-to-givers and a common sentiment was, "I want to fund good AI safety-related projects, but I don't know where to find them." At the same time, applicants don’t know how to find them either. And would-be applicants are often aware of just one or two funders - some think it’s “LTFF or bust” - causing many to give up before they’ve started, demoralized, because fundraising seems too hard.
    As a result, we’re trying an experiment to help folks get in front of donors and vice versa. In brief:
    Looking for funding?
    Why apply to just one funder when you can apply to dozens?
    If you've already applied for EA funding, simply paste your existing application. We’ll share it with relevant funders (~30 so far) in our network.
    You can apply if you’re still waiting to hear from other funders. This way, instead of having to awkwardly ask dozens of people and get rejected dozens of times (if you can even find the funders), you can just send in the application you already made.
    We’re also accepting non-technical projects relevant to AI safety (e.g. meta, forecasting, field-building, etc.)
    Application deadline: May 17, 2023.
    Looking for projects to fund?
    Apply to join the funding round by May 17, 2023. Soon after, we'll share access to a database of applications relevant to your interests (e.g. interpretability, moonshots, forecasting, field-building, novel research directions, etc).
    If you'd like to fund any projects, you can reach out to applicants directly, or we can help coordinate. This way, you avoid the awkwardness of directly rejecting applicants, and don’t get inundated by people trying to “sell” you.
    Inspiration for this project
    When the FTX crisis broke, we quickly spun up the Nonlinear Emergency Fund to help provide bridge grants to tide people over until the larger funders could step in.
    Instead of making all the funding decisions ourselves, we put out a call to other funders/earn-to-givers. Scott Alexander connected us with a few dozen funders who reached out to help, and we created a Slack to collaborate.
    We shared every application (that consented) in an Airtable with around 30 other donors. This led to a flurry of activity as funders investigated applications. They collaborated on diligence, and grants were made that otherwise wouldn’t have happened.
    Some funders, like Scott, after seeing our recommendations, preferred to delegate decisions to us, but others preferred to make their own decisions. Collectively, we rapidly deployed roughly $500,000 - far more than we initially expected.
    The biggest lesson we learned: openly sharing applications with funders was high leverage - possibly leading to four times as many people receiving funding and 10 times more donations than would have happened if we hadn’t shared.
    If you’ve been thinking about raising money for your project idea, we encourage you to do it now. Push through your imposter syndrome because, as Leopold Aschenbrenner said, nobody’s on the ball on AGI alignment.
    Another reason to apply: we’ve heard from EA funders that they don’t get enough applications, so you should have a low bar for applying - many fund over 50% of applications they receive (SFF, LTFF, EAIF).
    Since the Nonlinear Network is a diverse set of funders, you can apply for a grant size anywhere between single digit thousands to single digit millions.
    Note: We’re aware of many valid critiques of this idea, but we’re keeping this post short so we actually ship it. We’re starting with projects related to AI safety because our timelines are short, but if this is successful, we plan to expand to the other cause areas.
    Apply here.
    Reminder that you can listen to LessWrong ...
    4 min
  • LW - Repugnant levels of violins by Solenoid Entity
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Repugnant levels of violins, published by Solenoid Entity on April 12, 2023 on LessWrong.
    This is a rambling post about the repugnant conclusion, mass-manufacturing, metis and woo, and the fragility of expert discernment — through the lens of my experiences playing, repairing, and selling violins. (And eating discount sushi.)
    Epistemic status: Discussing utilitarianism-related issues that I'm poorly educated on. (Through the lens of violins, on which I fare much better.) Come for the interesting violin stuff, (don't stay for the philosophy unless you want to watch me repeatedly hitting myself on the head.)
    If you work in a violin shop, answering the question "how much should I spend on a violin for my child" is hard to do without revealing that you're a huge nerd.
    Playing violin is great. It's the most important instrument.
    Teaching it is a challenge. How do you help someone progress from playing like 'oh no' girl to, like, 'wow, she's only three?!' to a really, really good college graduate to being literally Jascha Heifetz?
    And more fundamentally: what makes Jascha Heifetz sound so damn good, and how did he get there?
    I think if you zoom out far enough, there are three interlinked factors at play in a violinist's ✨ sound ✨:
    What does their technical ability facilitate them doing, restrict them from doing, and what does it lead them towards doing?
    This comprises all of their movement-based competencies, the 'muscle-memory' they develop, the actual physical mechanics of how, through decades of intensive practice, their body has perfectly exapted the violin as the physical part of their aural proprioception.
    What are the bounds of their audiation? This is their musical creativity and imagination, the sound they hear in their head when they imagine playing. It's the mental and emotional component of their aural proprioception.
    Even though classical violinists are playing music written by someone else, there's creativity and emotion in their interpretation. Compare this, this, this, this, and this.
    Just like developing your own writing style, some of this you get from listening to other violinists play. Some of it you generate yourself through your own practice.
    Do they have a good instrument? Some violins sound better than others. The best violins in the world are prized (partly) for their unique, beautiful voices.
    Violins are usually made of wood (normally spruce and maple), and even now are almost all handmade to some extent (with more and more machine assistance for mass-produced ones). Very small differences between instruments make a big difference in sound quality.
    The craft of making really great violins is shrouded in centuries of metis and woo. There'll be a whole post on this, at some point, but suffice it to say that the people who make amazing violins, today, don't even themselves know precisely how they do it, and they contradict each other and themselves constantly. They sure do make them, though.
    Just take my word for it that a good violin matters, a lot.
    The above implies six one-way interactions between the factors. For this post, I just want you to take my word on three of them (keeping in mind they're part of this ever-pushing-pulling triangle):
    Having a good violin helps you to develop your audiation.
    Having a bad violin makes it hard to develop good technique.
    (And the obvious) Better violins just produce better sound for any given input.
    A good player can make a bad violin sound good, up to a point. At a certain point, a good player needs a good violin in order to improve their technique and audiation. Like how your skill as a race-car driver is sorta independent of the car you drive, BUT at a certain point you're going to need a Formula-1 car to compete at the top level, AND you won't even develop the skillset to drive a car like that if you've ne...
    22 min
  • LW - [Link] Sarah Constantin: "Why I am Not An AI Doomer" by lbThingrb
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: [Link] Sarah Constantin: "Why I am Not An AI Doomer", published by lbThingrb on April 12, 2023 on LessWrong.
    This is a good post from Sarah Constantin explaining why her expected timeline to agentic AGI is long (> 10 years). Topics discussed include whether LLMs and other current major research directions will endow AIs with adequate world models, causal inference, and goal robustness across ontological shifts.
    Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.
    1 min
  • EA - AIs accelerating AI research by Ajeya
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: AIs accelerating AI research, published by Ajeya on April 12, 2023 on The Effective Altruism Forum.
    Note: This post was crossposted from Planned Obsolescence by the Forum team, with the author's permission. The author may not see or respond to comments on this post.
    Researchers could potentially design the next generation of ML models more quickly by delegating some work to existing models, creating a feedback loop of ever-accelerating progress.
    The concept of an “intelligence explosion” has played an important role in discourse about advanced AI for decades. Early computer scientist I.J. Good described it like this in 1965:
    Let an ultraintelligent machine be defined as a machine that can far surpass all the intellectual activities of any man however clever. Since the design of machines is one of these intellectual activities, an ultraintelligent machine could design even better machines; there would then unquestionably be an ‘intelligence explosion,’ and the intelligence of man would be left far behind. Thus the first ultraintelligent machine is the last invention that man need ever make, provided that the machine is docile enough to tell us how to keep it under control.
    This presentation, like most other popular presentations of the intelligence explosion concept, focuses on what happens after we have a single AI system that can already do better at every task than any human (which Good calls an “ultraintelligent machine” above, and others have called “an artificial superintelligence”). It calls to mind an image of AI progress with two phases:
    In Phase 1, humans are doing all the AI research, and progress ramps up steadily. We can more or less predict the rate of future progress (i.e. how quickly AI systems will improve their capabilities) by extrapolating from past rates of progress.[1]
    Eventually humans succeed at building an artificial superintelligence (or ASI), leading to Phase 2. In Phase 2, this ASI is doing all of the AI research by itself. All of a sudden, progress in AI capabilities is no longer bottlenecked by slow human researchers, and an intelligence explosion is kicked off. The rate of progress in AI research goes up sharply — perhaps years of progress is compressed into days or weeks.
    But I think this picture is probably too all-or-nothing. Today’s large language models (LLMs) like GPT-4 are not (yet) capable of completely taking over AI research by themselves — but they are able to write code, come up with ideas for ML experiments, and help troubleshoot bugs and other issues. Anecdotally, several ML researchers I know are starting to delegate simple tasks that come up in their research to these LLMs, and they say that makes them meaningfully more productive. (When chatGPT went down for 6 hours, I know of one ML researcher who postponed their coding tasks for 6 hours and worked on other things in the meantime.[2])
    If this holds true more broadly, researchers could potentially design and train the next generation of ML models more quickly and easily by delegating to existing LLMs.[3] This calls to mind a more continuous “intelligence explosion” that begins before we have any single artificial superintelligence:
    Currently, human researchers collectively are responsible for almost all of the progress in AI research, but are starting to delegate a small fraction of the work to large language models. This makes it somewhat easier to design and train the next generation of models.
    The next generation is able to handle harder tasks and more different types of tasks, so human researchers delegate more of their work to them. This makes it significantly easier to train the generation after that. Using models gives a much bigger boost than it did the last time around.
    Each round of this process makes the whole field move faster and faster. In each round, human...
    6 min