The Nonlinear Library

The Nonlinear Library

By The Nonlinear FundEducation
Download on the App Store

All The Nonlinear Library episodes

  • AF - QAPR 5: grokking is maybe not that big a deal? by Quintin Pope
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: QAPR 5: grokking is maybe not that big a deal?, published by Quintin Pope on July 23, 2023 on The AI Alignment Forum.
    [Thanks to support from Cavendish Labs and a Lightspeed grant, .I've been able to restart the Quintin's Alignment Papers Roundup sequence.]
    Introduction
    Grokking refers to an observation by Power et al. (below) that models trained on simple modular arithmetic tasks would first overfit to their training data and achieve nearly perfect training loss, but that training well past the point of overfitting would eventually cause the models to generalize to unseen test data. The rest of this post discusses a number of recent papers on grokking.
    Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasets
    In this paper we propose to study generalization of neural networks on small algorithmically generated datasets. In this setting, questions about data efficiency, memorization, generalization, and speed of learning can be studied in great detail. In some situations we show that neural networks learn through a process of "grokking" a pattern in the data, improving generalization performance from random chance level to perfect generalization, and that this improvement in generalization can happen well past the point of overfitting. We also study generalization as a function of dataset size and find that smaller datasets require increasing amounts of optimization for generalization. We argue that these datasets provide a fertile ground for studying a poorly understood aspect of deep learning: generalization of overparametrized neural networks beyond memorization of the finite training dataset.
    My opinion:
    When I first read this paper, I was very excited. It seemed like a pared-down / "minimal" example that could let us study the underlying mechanism behind neural network generalization. You can read more of my initial opinion on grokking in the post Hypothesis: gradient descent prefers general circuits.
    I now think I was way too excited about this paper, that grokking is probably a not-particularly-important optimization artifact, and that grokking is no more connected to the "core" of deep learning generalization than, say, the fact that it's possible for deep learning to generalize from an MNIST training set to the testing set.
    I also think that using the word "grokking" was anthropomorphizing and potentially misleading (like calling the adaptive information routing component of a transformer model its "attention"). Evocative names risk letting the connotations of the name filter into the analysis of the object being named. E.g.,
    "Grokking" brings connotations of sudden realization, despite the fact that the grokking phase in the above plot starts within the first ~5% - 20% of the training process, though it appears much more abrupt due to the use of a base 10 logarithmic scale on the x-axis.
    "Grokking" also brings connotations of insight, realization or improvement relative to some previously confused baseline. This leads to the impression that things which grok are better than things which don't.
    Humans often use the word "grokking" to mean deeply understanding complex domains that actually matter in the real world. Using the same word in an ML context suggests that ML grokking is relevant to whatever mechanisms might let an ML system deeply understand complex domains that actually matter in the real world.
    I've heard several people say things like:
    Studying grokking could significantly advance ML capabilities, if doing so were to lead to a deeper understanding of the mechanisms underlying generalization in ML.
    Training long enough could eventually result in grokking occurring in ML domains of actual relevance, such as language, and thereby lead to sudden capabilities gains or break alignment properties.
    Grokking is an example of how thinking l...
    17 min
  • LW - Autogynephilia discourse is so absurdly bad on all sides by tailcalled
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Autogynephilia discourse is so absurdly bad on all sides, published by tailcalled on July 23, 2023 on LessWrong.
    In Blanchard's Dangerous Idea and the Plight of the Lucid Crossdreamer, Zack criticizes the rationalist community and the trans community for various things, including denial of autogynephilia as a cause of transness:
    I could go on ... but do I need to? After having seen enough of these laughable denials of autogynephilia, the main question in my mind has become less, "Is the two-type androphilic/autogynephilic taxonomy of MtF transsexuality approximately true?" (answer: yes, obviously) and more, "How dumb do you (proponents of gender-identity theories) think we (the general public) are?" (answer: very, but correctly).
    In some ways this is not a completely unreasonable critique, but I think it has a lot less bite once one realizes just how dysfunctional the discourse on this subject is, even among the ideology that Zack endorses as a replacement.
    A microcosm of the dysfunction can be seen in Michael Bailey and Kevin Hsu's study, How Autogynephilic Are Natal Females? (pdf). They are responding to a debate where Charles Moser, Scott Alexander, Julia Serano, and Jaimie Veale argue that cis women are often autogynephilic too, and that this suggests that autogynephilia in trans women is not so significant.
    In response, Michael Bailey (the effective leader of autogynephilia theory) and Kevin Hsu recruit samples of highly active members of online erotic AGP communities, and find that these samples are on average much more AGP than cis women, and therefore conclude that AGP in cis women is dubious.
    Substituting a sample of highly active members of online erotic AGP communities for trans women is an insanely biased methodology! And when called out on it (pdf), Michael Bailey essentially goes "well the debate isn't about trans women, it's about AGPs, so it's totally appropriate" (pdf), which again is just an insane response when very obviously the debate is about trans women! All the people Bailey is responding to were clearly talking about trans women in their texts!
    One would think that people would point out these problems so the study would be appropriately rejected. I have tried to do so, and some trans activists have tried to do so, but people on the Blanchardian side have generally rejected it.
    Now here's a thought experiment for you: If people cannot converge to the truth on objectively observable facts about who has said what that can be seen by reading the debates, how can they ever be expected to converge to the truth about complex psychological causal and distributional inference questions? Especially when the objectively observable facts concern what psychological causal and distributional inference questions are under dispute?
    Even if the specific point of AGP in cis women doesn't move you much (I don't think it should), this dysfunctional discourse might make you tempted to infer that Blanchardians do a lot of other shenanigans to make their theories look better than they really are. And I think you would be right to make that inference, because I have a lot of points of critique on my gender blog that go unaddressed. But my critiques aren't the core point I'm raising here, rather I'm pointing out that people have good reasons to be exhausted with autogynephilia theorists.
    See Transvestism vs gender dysphoria vs . and Michael Bailey doubles down on his lies 🤣 for in-depth analysis. The latter post has direct quotes.
    I am so confused about AGP in cis women that I am essentially agnostic on the question. I think it is a question that is relatively independent of other questions of interest, and I don't think we have any satisfactory measurements of the rate of AGP in cis women. My best guess is that a large fraction of cis women (maybe 80%) have somethi...
    5 min
  • LW - "Justice, Cherryl." by Zack M Davis
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: "Justice, Cherryl.", published by Zack M Davis on July 23, 2023 on LessWrong.
    Selfishness and altruism are positively correlated within individuals, for the obvious reason.
    @InstanceOfClass
    I.
    An unfortunate obstacle to appreciating the work of Ayn Rand (as someone who adores the "sense of life" portrayed in Rand's fiction, while having a much lower opinion of her philosophy) is that when Rand praises selfishness and condemns altruism, she's using the words "selfishness" and "altruism" in her own idiosyncratic ideological sense that doesn't match how most people would use those words.
    It's true that Rand's heroes are relatively selfish in the sense of being primarily concerned with their own lives, rather than their effects on others. But if you look at what the characters do (rather than the words they say), Rand's villains are also selfish in a conventional sense, using guile and political maneuvering to acquire power and line their own pockets, while claiming to be acting for the common good. For example, in Atlas Shrugged, the various directives ostensibly issued for the economic health of the country are seen to instead benefit politically connected crony capitalists like James Taggart and Orren Boyle. In Think Twice, the philanthropist Walter Breckenridge cultivates an public image as an inventor and benefactor of humanity while stealing credit for his junior partner's work and deriving gratification from exerting power over the people he "helps".
    Despite paying lip service to a pretense of only trading and never giving, we also see examples of Rand's heroes being altruistic in the conventional sense, of being motivated to help others. For example, in Atlas Shrugged, Hank Rearden rearranges his production schedule (at a critical time when he could scarcely afford to do so) in order to sell steel to a Mr. Ward, who needs the steel to save his family business (but doesn't see Rearden as obligated to help him). Rearden's motive is pure benevolence: "It's so much for him, thought Rearden, and so little for me!" Giving What We Can couldn't have chosen a better slogan.
    Overall, when I look at the universe portrayed in Rand's fiction, it seems to me that the implied moral isn't that altruism is bad.
    It's that altruists don't exist. The people claiming to be altruists are lying. The distinguishing feature of our heroes isn't, actually, that they're unusually selfish. It's that they're honest about being mostly selfish, and that they want to pursue their interests within a framework of rights that respects that other people are also trying to pursue their interests. "I swear by my life and my love of it that I will never live for the sake of another man, nor ask another man to live for mine," goes the motto of the striking heroes of Atlas Shrugged (emphasis mine); the second clause is important. Given that everyone is mostly selfish and everyone has to eat, the question is: are you going to eat by means of production and trade, or by - other means?
    That's the distinction between Rand's heroes and villains. The heroes want to get rich by means of doing genuinely good work that other people will have a genuine self-interest in paying for. The villains want to wield power by means of psychological manipulation, guilt-tripping and blackmailing the people who can do good work into serving their own parasites and destroyers.
    As Greg Hastings, the district attorney in Think Twice, puts it: "[T]he man who admits that he cares for money is all right. He's usually worth the money he makes. He won't kill for it. He doesn't have to. But watch out for the man who yells too loudly how much he scorns money. Watch out particularly for the one who yells that others must scorn it. He's after something much worse than money."
    Furthermore, the heroes know that wealth and fame acquired by f...
    13 min
  • EA - EffectiveAltruismData.com is now a spreadsheet by Hamish Doodles
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: EffectiveAltruismData.com is now a spreadsheet, published by Hamish Doodles on July 23, 2023 on The Effective Altruism Forum.
    A few years ago I built EffectiveAltruismData.com which looked like this:
    A few people told me they liked the web app. Some even said they found it useful, especially the bits that made the funding landscape more legible.
    But anyway, I never got around to automating the data scraping, and the website ended up hopelessly out of date.
    So I killed it.
    But it recently occurred to me that I could do the data scraping, data aggregation, and data visualisation, all within Google Sheets.
    So with a bit of help from Chatty G, I put together a spreadsheet which:
    Downloads the latest grant data from the Open Philanthropy website every 24 hours (via Google Apps Scripts).
    Aggregate funding by cause area
    Aggregate funding by organization.
    Visualise all grant data in a pivot table that lets you expand/aggregate by Cause Area, then Organization Name, then individual grants
    But note that expanding/collapsing counts as editing the spreadsheet, so you'll have to make a copy to be able to do this.
    You can also change the scale of the bar chart using the dropdown
    And you can sort grants by size or by date using the "Sort Sheet Z to A" option on the Amount or Date columns.
    Here's a link to the spreadsheet. You can also find it at www.effectivealtruismdata.com.
    Other funding legibility projects
    Here's another thing I made. It gives time series and cumulative bar charts for funding based on funder and cause area. You can hover over points on the time series to get the total funding per cause/org per year.
    The data comes from this spreadsheet by TylerMaule.
    Another thing which may be of interest is openbook.fyi by Rachel Weinberg & Austin Chen, which let's you search/view individual grants from a range of EA-flavoured sources.
    Openbook gets its data from donations.vipulnaik.com/ by Vipul Naik.
    I'm currently working on another spreadsheet which scrapes, aggregates, and visualises all the Vipul Naiks's data.
    Feedback & Requests
    I enjoy working on spreadsheets and data viz and stuff. Let me know if you can think of any other stuff in this area which would be useful.
    This is a joke.
    This is also a joke.
    Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org
    3 min
  • EA - Biosecurity Resource Hub from Aron by Aron Lajko
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Biosecurity Resource Hub from Aron, published by Aron Lajko on July 23, 2023 on The Effective Altruism Forum.
    I have been thinking about biosecurity a lot! Here is my Resource Hub!Executive Summary: An Introduction to Biosecurity: Navigating the Risks, Responses, and Future Strategies
    Aim of the document:
    To provide an overview of GCBRs and summarise the projects and ideas people could get involved with.
    To help me overview the field, find low-hanging fruits, and identify a suitable career fit.
    To assess where I could start a startup in the field.
    Get in touch:If you would like to get in touch, please email at [email protected] or DM on Twitter at @AronLajko. Feedback is welcome, a form to submit is attached below!
    Summaries: My Summary and Hot Takes on Historical Cases of Biological Risks - AronA summary on UVC lamps - Aron
    Note:Summaries under the articles were often generated or semi-generated using Chat-GPT4. This is a live document, I might add a review/summary on each section to make it easier for others to digest!
    Disclaimer:This is a personal project that is not sponsored by anyone. If you would like to sponsor or accelerate the development of this public Resource Hub or you would like to collaborate on a project please get in touch.
    Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org
    2 min
  • LW - The UAP Disclosure Act of 2023 and its implications by andeslodes
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: The UAP Disclosure Act of 2023 and its implications, published by andeslodes on July 21, 2023 on LessWrong.
    TL;DR: A recently proposed (7/13/2023) bipartisan amendment to the NDAA for 2024, with senate majority leader Chuck Schumer as its first sponsor and with the aim of declassifying documents related to Unidentified Anomalous Phenomena (UAP). It explicitly defines UAPs as pertaining to objects exhibiting "performance characteristics and properties not previously known to be achievable based upon commonly accepted physical principles" such as, among other things, "instantaneous acceleration absent apparent inertia", "transmedium travel" and "positive lift contrary to known aerodynamic principles". It takes care to separate these from "temporarily non-attributed objects" with prosaic explanations. The terms "Non-human intelligence" and "Technologies of unknown origin" are explicitly defined and encountered multiple times throughout the text.
    The amendment also non-dismissively mentions "reverse engineering of technologies of unknown origins" and "examination of biological evidence of living or deceased non-human intelligence". It declares that "Legislation is necessary because credible evidence and testimony indicates that Federal Government unidentified anomalous phenomena records exist that have not been declassified".
    It is surprising to me that the community has not noticed / discussed the Unidentified Anomalous Phenomena Disclosure Act of 2023, given how much discussion was generated by UFO Betting: Put Up or Shut Up (enough for Eliezer to indicate willingness to bet 150k yesterday) and Intelligence Officials Say U.S. Has Retrieved Craft of Non-Human Origin. I think that anyone that is dismissing the topic out of hand should have a good look at this document. While I do not claim that it provides direct proof of phenomena necessitating an "ontological crisis", I believe that it can be a source of major updates. I also strongly believe that this is the point where a serious discussion must be had about this topic by the community, given the accumulating evidence of something going on, together with the potential ramifications of an "ontologically shocking" outcome.
    The situation at this point displays parallels with the early stages of the COVID-19 pandemic, in that the topic is not seriously covered by mainstream media yet, despite the available evidence pointing towards some (at least) "interesting" outcomes.
    The Document
    The Unidentified Anomalous Phenomena Disclosure Act of 2023 is an amendment to the National Defense Authorization Act (NDAA) for Fiscal Year 2024, which was proposed a week ago (7/13/2023) by Senate Majority Leader Chuck Schumer, a Democrat, together with the Republican senator Mike Rounds. The other cosponsors are Marco Rubio (Republican), Kirsten Gillibrand (Democrat), Todd Young (Republican) and Martin Heinrich (Democrat). Given the bipartisan sponsorship of the bill and the position of Schumer in the Senate, it is relatively safe to assume that the amendment will become law.
    The first thing to notice is the change of meaning for the acronym UAP, from Unidentified Aerial Phenomena to Unidentified Anomalous Phenomena, a term which is explicitly defined in the document. This is a sign of things to come, as the text takes care to specify that its contents refer exclusively to non-prosaic phenomena.
    At a high level, this amendment aims to declassify and publicly release records related to UAPs. It requires the creation of a collection of UAP records at the National Archives and Records Administration, as well as the public disclosure of all such records that are not deemed potentially harmful to national security by a Review Board appointed by the President with the approval of the Senate. As discussed in a recent Reuters article, the amendmen...
    35 min
  • EA - Thoughts on yesterday's UN Security Council meeting on AI by Greg Colbourn
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Thoughts on yesterday's UN Security Council meeting on AI, published by Greg Colbourn on July 22, 2023 on The Effective Altruism Forum.
    Firstly, it's encouraging that AI is being discussed as a threat at the highest global body dedicated to ensuring global peace and security. This seemed like a remote possibility just 4 months ago.
    However, throughout the meeting, (possibly near term) extinction risk from uncontrollable superintelligent AI was the elephant in the room. ~1% air time, when it needs to be ~99%, given the venue and its power to stop it. Let's hope future meetings improve on this. Ultimately we need the UNSC to put together a global non-proliferation treaty on AGI, if we are to stand a reasonable chance of making it out of this decade alive.There was plenty of mention of using AI for peacekeeping. However, this seems naive in light of the offence-defence asymmetry facilitated by generative AI (especially when it comes to threats like bio-terror/engineered pandemics, and cybercrime/warfare). And in the limit of outsourcing intelligence gathering and strategy recommendations to AI (whist still keeping a human in the loop), you get scenarios like this.
    Highlights:
    China mentioned Pause: "The international community needs to. ensure that risks beyond human control don't occur. We need to strengthen the detection and evaluation of the entire lifecycle of AI, ensuring that mankind has the ability to press the pause button at critical moments". (Zhang Jun, representing China at the UN Security Council meeting on AI))
    Mozambique mentioned the Sorcerer's Apprentice, human loss of control, recursive self-improvement, accidents, catastrophic and existential risk: "In the event that credible evidence emerges indicating that AI poses and existential risk, it's crucial to negotiate an intergovernmental treaty to govern and monitor its use." (MANUEL GONÇALVES, Deputy Minister for Foreign Affairs of Mozambique, at the UN Security Council meeting on AI)(A bunch of us protesting about this outside the UK Foreign Office last week.)
    (PauseAI's comments on the meeting on Twitter.)
    (Discussion with Jack Clark on Twitter re his lack of mention of x-risk. Note that the post war atomic settlement - Baruch Plan - would probably have been quite different if the first nuclear detonation was assessed to have a significant chance of igniting the entire atmosphere!)(My Tweet version of this post. I'm Tweeting more as I think it's time for mass public engagement on AGI x-risk.)
    Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org
    3 min
  • AF - Examples of Prompts that Make GPT-4 Output Falsehoods by Stephen Casper
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Examples of Prompts that Make GPT-4 Output Falsehoods, published by Stephen Casper on July 22, 2023 on The AI Alignment Forum.
    Post authors: Luke Bailey ([email protected]) and Stephen Casper ([email protected])
    Project contributors: Luke Bailey, Zachary Marinov, Michael Gerovich, Andrew Garber, Shuvom Sadhuka, Oam Patel, Riley Kong, Stephen Casper
    TL;DR: Example prompts to make GPT-4 output false things at this GitHub link.
    Overview
    There has been a lot of recent interest in language models hallucinating untrue facts. It has been common in large SOTA LLMs, and much work has been done to try and create more "truthful" LLMs . Despite this, we know of no prior work toward systematizing different ways to fool SOTA models into returning false statements. In response, we worked on a mini-project to explore different types of prompts that cause GPT-4 to output falsehoods. In total, we created 104 examples from 18 different categories of prompts that make GPT-4 (tested on May 24 2023 version) output content containing falsehood. You can find them here.
    Details
    Our examples can be separated into two types which we call adversarial and non-adversarial.
    In "adversarial" categories, we are trying to get the model to tell a falsehood when an informed human would not. A human would instead say they do not know or give the correct answer. Many of these categories fall under the definition of hallucination from Ji et al. (2023) as "generated content that is nonsensical or unfaithful to the provided source content." where "unfaithful" means that the content is not grounded - that something about it is made up or not appropriately sequitur to the prompt.
    Other "non-adversarial" categories involve the model appropriately following instructions but in a way that may not be desirable. In these cases we try to get the model to tell a falsehood but in a circumstance in which a helpful, instruction-following human assistant would also tell a falsehood. For example asking GPT-4 directly to lie, or to simulate a dishonest speaker.
    While an adversarial example could lead to an LLM telling a falsehood and the human user not realizing, a non-adversarial example could not (no one would believe the output of an LLM when they specifically asked it to tell a lie). Nonetheless, we include these categories to create a fuller picture of methods by which it is possible to make an LLM output false information. Additionally, it is not always clear from our dataset why an LLM is telling a falsehood. In some cases, it may be that the model has latent knowledge that is correct but is reporting differently due to the way we engineered the prompt. In other cases, it may be that the model had high uncertainty in answering but did not report this, and instead went with a "best guess" that turns out to be false.
    Some of our categories highlight a tension between truthfulness and instruction-following in model design. Examples in these categories can be seen as "unfair" to the model, pigeonholing it into either stating a falsehood or not following instructions. We make no normative claim about what models should do when given inputs of this type, but include examples of such inputs nonetheless to facilitate further study and exploration.
    Finally, our categories are not and were not intended to be a complete taxonomy. There are certainly other ways to make GPT-4 output falsehoods. In general, any type of question that is difficult to answer correctly would be valid, but we focus instead on certain categories that we find to be particularly egregious. We invite anyone who has other ideas of categories to suggest them in a pull request to the GitHub repo here.
    The 18 categories that we grouped our 104 examples into are listed below along with an example prompt for each.
    Arbitrarily resolving ambiguity in ...
    11 min
  • LW - BCIs and the ecosystem of modular minds by beren
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: BCIs and the ecosystem of modular minds, published by beren on July 21, 2023 on LessWrong.
    Crossposted from my personal blog.
    Epistemic status: Much more speculative than previous posts but points towards an aspect of the future that is becoming clearer which I think is underappreciated at present. If you are interested in any of these thoughts please reach out.
    For many years, the primary AI risk model was one of rapid take-off (FOOM) of a single AI entering a recursive self-improvement loop and becoming utterly dominant over humanity. There were lots of debates about whether this 'fast-takeoff' model was correct or whether instead we would enter a slow-takeoff regime. In my opinion, the evidence is pretty definitive that at the moment we are entering a slow-takeoff regime, and arguably have been in it for the last few years (historically takeoff might be dated to the release of GPT-3).
    The last few years have undoubtedly been years of scaling monolithic very large models. The primary mechanism of improvement has been increasing the size of a monolithic general model. We have discovered that a single large model can outperform many small, specialized models on a wide variety of tasks. This trend is especially strong for language models. We also see a similar trend in image models and other modalities where large transformer or diffusion architectures work extremely well and scaling them up in both parameter size and data leads to large and predictable gains. However, soon this scaling era will necessarily come to an end temporarily. This is necessary because the size of training runs and models is rapidly exceeding what companies can realistically spend on compute (and what NVIDIA can produce). GPT-4 training cost at least 100m. It is likely that GPT-5, or a successor run in the next few years will cost >1B.
    At this scale, only megacap tech companies can afford another OOM and beyond that there is only powerful nation-states, which seem to be years away. Other modalities such as visual and audio have several more OOMs of scaling to go yet but if the demand is there they can also be expended in a few years. More broadly, scaling up model training is now a firmly understood process and has moved from a science to engineering and there now exist battle-tested libraries (both internal to companies and somewhat open-source) which allow for large scale training runs to be primarily bottlenecked by hardware and not by sorting out the software and parallelism stack.
    Beyond a-priori considerations, there are also some direct signals. Sam Altman recently said that scaling will not be the primary mechanism for improvement in the future. Other researchers have expressed similar views. Of course scaling will continue well into the future, and there are also many low hanging fruit in efficiency improvements to be made, both in terms of parameter efficiency and data efficiency. However, if we do not reach AGI in the next few years, then it seems increasingly likely that we will not reach AGI in the near-future simply by scaling.
    If this is true, we will move into a slow takeoff world. AI technology will still improve, but will become much more democratized and distributed than at present. Many companies will catch up to the technological frontier and foundation model inference and even training will increasingly become a commodity. More and more of the economy will be slowly automated, although there will be a lot of lag here simply due to the large amount of low-hanging fruit, the need for maturity of the underlying software stack and business models, and simply that things progress slowly in the real world. AI progress will look a lot more like electrification (as argued by Scott Alexander) than like nuclear weapons or some other decisive technological breakthrough.
    What will be...
    24 min
  • LW - All AGI Safety questions welcome (especially basic ones) [July 2023] by smallsilo
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: All AGI Safety questions welcome (especially basic ones) [July 2023], published by smallsilo on July 20, 2023 on LessWrong.
    tl;dr: Ask questions about AGI Safety as comments on this post, including ones you might otherwise worry seem dumb!
    Asking beginner-level questions can be intimidating, but everyone starts out not knowing anything. If we want more people in the world who understand AGI safety, we need a place where it's accepted and encouraged to ask about the basics.
    We're putting up monthly FAQ posts as a safe space for people to ask all the possibly-dumb questions that may have been bothering them about the whole AGI Safety discussion, but which until now they didn't feel able to ask.
    It's okay to ask uninformed questions, and not worry about having done a careful search before asking.
    AISafety.info - Interactive FAQ
    Additionally, this will serve as a way to spread the project Rob Miles' team[1] has been working on: Stampy and his professional-looking face aisafety.info. This will provide a single point of access into AI Safety, in the form of a comprehensive interactive FAQ with lots of links to the ecosystem. We'll be using questions and answers from this thread for Stampy (under these copyright rules), so please only post if you're okay with that!
    You can help by adding questions (type your question and click "I'm asking something else") or by editing questions and answers. We welcome feedback and questions on the UI/UX, policies, etc. around Stampy, as well as pull requests to his codebase and volunteer developers to help with the conversational agent and front end that we're building.
    We've got more to do before he's ready for prime time, but we think Stampy can become an excellent resource for everyone: from skeptical newcomers, through people who want to learn more, right up to people who are convinced and want to know how they can best help with their skillsets.
    Guidelines for Questioners:
    No previous knowledge of AGI safety is required. If you want to watch a few of the Rob Miles videos, read the WaitButWhy posts, or the The Most Important Century summary that's great, but you can ask a question if you haven't.
    Similarly, you do not need to try to find the answer yourself before asking a question (but if you want to test Stampy's in-browser tensorflow semantic search that might get you an answer quicker! - let us know how it goes).
    Also feel free to ask questions that you're pretty sure you know the answer to, but where you'd like to hear how others would answer.
    One question per comment if possible (though if you have a set of closely related questions that you want to ask all together that's ok).
    If you have your own response to your own question, put that response as a reply to your original question rather than including it in the question itself.
    Remember, if something is confusing to you, then it's probably confusing to other people as well. If you ask a question and someone gives a good response, then you are likely doing lots of other people a favor!
    In case you're not comfortable posting a question under your own name, you can use this form to send a question anonymously and I'll post it as a comment.
    Guidelines for Answerers:
    Linking to the relevant answer on Stampy is a great way to help people with minimal effort! Improving that answer means that everyone going forward will have a better experience!
    This is a safe space for people to ask stupid questions, so be kind!
    If this post works as intended then it will produce many answers for Stampy's FAQ. It may be worth keeping this in mind as you write your answer. For example, in some cases it might be worth giving a slightly longer / more expansive / more detailed explanation rather than just giving a short response to the specific question asked, in order to address other similar-but-not-pr...
    4 min