The Nonlinear Library

The Nonlinear Library

By The Nonlinear FundEducation
Download on the App Store

All The Nonlinear Library episodes

  • EA - Org Proposal: Effective Foundations by Kyle Smith
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Org Proposal: Effective Foundations, published by Kyle Smith on April 22, 2023 on The Effective Altruism Forum.
    Summary:
    In this post I propose an organization that consults with US private foundations to encourage more effective grantmaking. The general consensus of EA’s I’ve spoked to working in this area is that existing private foundations are simply too intractable to be worthy of investment. However, I propose the usage of US private foundation tax filings to identify more-tractable private foundations based on several criteria. The hope is that this data approach allows this idea to cross the line from too-intractable to tractable-enough-for-investment.
    Motivation:
    Since 2011, GiveWell has directed $1 billion in effective gifts. Private foundations in the US hold over $1.1T and give ~$70B+ a year in grants, likely to largely ineffective charities. If even a small portion of these private foundations are tractable, the amount of grants redirected toward effective charities could be quite large. Impact will be measurable as engagement outcomes.
    Tractability:
    Tractability is the key challenge of this idea. From speaking with a few people in related orgs, the consensus seems to be that existing foundations are entrenched in their processes and mission and are extremely unlikely to be persuaded to change. For example, Charity Entrepreneurship mostly works with new private foundations for this reason.
    I propose that the usage of private foundation tax returns (990-PF) may point toward tractability of even existing private foundations. A common response has been that foundation age is a critical factor, so starting with new foundations makes sense. Additional factors may be significant predictors of tractability (already give to international grantees, already give to highly effective charities, give to a diverse set of cause areas or cause areas associated with EA, information about their employees [low employees/assets may imply a smaller, more persuadable management structure]). It is possible that foundation age is the only significant predictor, and if so, solicitations could be mostly targeted toward brand new foundations, and these other factors could be used to create more personalized solicitations, potentially leading to greater tractability.
    What would the org actually do?
    Develop a process of using US private foundation tax filings to identify tractable private foundations to solicit
    Make a targeted solicitation for engagements with tractable private foundations
    If successful, establish engagements similar to those done by Effective Giving and Longview Philanthropy
    Engagements with new foundations could also be based on the work that Charity Entrepreneurship has done in working with new foundations.
    Challenges:
    The primary downside risk of this approach (other than wasting money on something not tractable) is that foundations may not wish to be solicited, and will sour on EA as a result.
    A major emphasis should be placed on ensuring the solicitations are respectful. A light touch is likely necessary!
    If the foundations do sour on EA, it is likely any attempt in the future to persuade them would fail anyway.
    The data process and solicitations (unless personalized) are relatively low-cost. A robust team for actual consulting would be fairly expensive.
    I am an academic researcher and would need to bring in co-founders/partners who have experience in conducting engagements of this nature
    Four potential outcomes I envision:
    The data approach is unsuccessful in identifying tractable foundations and ultimately no progress is made.
    The data approach is somewhat successful in identifying tractable foundations, and the optimal path forward is operating as an outreach/warm lead generating organization which funnels into an existing EA consultancy.
    The data approach ...
    5 min
  • LW - Introducing the Nuts and Bolts Of Naturalism by LoganStrohl
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Introducing the Nuts and Bolts Of Naturalism, published by LoganStrohl on April 22, 2023 on LessWrong.
    What Is Naturalism?
    Naturalism is a general-purpose procedure for advancing one’s art of rationality.
    “Naturalism” is the term I use for an investigative process that focuses attention on the points in daily life where subjective experience intersects with crucial information. It is a systematic method for taking your implicit stories and assumptions and holding them up against reality. In the process, you’ll find out whether they survive that contact, or get replaced with new stories.
    I’ve previously published a sequence that focuses on the philosophy and perspective underlying this method. In it, I describe what I mean by “naturalism” in a lot of detail, by expanding on the sentence, “Knowing the territory takes patient and direct observation.” Fundamentally, I think of naturalism as the practice of behaving as though you believe that knowing the territory takes patient and direct observation.
    What Is This Sequence?
    My first naturalism sequence was about the worldview. This one is the step-by-step how-to.
    The sequence you’re currently reading is the second in a larger series. This second sequence is coherent on its own; however, if you start to read and find yourself wondering, “But why would I bother with any of this?” then you should probably stop and go read the worldview sequence first. (If you instead find yourself wishing that there were many more in-depth examples, I’m afraid you’ll have to wait for my third sequence, which I expect will consist mainly of demonstration.)
    As I pointed out at the end of the intro sequence, the practice of naturalism is entirely focused on “patient observation”.
    When a person first begins to study naturalism with me, . I begin by helping them establish consistent habits of observation. . And then, throughout what has so far proven to be about a three month program, I never shift our focus away from consistent habits of observation. It’s not just where I start. It’s the entire curriculum. .
    From a practical perspective, this dogged persistence is the foundation of naturalism. “Direct observation of the territory”, without patience, gets you something like a bag of tricks. Valuable tricks, but still tricks. Isolated mental motions made when they are convenient and enjoyable, not when they are most needed.
    With patience, though, you get a life-long practice of epistemic rationality.
    The sort of patience I have in mind is not simply waiting. It is a long series of small, consistent efforts to observe openly and thoroughly.
    I’ve developed many detailed strategies and techniques for directing those efforts over time. In the past few years, I’ve woven these strategies together into a comprehensive curriculum that several people have completed, and many more have experimented with in part.
    In these essays, I will delineate my naturalist curriculum, piece by piece.
    These are the nuts and bolts of my practice, and I’ve done my best to touch on each of them.
    What This Sequence Isn’t
    What follows is a lot like an extremely detailed syllabus. It is not really the curriculum itself in text form.
    The full curriculum depends not just on instruction in the techniques, but on practical grounding in the underlying philosophy, plus a whole lot of real-time demonstration and practice. You can’t learn the violin just by reading a book, and you can’t learn naturalism that way either. You have to wrestle with all of it in real life.
    That said, I do hope to present here a sturdy framework that especially enterprising students of rationality could personalize and flesh out into a full independent-study curriculum, and from there into a systematic rationality practice. If that might be your aim, I have a few suggestions.
    Tips For the Ambitious
    Tip 1: Re...
    5 min
  • LW - The Security Mindset, S-Risk and Publishing Prosaic Alignment Research by marc/er
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: The Security Mindset, S-Risk and Publishing Prosaic Alignment Research, published by marc/er on April 22, 2023 on LessWrong.
    Note: Don’t take this as a cry to stop publishing alignment research, take this as a cry to approach doing so with a security mindset.
    Introduction
    When converging upon useful alignment ideas, it is natural to want to share them, get feedback, run experiments and iterate in the hope of improving them, but doing this suddenly can bare contradiction to the security mindset. The Waluigi Effect, for example, ended with a rather grim conclusion: “If this Semiotic–Simulation Theory is correct, then RLHF is an irreparably inadequate solution to the AI alignment problem, and RLHF is probably increasing the likelihood of a misalignment catastrophe. Moreover, this Semiotic–Simulation Theory has increased my credence in the absurd science-fiction tropes that the AI Alignment community has tended to reject, and thereby increased my credence in s-risks.” Could the same apply to other prosaic alignment techniques? What if they do end up scaling to superintelligence? I am realizing more and more that just like the capabilities researchers I blame for lacking the insight/drive to deviate from publishing, I am also falling victim to Moloch.
    It's easy to internally justify publishing in spite of this. Some examples of justifications I have given to myself are “You’re so new to this, this is not going to have any real impact anyway”, or just being so single mindedly focused on mitigating extinction, to the extent that despite recognizing the spectrum of possible outcomes, my internal world model is truly just binary (extinct or otherwise). These comments always sound reasonable at the time, but upon reflection it is easy to see that they are completely distinct from my true mindset when developing exciting new ideas, or thinking about explaining them to other people. While developing new ideas I would often tell myself “This is really interesting! I think this could have an impact on alignment!”, and while justifying publishing them I would be telling myself “It’s probably not that good of an idea anyway, just push through and wait until someone points out the obvious flaw in your plan.”
    Completely and detrimentally inconsistent!
    Arguments 1 and 3
    Of course many people have shared their musings about this before. Andrew Saur does so in his post “The Case Against AI Alignment”, Andrea Miotti explains here why they believe RLHF-esque research is a net-negative for existential safety (see Paul Christiano’s response), and Christiano provides an alternative perspective in his “Thoughts on the Impact of RLHF Research”. Generally, these arguments seem to fall into three different categories (or their inversions):
    This research will further capabilities more than it will alignment
    This alignment technique if implemented could actually elevate s/x-risks
    This alignment technique if implemented will increase the commercializability of AI, feeding into the capabilities hype cycle, indirectly contributing to capabilities more than alignment.
    To clarify, I am speaking here in terms of impact, not necessarily in terms of some static progress metric (e.g. one year of alignment progress is not necessarily equivalent to one year of capabilities progress in terms of impact).
    In an ideal world, we could trivially measure potential capabilities/alignment advances and make simple comparisons. Sadly this is not that world, and realistically most decisions are going to be intuition-derived. Worse yet is that argument 2 is even murkier than 1 and 3. Trying to pose responses to question 2 is quite literally trying to prove that a solution you don’t know the implications of won’t result in an outcome we can hardly begin to imagine through means we don’t understand. On the bright side; we a...
    10 min
  • LW - "Rate limiting" as a mod tool by Raemon
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: "Rate limiting" as a mod tool, published by Raemon on April 23, 2023 on LessWrong.
    A lot of my recent thoughts on moderation have included an emphasis on "rate limiting as a primary mod tool." This is very much in the "Ray has a hypothesis and it might not work out" phase, but I think it has promise.
    The general principle is "moderators should be able to employ the least-amount-of-constraint/force possible to enforce a decision or give that decision teeth." A full ban is a pretty extreme option, and being limited to that option makes me much more hesitant to take any kind of action against a person.
    A rate limit is a pretty flexible tool. You can rate limit comments, posts, or (possibly) wordcount. You can limit by hour, day, week or more. It can be limited to a specific post, or across posts in general (i.e. N comments per post per unit-of-time), or sitewide (N comments total per unit of time)
    But at least some people have reported experiencing limits as very costly, and have described it as "being banned." Ruby has argued it might basically just feel like moderators are trying to ban people without really acknowledging the magnitude of what they're doing.
    I'm curious how people feel about it. I want to outline a bit how I'm thinking about it.
    There's roughly 3 ways I think I want to use rate-limits
    1. As a soft, temporary tool to give moderator-warnings a little more teeth.
    If someone(s) is/are spiralling into a demon thread, we could give them a temporary "1 comment per ~hour" rate limit on that particular post, to force them to slow down and chill out. If they've done something that seems particularly norm-violating, we can give them a temporary significant rate limit. The alternative here would be to do nothing, or give them a temporary 1-week ban. It seems at least in some cases better to do the rate-limit, although I could imagine in some cases "taking an actual break for a week" would be better.
    2. As a response to serious problem users who are either new, or established but have been given previous warnings.
    The alternatives are "permaban," "doing nothing", and "moderators consistently have to be-on-top-of-things giving 'temp-bans' or other local-reactions to serious problem users each time they do something bad."
    Doing nothing means leaving the garden untended, falling into entropy. "Moderators have to keep paying attention" is just... way too much work. From this perspective, a significant rate-limit is harsh, but the alternative seems probably worse. Rate limits give the user an option for proving they can change their commenting habits.
    A thing that might change my mind here is if in practice, either most users just prefer exile to second-class-citizenship, or if longterm rate limits just in practice turn out to be a trap for people who really should move on.
    3. As a default-state for new users.
    New users sometimes show up and start commenting prolifically, before they really understand the site norms or background knowledge.
    I think it's a pretty reasonable position for "Users start out with one comment per day. If they get even slightly upvoted, they get a bit more commenting permissions, and if they get, say, 5 or 10 karma they become unfettered." I predict that, once the site fully shifted over to this, it'd feel fairly reasonable and gamified in a positive way.
    There's an awkwardness right now of "there's a lot of users I would have wanted to start out more rate-limited in this way, but who showed up in the past year or 2, who've been making lots of downvoted comments and not really learning from it." So there is an awkward/painful readjustment period for those users.
    The main alternative to this (which is what we're actually doing right now, as of a couple weeks ago), is manually approving each individual comment from new users, and mostly not lett...
    7 min
  • LW - ask me about my battery by bhauth
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: ask me about my battery, published by bhauth on April 21, 2023 on LessWrong.
    Some years ago, I designed a battery chemistry. Then, I got a semi-related chemical industry job. Then, in 2020, I did some testing at a university lab. Here's the basic design:
    As far as I was able to test it, the battery design worked, but the project died. I haven't been able to do much with that design since.
    The current name for this battery chemistry is "SMAC" for "sodium metal aqueous cathode". If you want, you can ask about:
    technical details
    experimental results
    why I didn't do X
    the response from X types of people
    other battery chemistries
    I could certainly provide more details here, but I'm curious what people are most interested in.
    Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.
    1 min
  • EA - If your AGI x-risk estimates are low, what scenarios make up the bulk of your expectations for an OK outcome? by Greg Colbourn
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: If your AGI x-risk estimates are low, what scenarios make up the bulk of your expectations for an OK outcome?, published by Greg Colbourn on April 21, 2023 on The Effective Altruism Forum.
    There seem to be two main framings emerging from recent AGI x-risk discussion: default doom, given AGI, and default we're fine, given AGI.
    I'm interested in what people who have low p(doom|AGI) think are the reasons that things will basically be fine once we have AGI (or TAI, PASTA, ASI). What mechanisms are at play? How is alignment solved so that there are 0 failure modes? Can we survive despite imperfect alignment? How? Is alignment moot? Will physical limits be reached before there is too much danger?If you have high enough p(doom|AGI) to be very concerned, but you're still only at ~1-10%, what is happening in the other 90-99%?Added 22Apr: I'm also interested in detailed scenarios and stories, spelling out how things go right post-AGI. There are plenty of stories and scenarios illustrating doom. Where are the similar stories illustrating how things go right? There is the FLI World Building Contest, but that took place in the pre-GPT-4+AutoGPT era. The winning entry has everyone acting far too sensibly in terms of self-regulation and restraint.
    I think we can now say, given the fervour over AutoGPT, that this will not happen, with high likelihood.
    Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.
    2 min
  • LW - Thinking about maximization and corrigibility by James Payor
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Thinking about maximization and corrigibility, published by James Payor on April 21, 2023 on LessWrong.
    Thanks in no small part to Goodhart's curse, there are broad issues with getting safe/aligned output from AI designed like "we've given you some function f(x), now work on maximizing it as best you can".
    Part of the failure mode is that when you optimize for highly scoring x, you risk finding candidates that break your model of why a high-scoring candidate is good, and drift away from things you value. And I wonder if we can repair this by having the AI steer away from values of x that break our models, by being careful about disrupting structure/causal-relationships/etc we might be relying on.
    Here's what I'd like to discuss in this post:
    When unstructured maximization does/doesn't work out for the humans
    CIRL and other schemes mostly pass the buck on optimization power, so they inherit the incorrigibility of their inner optimization scheme
    It's not enough to sweep the maximization under a rug; what we really need is more structured/corrigible optimization than "maximize this proxy"
    Maybe we can get some traction on corrigible AI by detecting and avoiding internal Goodhart
    When does maximization work?
    In cases when it just works to maximize, there will be a structural reason that our model connecting "x scores highly" to "x is good" didn't break down. Some of the usual reasons are:
    Our metric is robustly connected to our desired outcome. If the model connecting the metric and good things is simple, there's less room for it to be broken.Examples: theorem proving, compression / minimizing reconstruction error.
    The space we're optimizing over is not open-ended. Constrained spaces leave less room for weird choices of x to break the correspondences we were relying on.Examples: chess moves, paths in a graph, choosing from vetted options, rejecting options that fail sanity/legibility checks.
    The optimization power being applied is limited. We can know our optimization probably won't invent some x that breaks our model if we know what kinds of search it is performing, and can see that these reliably don't seek things that could break our model.Examples: quantilization, GPT-4 tasked to write good documentation.
    The metric f is actively optimized to be robust against the search. We can sometimes offload some of the work of keeping our assessment f in tune with goodness.Examples: chess engine evaluations, having f evaluate the thoughts that lead to x.
    There's a lot to go into about when and whether these reasons start breaking down, and what happens then. I'm leaving that outside the scope of this post.
    Passing the buck on optimization
    Merely passing-the-buck on optimization, pushing the maximization elsewhere but not adding much structure, isn't a satisfactory solution for getting good outcomes out of strong optimizers.
    Take CIRL for instance, or perhaps more broadly the paradigm: "the AI maximizes an uncertain utility function, which it learns about from earmarked human actions". This design has something going for it in terms of corrigibility! When a human tries to turn it off, there's scope for the AI to update about which sort of thing to maximize, which can lead to it helping you turn itself off.
    But this is still not the sort of objective you want to point maximization at. There are a variety of scenarios in which there are "higher-utility" plans than accepting shutdown:
    If the AI thinks it already knows the broad strokes of the utility function, it can calculate that utility would not be maximized by shutting off. It's learning something from you trying to press the off switch, but not what you wanted.
    It might seem better to stay online and watch longer in order to learn more about the utility function.
    Maybe there's a plan that rates highly on "utility" that works ...
    11 min
  • EA - 500 Million, But Not A Single One More - The Animation by Writer
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: 500 Million, But Not A Single One More - The Animation, published by Writer on April 21, 2023 on The Effective Altruism Forum.
    This video is an animation of 500 Million, But Not A Single One More, by @jai
    Some notes for the curious about the cult of Sopona, the god of Smallpox of the Yoruba people
    When I was getting feedback on the storyboard from Jai, he mentioned that there is an actual god of Smallpox: Sopona. We decided to use it to lightly inspire our design by adding a mask similar to the ones found in wooden statues representing the god.
    Apparently, in Nigeria there were cults of Sopona that strongly opposed vaccination. At the bottom-left paragraph at this link, there's an interesting story. Dr. O. Sapara, a Nigerian physician, infiltrated "the sopono cult", described as a large secret society in western Nigeria that used to cause smallpox outbreaks to generate clients to "cure" and to follow-up on blackmails. Sapara infiltrated the society and helped the government ban the cult as an illegal organization. After the cult was banned worship of the God continued.
    Jai rightly mentioned the priests of the cult as a striking example of the most hollow humans allying with the enemy.
    Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.
    2 min
  • LW - Would we even want AI to solve all our problems? by So8res
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Would we even want AI to solve all our problems?, published by So8res on April 21, 2023 on LessWrong.
    Status: more of the basics, that I find myself regularly repeating.
    When people from the wider population talk to me about politics or global warming or various other large global issues (as they're prone to do), I am prone to noting that most problems would be easier to fix if we had superintelligent friends at our backs.
    A common objection I get goes something like:
    So you think you can get AI to solve all the world's problems? Wouldn't that kinda suck, though? Wouldn't a problem-free life be kinda terrible? Isn't overcoming obstacles part of the fun of life?
    To which my answer is: that's completely right. If we aren't careful, artificial intelligence could easily destroy all the value in the future, e.g. by aggressively solving everyone's problems and giving everyone everything they ask for, in a way that sucks all the joy and meaning out of life.
    Value is fragile. The goal is not to alleviate every ounce of discomfort; the goal is to make the future awesome. My guess is that that involves leaving people with real decisions that have real consequences, that it involves giving people the opportunity to screw up, that it involves allowing the universe to continue to be a place of obstacles that people must overcome.
    But it also probably involves putting an end to a bunch of the senseless death, and putting an end to bad things happening to good people for no reason, and putting an end to the way that reality can punish a brief lapse of judgment with disproportionate, horrific, and irreversible consequences.
    And it probably involves filling the universe with much more fun and joy and laughter. (And not just by building a bunch of speakers that play laugh tracks on a loop, but by filling the universe with the sorts of things that cause joyful laughter among healthy people, plus the healthy people to experience those things and laugh joyfully.)
    The problem of making the future awesome is tricky and subtle. The way to get AI to help in the long term, is not to tell them exactly what our current best guess of what's "good" consists in, in a long list of current philosopher's best guesses, and then hope that modern philosophers got it exactly right. Because modern philosophers haven't got it exactly right; and that whole approach is doomed to failure.
    The hope is that we can somehow face our AI and point at all the things we care about, and say "we don't know how to describe this in detail, and we don't know what the limits are, but we care about it; can you look at us and look at all this lovely stuff and care about it too, and help us in the great endeavor to have a wonderful future?". The win-state is ultimately to make friends, to make allies that help us fight the good fight.
    The field of "what is the good stuff even (given that it's not something simple, like mere cessation of suffering or understanding of the universe)?" is called fun theory, and people who took the AI problem seriously invented it, once it became clear that our species is going to face a question like this, as we grow up.
    The word I use for this concept is Fun (with a capital F), and I don't pretend to know exactly what it consists of, and the long-term goal is to build superintelligences that also want the future to be Fun.
    And make no mistake, this is a long term goal. None of this is something humanity should be shooting for on its first try building AIs. The issue of AI alignment today is less about making the AIs care about fun, and more about making them perform some minimal pivotal act that ends the acute risk period, and buys us time to figure out how to make the superintelligent friends.
    Because it's important not to screw that task up, and we shouldn't attempt it under time pressure befo...
    4 min
  • EA - DeepMind and Google Brain are merging [Linkpost] by Akash
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: DeepMind and Google Brain are merging [Linkpost], published by Akash on April 20, 2023 on The Effective Altruism Forum.
    Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.
    1 min