The Nonlinear Library

The Nonlinear Library

By The Nonlinear FundEducation
Download on the App Store

All The Nonlinear Library episodes

  • EA - 'Psychology of Effective Altruism' course syllabus by Geoffrey Miller
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: 'Psychology of Effective Altruism' course syllabus, published by Geoffrey Miller on September 7, 2022 on The Effective Altruism Forum.
    In case anyone's interested, here's a link to the syllabus for a course titled 'The Psychology of Effective Altruism' that I've taught 3 times at University of New Mexico (in 2018, 2019, and 2020):
    I'd posted an earlier version of this syllabus to EA Forum in 2018 (see )
    Feel free to borrow any of this material if you teach a course on EA, X risk, longtermism, moral psychology, etc.
    This is an advanced undergraduate seminar with about 20-25 students, mostly psychology majors. It was designed to be suitable for diverse students at a large state university. Whereas some other EA courses have focused mostly on technical moral philosophy, this is pitched more towards the psychology of EA -- both why it's appealing, and why it's difficult.
    The course's main topics are EA principles, cause prioritization, utilitarian ethics, moral psychology, charity evaluation, global poverty and health, existential risks, AI safety, moral & cognitive enhancement, near-term technologies (robots, EMs, VR), animal sentience & welfare, and career choices.
    I'm currently updating this course to teach it again in spring 2023, so would welcome anybody else's syllabi that might be helpful, and any other suggestions of good readings, videos, exercises, etc. (I've already looked at the helpful course materials mentioned on this Open Philanthropy site:/ )
    -- Geoffrey
    Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.
    2 min
  • AF - Linkpost: Github Copilot productivity experiment by Daniel Kokotajlo
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Linkpost: Github Copilot productivity experiment, published by Daniel Kokotajlo on September 8, 2022 on The AI Alignment Forum.
    We recruited 95 professional developers, split them randomly into two groups, and timed how long it took them to write an HTTP server in JavaScript. One group used GitHub Copilot to complete the task, and the other one didn’t. We tried to control as many factors as we could–all developers were already familiar with JavaScript, we gave everyone the same instructions, and we leveraged GitHub Classroom to automatically score submissions for correctness and completeness with a test suite. We’re sharing a behind-the-scenes blog post soon about how we set up our experiment!
    In the experiment, we measured—on average—how successful each group was in completing the task and how long each group took to finish.
    The group that used GitHub Copilot had a higher rate of completing the task (78%, compared to 70% in the group without Copilot).
    The striking difference was that developers who used GitHub Copilot completed the task significantly faster–55% faster than the developers who didn’t use GitHub Copilot. Specifically, the developers using GitHub Copilot took on average 1 hour and 11 minutes to complete the task, while the developers who didn’t use GitHub Copilot took on average 2 hours and 41 minutes.
    My opinion: Because of the usual reasons (publication bias, replication crisis, the task being "easy," etc.) I don't think we should take this particularly seriously until much more independent experiments have been run. However, it's worth knowing about at least. Related:
    We compare the hybrid semantic ML code completion of 10k+ Googlers (over three months across eight programming languages) to a control group and see a 6% reduction in coding iteration time (time between builds and tests) when exposed to single-line ML completion. These results demonstrate that the combination of ML and SEs can improve developer productivity. Currently, 3% of new code (measured in characters) is now generated from accepting ML completion suggestions.
    Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.
    3 min
  • LW - On oxytocin-sensitive neurons in auditory cortex by Steven Byrnes
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: On oxytocin-sensitive neurons in auditory cortex, published by Steven Byrnes on September 6, 2022 on LessWrong.
    (For the big picture of how I wound up on this topic, see Symbol Grounding and Human Social Instincts. But I wound up feeling like oxytocin-sensitive neurons in auditory cortex are NOT an important piece of that particular puzzle.)
    (I tried to minimize neuroscience jargon (as usual), but I don’t really expect that non-neuroscientists would want to read this post anyway.)
    I just read the paper “Oxytocin enables maternal behaviour by balancing cortical inhibition” (free PDF link) by Bianca J. Marlin, Mariela Mitre, James A. D’amour, Moses V. Chao, & Robert C. Froemke at NYU (Nature, 2015). And then I spent a while feeling confused. But I think I’m no longer confused. In this post I’ll explain both why I was confused, and how I got over it.
    Background: pup-retrieval behavior
    The paper concerns a behavior which is amusingly described in the following book excerpt (emphasis added):
    After a rat gives birth, she displays a complex repertoire of maternal behaviors. If given paper, she will shred it and use the strips to build a nest. Virgin and early pregnant rats avoid newborn pups, but a mother rat will gather her young into the nest and allow them to suckle, and if any pup wanders away she will promptly retrieve it. Indeed, she will retrieve any pup that she sees close to her nest, whether hers or not, seemingly without limit. If 20 or 30 strange pups are placed in her cage, all will be retrieved, and she will strive to ensure that all are groomed and fed. . These behaviors are expressed after normal vaginal delivery, but are disrupted by interventions that impair oxytocin release .
    Strikingly, maternal behavior can be induced by injecting small amounts of oxytocin into the brain, both in virgin rats when at the stage of the cycle when estrogen levels are high, and in ovariectomized rats that have been infused with estrogen. —Gareth Leng, The Heart Of the Brain: The Hypothalamus and its Hormones, p191
    The Marlin et al. paper is about this pup-retrieval behavior, which the mother does in response to an isolated pup shouting a high-pitched “distress call”.
    Marlin et al. results (specifically concerning primary auditory cortex)
    Marlin et al. did a series of experiments where they found (among other things) the following:
    When they temporarily inactivated the left primary auditory cortex (with a muscimol infusion), it reduced pup-retrieval in experienced mothers.
    More specifically, it eliminated pup-retrieval in 5/16 mice, and reduced it in 8/16 mice, while the last 3/16 heroic mice continued to retrieve every last distressed pup despite the infusion.
    On the other hand, when they temporarily inactivated the right primary auditory cortex, it didn’t do much (if anything) to pup-retrieval in experienced mothers.
    Specifically, it did nothing in 4/5 mice, and reduced pup retrieval in the other one, but that might have just been a random blip.
    They found that there were 40% more oxytocin receptors in the left primary auditory cortex than right primary auditory cortex. They also found a plausible source of oxytocin for those receptors to be detecting, namely, some fibers in primary auditory cortex coming from oxytocin-producing neurons in the hypothalamus. (There were similar numbers of such fibers on the left and right sides.)
    They found that if they took an experienced mother, and blocked the activity of oxytocin in her left primary auditory cortex, it did not change her pup-retrieval behavior.
    They found that if they took a virgin female, and locally infused oxytocin into her left primary auditory cortex shortly before pup-retrieval tests, her first pup-retrieval would happen earlier. (I think this result is probably real, but be warned that the data analysis barely rea...
    21 min
  • EA - Fundraising Campaigns at Your Organization: A Reliable Path to Counterfactual Impact by High Impact Professionals
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Fundraising Campaigns at Your Organization: A Reliable Path to Counterfactual Impact, published by High Impact Professionals on September 7, 2022 on The Effective Altruism Forum.
    TL;DR
    Running a fundraising campaign at your workplace can be a highly effective way to multiply your impact.
    In 2021, High Impact Professionals (“HIP”) supported EAs in organizing 8 fundraising campaigns at 8 different companies, counterfactually raising 240,000 USD for effective charities.
    The average amount raised per event was about 30,000 USD (median 3,900 USD), with most donations coming from a few events.
    On average, it took about 25 hours to organize and run a campaign (20 hours by organizers and 5 hours by HIP).
    The events generated an average of 786 USD per hour of counterfactual donations to effective charities.
    This makes fundraising campaigns a very cost effective means of counterfactual impact; as a comparison, direct work that generates 1,000,000 USD of impact equivalent per year equates to around 500 USD per hour.
    If you’re interested in exploring what a fundraising campaign could look like at your organization, please contact us and check out our step-by-step fundraising guide.
    If you’re interested in learning more about the 2021 campaigns’ data and methodology, please keep reading.
    We would love to get feedback on our data and methodology, so don’t hesitate to reach out here or in the comments.
    Intro
    Running a fundraising campaign at your workplace can be a highly effective way to multiply your impact as a working professional. Getting colleagues to donate money to effective charities not only increases your donation leverage but also has the potential to get others involved in the EA movement. It can also help you build relevant EA career capital.
    During the 2021 giving season, HIP supported EAs in organizing 8 fundraising campaigns at 8 different companies. Essentially, the workplace campaigns encouraged those EAs’ colleagues to donate to effective charities. The results detailed below point to these events being a strong way to enhance one’s impact.
    Results
    In total, the 8 campaigns counterfactually raised about 240,000 USD for effective charities. The average amount raised per event was about 30,000 USD, with a 90% Confidence Interval (“CI”) of 80 USD to 140,000 USD. The graph below shows the distribution in logarithmic scale.
    The main cost of the initiative is the time taken to organize the campaign. On average, organizers spent 20 hours on their events (CI 6 hours to 42 hours) and received 5 hours of support from HIP (CI 2 hours to 10 hours). The distributions are shown below.
    Computing the ratio of money raised to time spent, we arrive at an average of 786 USD per hour (CI 7 USD to 3,100 USD).
    To put this into perspective, let’s assume that direct work generates between 100,000 USD and 1,000,000 USD of impact equivalent per year and that people work between 1,800 and 2,200 hours per year. This computes to an average of 290 USD per hour (CI 57 USD to 530 USD), which means that organizing fundraising campaigns can be about 2.7x as effective as direct work on an hourly basis.
    Of course, we are not saying that people should stop considering direct work and do fundraising events instead since, for example, the impact of doing fundraising campaign work year-round may not scale as well. Still, strategically timed campaigns (e.g., around giving season, your organization’s raise/bonus season, or another liquidity event) could be a very high impact seasonal side gig.
    Heavy Tail
    One could raise a fair argument that the donation distribution above is quite heavily tailed, and because of the limited data points, the result may be anomalous. Though we have already been quite conservative in the estimation of the counterfactual impact of the events, we could take an even more conse...
    9 min
  • EA - Agree/disagree voting (& other new features September 2022) by Lizka
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Agree/disagree voting (& other new features September 2022), published by Lizka on September 7, 2022 on The Effective Altruism Forum.
    We’re enabling two-factor voting (karma and agree/disagree) on all (new) comment threads in the Forum. We’ll be reviewing feedback and will evaluate how the change has gone in October, but we expect to keep the feature.
    We wrote this post to share this update and some other changes. On that note, the Forum team continues to grow; Ollie Etherington and Will Howard recently joined our team, and we’re eagerly anticipating our new product manager. We’re also still excited to hear about promising UX designers interested in working on the Forum.
    As always, feedback is welcome — you can comment on this post with specific input, or request more features via the Forum feature suggestion thread.
    Summary of the changes
    Two-factor voting is going live on the Forum. This adds agree-disagree voting on top of the usual karma system. ⬇️
    We’ve started curating Forum posts that don’t get enough visibility or are especially good as examples of what Forum posts should be (according to us). ⬇️
    There’s finally a way to copy-paste from a Google Doc with footnotes. ⬇️
    We’re testing a 1:1 service to connect people interested in working in a field with experts in that field (the current service is for people interested in mitigating global catastrophic biological risks). ⬇️
    Crossposting to and from LessWrong will be easier. ⬇️
    You can add topics to your public profile. ⬇️
    Some other changes ⬇️
    What’s new
    Two-factor voting (karma and agree/disagree) on all comment threads
    You can now vote separately on whether you appreciate a comment (and think more people should see it) and on whether you agree with the contents of the comment. Only the first of these will affect the poster’s karma.
    The LessWrong team implemented this feature on LessWrong two months ago, and their announcement post shares details about how it works and what they like about it; we echo basically everything in that post. Some key points made in that post:
    Agree/disagree voting does not translate into a user's or post's karma — its sole function is to communicate agreement/disagreement. It has no other direct effects on the site or content visibility (i.e. no effect on sorting algorithms).
    For both regular voting and the new agree/disagree voting, you have the ability to normal-strength vote and strong-vote. Click once for normal-strength vote. For strong-vote, click-and-hold on desktop or double-tap on mobile. The weight of your strong-vote is approximately proportional to your karma on a log-scale (exact numbers here).
    We’re really grateful to the LessWrong team for all their work on this and are excited about this as a potential improvement on the Forum.
    How sure are we that we want to keep this feature?
    Our default plan is to keep it, although we welcome feedback and plan to re-evaluate in October. We had planned to test the feature on the Forum more thoroughly by asking authors to opt into enabling it on their posts’ comment sections. However, the testing has been very slow, and, seeing the results on LessWrong (and on the Forum), we were increasingly feeling like we were wasting a great tool.
    Curation of especially good posts
    The number of posts that get shared on a single day is growing, and it’s increasingly hard for Forum users to keep up and see the best and most relevant-to-them posts. One way we’re trying to address this is by curating some of our favorite posts. We’ll generally leave a comment explaining why we curated the post.
    How this works
    The exact details of this system might change, but for now, they’re as follows. When a post is curated, it reappears at the top of the Frontpage with a little star next to it. The timestamp on it will show the time it got curated (although the ...
    8 min
  • LW - Framing AI Childhoods by David Udell
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Framing AI Childhoods, published by David Udell on September 6, 2022 on LessWrong.
    Generated as part of SERI MATS, under John Wentworth. Thanks to Alex Turner and Garrett Baker for related discussion, and Justis Mills for draft feedback.
    All bold claims and ignorant mistakes herein are my own.
    Do you remember when Evan Hubinger became really enamored with 'training stories,' a way of carving up the alignment problem into 'training rationales' and 'training goals'? Evan's idea was that we ought to think of alignment as choosing a target model that we want to end up with after training, plus choosing a training procedure that will actually yield that model. In my prior experience talking to people about this way of framing the alignment problem. people didn't especially get it. The typical response of those who had heard of this was, "Yeah, that's one way to carve up the problem apart from, e.g., inner and outer alignment. But so what? How does this framing help us actually reduce the problem? It seems no better or worse than our old framings."
    I used to have that response myself. However, I now think I understand training stories! Here's my take:
    What Do Training Stories Buy You That Inner/Outer Alignment Doesn't?
    It’s worth pointing out how phrasing inner and outer alignment in terms of training stories makes clear what I think was our biggest mistake in formulating that terminology, which is that inner/outer alignment presumes that the right way to build an aligned model is to find an aligned loss function and then have a training goal of finding a model that optimizes for that loss function. However, as I hope the more general framework of training stories should make clear, there are many possible ways of trying to train an aligned model. Microscope AI and STEM AI are examples that I mentioned previously, but in general any approach that intends to use a loss function that would be problematic if directly optimized for, but then attempts to train a model that doesn’t directly optimize for that loss function, would fail on both outer and inner alignment—and yet might still result in an aligned model.
    --Evan, How do we become confident in the safety of a machine learning system?
    What training stories buy you is a framing of the alignment problem that doesn't imply that the only way you can use a loss function is as an outer objective that you would like to see directly optimized for. This is one way you can use a loss function -- but it definitely isn't the only such way! It's badly misleading for your framing of the alignment problem to suggest that inner/outer alignment of a powerful model is the only way to survive.
    For example, say that you were raising a kid. One childrearing scheme is to carefully make your kid's childhood robust to arbitrary levels of precocious genius in your kid. You'd build a childhood such that overachieving in it would only ever be a good thing. You'd drill athletics, navigating complex adult social situations, difficult moral dilemmas, etc., always making sure that there isn't some perverse victory condition way up near the skill ceiling of the task. For on this approach, you don't ever want optimization power being pointed in a direction you wouldn't want to see optimized, in the way that you don't ever want to needlessly point a gun barrel at anything you don't want destroyed. This childrearing scheme revolves around designing for overachievers, so that your kids will leave childhood once they rise to the level of "overachiever," and then generalize their instilled drive to overachievement in the same spirit after they leave the nest.
    You'll notice that the above approach to childrearing is pretty weird, and looks more like Ender's Game or Molly's Game than it does any kind of conventionally advised childrearing. It's in fact okay for...
    8 min
  • EA - Announcing a Philosophy Fellowship for AI Safety by Anders Edson
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Announcing a Philosophy Fellowship for AI Safety, published by Anders Edson on September 7, 2022 on The Effective Altruism Forum.
    Overview
    The Center for AI Safety (CAIS) is announcing the CAIS Philosophy Fellowship, a program for philosophy PhD students and postdoctorates to work on conceptual problems in AI safety.
    Why Philosophers?
    Conceptual AI safety researchers aim to help orient the field and clarify its ideas, but in doing so, they must wrestle with imprecise, hard-to-define problems. Part of the difficulty involved in conducting conceptual AI safety research is that it involves abstract thinking about future systems which have yet to be built. Additionally, the concepts involved in conceptual AI safety research (e.g., “power”, “intelligence”, “optimization”, “agency”, etc.) can be particularly challenging to work with.
    Philosophers specialize in working through such nebulous problems; in fact, many active fields within philosophy present a similar type of conceptual challenge, and have nothing empirical to lean on. As an example, a philosopher may take up the question of whether or not ethical claims can possess truth values. Questions such as this one cannot be approached by looking carefully at the world, making accurate measurements, or monitoring the ethical behavior of real people. Instead, philosophers must grapple with intuitions, introduce multiple perspectives, and provide arguments for selecting between these perspectives. We think that this skill set makes philosophers especially fit for conceptual research.
    Philosophy has already proven itself to be useful for orienting the field of conceptual AI safety. Many of the foundational arguments for AI-risk were philosophical in nature. (As an example, consider Bostrom’s Superintelligence.) More recently, philosophy has proven itself to have direct influence on the important research directions in AI safety. Joseph Carlsmith’s work on power-seeking AI, for example, has directly influenced research currently being conducted by Beth Barnes and, separately, Dan Hendrycks. Peter Railton’s lectures on AI have provided a compelling justification for further research on cooperative behavior in AI agents. Evan et al.'s exploration into truthful AI prompted more technical works into truthful and honest AI.
    Since philosophers have historically produced valuable conceptual AI safety work, we believe that introducing more philosophy talent into this research space has the potential to be highly impactful. By offering good incentives, we hope to attract strong philosophy talent with a high likelihood of producing quality conceptual research.
    The Program
    Our program will be a paid, in-person opportunity running from January to August 2023. Our ideal candidate is a philosophy PhD student or graduate with an interest in AI safety, exceptional research abilities, demonstrated philosophical rigor, self-motivation, and a willingness to spend time working with more technical subjects. No prior experience in AI or machine learning is necessary for this fellowship. There will be an in-depth onboarding program at the start of the fellowship to get the researchers up to speed on the current state of AI/AI safety.
    Fellows will receive a $60,000 grant, covered student fees, and a housing stipend to relocate to San Francisco, CA. The program will feature guest lectures from top philosophers and AI safety experts including Nick Bostrom, Peter Railton, Hilary Greaves, Jacob Steinhardt, Rohin Shah, David Kruger, and Victoria Krakovna among others.
    As an organization that places a high value on good conceptual researchers, we plan on extending full-time employment offers at our organization to top-performing fellows. Additionally, many institutions such as the Center for Human-Compatible AI (UC Berkeley), the Kavli Center for Ethics,...
    5 min
  • EA - 13 background claims about EA by Akash
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: 13 background claims about EA, published by Akash on September 7, 2022 on The Effective Altruism Forum.
    I recently attended EAGxSingapore. In 1-1s, I realized that I have picked up a lot of information from living in an EA hub and surrounding myself with highly-involved EAs.
    In this post, I explicitly lay out some of this information. I hope that it will be useful for people who are new to EA or people who are not living an EA Hub.
    Here are some things that I believe to be important “background claims” that often guide EA decision-making, strategy, and career decisions. (In parentheses, I add things that I believe, but these are "Akash's opinions" as opposed to "background claims.")
    Note that this perspective is based largely on my experiences around longtermists & the Berkeley AI safety community.
    General
    1. Many of the most influential EA leaders believe that there is a >10% chance that humanity goes extinct in the next 100 years. (Several of them have stronger beliefs, like a 50% of extinction in the next 10-30 years).
    2. Many EA leaders are primarily concerned about AI safety (and to a lesser extent, other threats to humanity’s long-term future). Several believe that artificial general intelligence is likely to be developed in the next 10-50 years. Much of the value of the present/future will be shaped by the extent to which these systems are aligned with human values.
    3. Many of the most important discussions, research, and debates are happening in-person in major EA hubs. (I claim that visiting an EA Hub is one of the best ways to understand what’s going on, engage in meaningful debates about cause prioritization, and receive feedback on your plans.)
    4. Several “EA organizations” are not doing highly impactful work, and there are major differences in impact between & within orgs. Some people find it politically/socially incorrect to point out publicly which organizations are failing & why. (I claim people who are trying to use their careers in a valuable way should evaluate organizations/opportunities for themselves, and they should not assume that generically joining an “EA org” is the best strategy.)
    AI Safety
    5. Many AI safety researchers and organizations are making decisions on relatively short AI timelines (e.g., artificial general intelligence within the next 10-50 years). Career plans or research proposals that take a long time to generate value are considered infeasible. (I claim that people should think about ways to make their current trajectory radically faster— e.g., if someone is an undergraduate planning a CS PhD, they may want to consider alternative ways to get research expertise more quickly).
    6. There is widespread disagreement in AI safety about which research agendas are promising, what the core problems in AI alignment are, and how people should get started in AI safety.
    7. There are several programs designed to help people get started in AI safety. Examples include SERI-Mats (for alignment research & theory), MLAB (for ML engineering), the ML Safety Scholars Program (for ML skills), AGI Safety Fundamentals (for AI alignment knowledge), PIBBS (for social scientists), and the newly-announced Philosophy Fellowship. (I suggest people keep point #6 in mind, though, and not assume that everything they need to know is captured in a well-packaged Program or Reading List).
    8. There are not many senior AIS researchers or AIS mentors, and the ones who exist are often busy. (I claim that the best way to “get started in AI safety research” is to apply for a grant to spend ~1 month reading research, understanding the core parts of the alignment problem, evaluating research agendas, writing about what you’ve learned, and visiting an EA hub).
    9. People can apply for grants to skill-up in AI safety. You do not have to propose an extremely specific project...
    6 min
  • EA - [Link post] Optimistic “Longtermism” Is Terrible For Animals by BrianK
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: [Link post] Optimistic “Longtermism” Is Terrible For Animals, published by BrianK on September 6, 2022 on The Effective Altruism Forum.
    Oxford philosopher William MacAskill’s new book, What We Owe the Future, caused quite a stir this month. It’s the latest salvo of effective altruism (EA), a social movement whose adherents aim to have the greatest positive impact on the world through use of strategy, data, and evidence. MacAskill’s new tome makes the case for a growing flank of EA thought called “longtermism.” Longtermists argue that our actions today can improve the lives of humans way, way, way down the line — we’re talking billions, trillions of years — and that in fact it’s our moral responsibility to do so.
    In many ways, longtermism is a straightforward, uncontroversially good idea. Humankind has long been concerned with providing for future generations: not just our children or grandchildren, but even those we will never have the chance to meet. It reflects the Seventh Generation Principle held by the indigenous Haudenosaunee (a.k.a. Iroquois) people, which urges people alive today to consider the impact of their actions seven generations ahead. MacAskill echoes the defining problem of intergenerational morality — people in the distant future are currently “voiceless,” unable to advocate for themselves, which is why we must act with them in mind. But MacAskill’s optimism could be disastrous for non-human animals, members of the millions of species who, for better or worse, share this planet with us.
    Read the rest on Forbes.
    Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.
    2 min
  • AF - AI-assisted list of ten concrete alignment things to do right now by Luke H Miles
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: AI-assisted list of ten concrete alignment things to do right now, published by Luke H Miles on September 7, 2022 on The AI Alignment Forum.
    Background
    So I'm thinking that AI-assisted summarization, math, bug-finding in code, and logical-error finding in writing is at a point where it is quite useful, if we can improve the tooling/integration a little bit.
    In code I've found it helpful to comment out some lines and write // WRONG: above them and // FIXED VERSION: below them then let copilot try a few things.
    For writing you could take a paragraph excerpt and write a critique post: "John Smith wrote '...' This immediately strikes me as absurd because"
    Imagine you were doing chemistry research in uh 1650 and had direct immediate written feedback from uh Robert Boyle on directions to pursue, dead ends, errors, etc except that 75% of the time he says something backwards or is just pattern matching. I think you might still do much better work than you would've without noisy-Boyle.
    I'm not aware of anyone trying to actually use LLMs for meaningful writing/thinking assistance so I decided to try. I wrote the below text in about an hour. Consider this merely a demonstration that you can get a decent amount of semi-meaningful content in the right direction quite quickly. It's rare you can expect that much from someone.
    List
    Rob Bensinger gave this suggestion in a comment:
    I think that this is a really good exercise that more people should try: Imagine that you’re running a project yourself that’s developing AGI first, in real life. Imagine that you are personally responsible for figuring out how to make the thing go well. Yes, maybe you’re not the perfect person for the job; that’s a sunk cost. Just think about what specific things you would actually do to make things go well, what things you’d want to do to prepare 2 years or 6 years in advance, etc.
    Try to think your way into near-mode with regard to AGI development, without thereby assuming (without justification) that it must all be very normal just because it’s near. Be able to visualize it near-mode and weird/novel. If it helps, start by trying to adopt a near-mode, pragmatic, gearsy mindset toward the weirdest realistic/plausible hypothesis first, then progress to the less-weird possibilities.
    I took the above prompt together with below underlined text and used BLOOM to generate ten concrete things to do right now. For each item, I generated three completions and picked my favorite one then did bit of editing. I had trouble editing the math into something reasonable so consider it creative inspiration.
    (Repeat, my writing / prompting is underlined and AI completion is not:)
    Here, I will try it right now. I'm in charge of OpenMind and it is clear from our rate of progress that we will have AGI in less than 24 months. It's been decided that we will build it and deploy it but I have some influence on additional efforts we can take to reduce the risk. Here's ten things I would try:
    For starters, I would try to test each new model in an isolated (ie not internet connected) environment with constrained IO channels for a month before deploying it.
    I would also use any and all interpretability methods available to inspect the model and try to predict off-distribution behavior. If I had the sway, I might run a red-team blue-team contest to see how good we are at predicting off-distribution.
    There are some old posts about safety implications of on-policy vs off-policy RL that I read but did not fully digest. I would go back and absorb those more fully and see if the distinction can be adapted to our use case. (Off-policy was much better in some ways if I recall correctly)
    I'm pretty strongly in favor of the near miss principle: if an adversary or an incompetent user almost succeeds in exploiting the system, then we should log t...
    7 min