The Nonlinear Library

The Nonlinear Library

By The Nonlinear FundEducation
Download on the App Store

All The Nonlinear Library episodes

  • EA - Teaching EA through superhero thought experiments by Geoffrey Miller
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Teaching EA through superhero thought experiments, published by Geoffrey Miller on October 28, 2022 on The Effective Altruism Forum.
    Concrete, thought-provoking discussion questions can often spark more interest in new ideas (such as EA principles) than abstract arguments, moral preaching, or theoretical manifestos can. This is true in college seminars, but more generally in small social gatherings of any type.
    When I've taught my 'Psychology of Effective Altruism' class (syllabus here), it's sometimes hard to get ordinary college students interested in abstract Effective Altruism ideas. I teach at a large American state university that is not very selective in student admissions, so there is a wide range of cognitive abilities and curiosity levels in the college juniors and seniors who take my EA class. They sometimes struggle to follow abstract presentations of key EA concepts such as scope-sensitivity, tractability, neglectedness, charity effectiveness, utility, sentience, and longtermism.
    But they all love superhero movies. Whatever their religious affiliation, they're all familiar with the DC and Marvel pantheons of demi-gods. For better or worse, these superhero pantheons are at the heart of modern global entertainment culture. In a politically polarized era when people can't agree on much, and tend to stay within their partisan news media bubbles, superhero movies and TV series offer some rare common ground for thinking about issues of power, altruism, existential risk, counterfactuals, moral dilemmas, etc.
    So, I've found it useful to provoke class discussions with some superhero thought experiments, such as: "If you had all of Superman's superpowers for 24 hours, and you wanted to do the most good in the world during that one day, what would you do?" (For a sample of about 150 replies to this question on Twitter (from ordinary followers, not from college students), see here.)
    This question usually provokes immediate and spirited discussion. Almost all students are familiar with Superman's imaginary superpowers, and almost all accept the premise that Superman is a good guy with good intentions.
    The most frequent initial responses usually involve geopolitical vigilante justice, on the principle that the fastest way to do good is to eliminate bad guys -- through killing them, jailing them, or otherwise neutralizing them. So, many students will start off saying 'Superman should simply kill foreign leader X', where X is whoever the American news media is currently demonizing as the Global Bad Guy. However, other students will usually point out that political assassinations often create martyrs, generate adverse publicity, and provoke blowback, so the longer-term effects may be neutral or negative. This can lead to a good discussion of unintended consequences, counterfactuals, moral legitimacy, public sentiment, and the global catastrophic risks of geopolitical instability.
    Other students will sometimes suggest that our hypothetical EA Superman should simply do what classical Superman has done ever since the comics started in 1939 -- monitor the news, look for people in distress, and go save the individuals and small groups who can be saved. This typically leads to a good discussion of scope-sensitivity: why should Superman save a few people at a time through heroic actions, when he might be able to save many more through delivering water, food, or medicines to the needy? Or through infrastructure projects like digging canals, tunnels, and harbors, building dikes and dams, or delivering metal-rich asteroids to Earth? Students enjoy debating which kinds of interventions would be the best use of Superman's time -- and the 24-hour time limit in the question makes the opportunity costs of each intervention salient.
    It's also easy to nudge these discussions into epistem...
    8 min
  • EA - Using artificial intelligence (machine vision) to increase the effectiveness of human-wildlife conflict mitigations could benefit WAW by Rachel Norman
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Using artificial intelligence (machine vision) to increase the effectiveness of human-wildlife conflict mitigations could benefit WAW, published by Rachel Norman on October 28, 2022 on The Effective Altruism Forum.
    1. Overview
    This report explores using artificial intelligence (AI) to increase the effectiveness of human-wildlife conflict (HWC) mitigations in order to benefit wild animal welfare (WAW). Two concrete examples are providing more funding, research and direct work into reducing fatalities due to 1) collisions between bats and wind turbines, and 2) culling crop-raiding starlings. The report aims merely to raise awareness of this topic and introduce the idea for discussion, but not yet strongly suggest it is a cost-effective intervention on par with other interventions - see uncertainties, limitations, and potential for harm.
    What's the problem profile?
    HWC is increasing due to human expansions and climate change, (Gross et al., 2021) and is starting to be considered in government strategies and policy. The expected future impact of innovative and effective solutions to HWC could be even larger than currently appreciated.
    Lethal control or other methods which significantly impact animal welfare are still widely used (such as culling), despite preventative non-lethal strategies growing in more recent wildlife management approaches.
    Currently deployed AI systems directed towards HWC could be expanded further within the next 10-20 years as they become more reliable, more effective, and cheaper. We should not assume they will prioritize WAW concerns, or be widely used for animals of WAW concern, so this should be embedded before they are potentially rolled out at scale.
    There are already companies working on AI solutions for specific problems involving endangered species, such as protected areas using AI assisted technology for poacher detection. There is already proof-of-concept of an NGO-backed early warning AI system, ‘WildEyes’, with this type of solution being invested in by a local governmental department in Tamil Nadu, India. Buy-in from a range of stakeholders (especially when it benefits humans and profits too) offers a way in with conservationists and researchers who may not otherwise consider WAW. Research and development (R&D) on AI-assisted HWC mitigations would likely attract researchers who would not otherwise consider or be motivated by WAW concerns.
    What should we be doing differently?
    A very tentative theory of change: if machine vision-based methods prevent HWC, they could be adopted, even on a small scale helps drop prices allowing for systems to be more widely adopted leads to more support and R&D continued price drops and adoption could create space for legislation to ban harmful or lethal methods of animal control preventing HWC could reduce apathy and antagonism towards “problem species” and make it easier for people to consider the welfare of animals, while also directly reducing negative WAW effects of HWC.
    This report highlights two examples of HWC where advocates could influence AI-assisted mitigation to directly affect substantial numbers of animals, and spread welfare considerations in software and norms:
    Wind turbine collisions are a leading anthropogenic cause of bat deaths and cause a significant number of bird deaths (600,000 to 949,000 bats and 140,000 to 679,000 birds annually in North America). We should expect fatalities to increase due to expansions in wind power.
    Culling of crop-raiding species. In one year, the USDA’s Wildlife Services culled 1,028,642 European starlings responsible for agricultural crop damage, because other mitigations are ineffective. Despite this, starlings still cause extensive damage each year. More effective mitigation measures would hold value and could prevent culls.
    There are a number of r...
    1 hr
  • AF - Some Lessons Learned from Studying Indirect Object Identification in GPT-2 small by KevinRoWang
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Some Lessons Learned from Studying Indirect Object Identification in GPT-2 small, published by KevinRoWang on October 28, 2022 on The AI Alignment Forum.
    To learn more about this work, check out the paper. We assume general familiarity with transformer circuits.
    Intro:
    There isn’t much interpretability work that explains end-to-end how a model is able to do some task (except for toy models). In this work, we make progress towards this goal by understanding some of the structure of GPT-2 small “in the wild” by studying how it computes a simple natural language task.
    The task we investigate is what we call indirect object identification (IOI), where sentences like “When John and Mary went to the store, John gave a drink to” should be completed with “Mary.”
    We discovered the structure of a circuit of 26 attention heads grouped into 7 main classes, the largest end-to-end attempt to reverse engineer how a LM computes a natural behavior (to our knowledge). There is still much missing from our explanation, however, and our explanation doesn’t go to the parameter level.
    Besides discovering the particular circuit shown above, we gained some interesting insights about low-level phenomena arising inside language models. For example, we found attention heads communicating with pointers (sharing the location of a piece of information instead of copying it). We also identified heads compensating for the loss of function of other heads, and heads contributing negatively to the correct next-token prediction. We're excited to see if these discoveries generalize beyond our case study.
    Since explanations of model behavior can be confused or non-rigorous, we used our knowledge to design adversarial examples. Moreover, we formulate 3 quantitative criteria to test the validity of our circuit. These criteria partially validate our circuit but indicate that there are still gaps in our understanding.
    This post is a companion post to our paper where we share lessons that we learned from doing this work and describe some of Redwood’s interpretability perspectives. We share high-level takeaways and give specific examples from the work to illustrate them.
    Goals of Interpretability for this Investigation
    In this kind of mechanistic interpretability work, we tend to use the circuits abstraction. If we think of a model as a computational graph where nodes are terms in its forward pass (neurons, attention heads, etc) and edges are the interactions between those terms (residual connections, attention, etc), a circuit is a subgraph of the model responsible for some behavior.
    Note that our work is slightly different from Chris Olah's/Anthropic’s idea of a circuit in that we investigate this circuit on a specific distribution (instead of the entire distribution of text) and we also don’t attain an understanding of the circuit at the parameter-level.
    Structure: Having all the nodes and edges
    One kind of valuable interpretability insight is to ensure that we have the correct subgraph, which is one of the main goals in this work.
    We formulate three main quantitative criteria to measure progress towards this goal. These criteria rely on the idea of "knocking out" or turning off a node from the computational graph by replacing its activation by its mean on a distribution where the IOI task is not present. (Note that we now believe that our causal scrubbing algorithm provides a more robust way to validate circuits.)
    Faithfulness:
    Intuition: The circuit performs the task as well as the model.
    Computation: A model with everything but the circuit knocked out should achieve the same performance according to some task benchmark.
    Completeness:
    Intuition: The circuit contains all the nodes that are important for the task. (Note that completeness implies faithfulness).
    Computation: For every subset of nodes in the ci...
    16 min
  • EA - Join the interpretability research hackathon by Esben Kran
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Join the interpretability research hackathon, published by Esben Kran on October 28, 2022 on The Effective Altruism Forum.
    TLDR; Participate online or in-person in London, Aarhus, and Tallinn on the weekend 11th to 13th November in a fun and intense AI safety research hackathon focused on interpretability research. We invite mid-career professionals to join but it is open for everyone (also no-coders) and we will create starter code templates to help you kickstart your team’s projects. Join here.
    Below is an FAQ-style summary of what you can expect (navigate it with the table of contents on the left).
    What is it?
    The Interpretability Hackathon is a weekend-long event where you participate in teams (1-6) to create interesting and fun research. You submit a PDF report that summarizes and discusses your findings in the context of AI safety. These reports will be judged by our panel and you can win up to $1,000!
    It runs from 11th Nov to 13th Nov (in two weeks) and you’re welcome to join for a part of it (see further down). We get an interesting talk by an expert in the field and hear more about the topic.
    Everyone can participate and we encourage you to join especially if you’re considering AI safety from another career . We prepare templates for you to start out your projects and you’ll be surprised what you can accomplish in just a weekend – especially with your new-found friends!
    Read more about how to join, what you can expect, the schedule, and what previous participants have said about being part of the hackathon below.
    Where can I join?
    You can join the event both in-person and online but everyone needs to make an account and join the jam on the itch.io page.
    The in-person locations include the LEAH offices in London right by UCL, Imperial, King’s College, and London School of Economics (link); Aarhus University in Aarhus, Denmark (link), and Tallinn, Estonia (link). The virtual event space is on GatherTown (link).
    Everyone should join the Discord to ask questions, see updates and announcements, find team members, and more. Join here.
    What are some examples of interpretability projects I could make?
    You can check out a bunch of interesting, smaller interpretability project ideas on AI Safety Ideas such as reconstructing the input from neural activations, evaluating the alignment tax of interpretable models, or making models’ uncertainty interpretable.
    Other examples of practical projects can be to find new ways to visualize features in language models, such as Anthropic has been working on, distilling mechanistic interpretability research, create a demo for a much more interpretable language model, or map out how a possible interpretable AGI might look with our current lens.
    You can also do projects in explainability about how much humans understand why the outputs of language models look the way they do, how humans see attention visualizations, or maybe even the interpretability of humans themselves and take inspiration from the brain and neuroscience.
    Also check out the results from the last hackathon to see what you might accomplish during just one weekend. The judges were really quite impressed with the full reports given the time constraint! You can also read the complete projects here.
    Inspiration
    Redwood Research's interpretability tools: http://interp-tools.redwoodresearch.org/
    The activation atlas:/
    The Tensorflow playground:/
    The Neural Network Playground (train simple neural networks in the browser):/
    Visualize different neural network architectures: http://alexlenail.me/NN-SVG/index.html
    Digestible research
    Distill publication on visualizing neural network weights
    Andrej Karpathy's "Understanding what convnets learn"
    Looking inside a neural net
    You can also see more on the resources page.
    Why should I join?
    There’s loads of reasons to join! Here are ju...
    9 min
  • EA - A Potential Cheap and High Impact Way to Reduce Covid in the UK this Winter by Lawrence Newport
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: A Potential Cheap and High Impact Way to Reduce Covid in the UK this Winter, published by Lawrence Newport on October 28, 2022 on The Effective Altruism Forum.
    Brief problem summary: Under current plans, millions of adults in the UK this winter will be unable to get a covid booster, even if they are willing to pay for it, because there will be no option for them to take the vaccine in the NHS rollout, and they will not be allowed to pay privately for a bivalent booster. That means overall population immunity will be lower and will lead to more covid cases than otherwise, which will increase the strains on NHS services this winter.
    This is an area in which EAs might have a potentially very large impact for small investment by creating a simple website to encourage people who would be willing to pay for a booster to give their email address. That list could then be used to convince Pfizer or another company to choose to make covid booster vaccines privately available in the UK, just as they will be in the US from January 2023 and as flu vaccines currently are in the UK.
    The Issue
    Pfizer's bivalent vaccine (vaccinating against both the original covid and Omicron variants) is being rolled out in the UK via the NHS. The original vaccination and initial booster programme in the UK started with the most vulnerable and frontline health workers, and expanded down risk profiles until all adults in the UK had been offered the opportunity to have a vaccine and a booster.
    However, the plan is more limited for Pfizer's bivalent booster vaccine. This winter the UK has opted for a policy which will see only the most vulnerable and frontline health workers being offered the vaccine. Unlike flu vaccines, where anyone can walk into a pharmacy and pay a small fee for a flu vaccine if they are not entitled to a free vaccine, that will not be possible for covid boosters. To put this in perspective:
    Number of people who were vaccinated in the UK:
    First dose: ~54M
    Second dose: ~51M
    Booster: ~40M
    Those to be offered a bivalent booster:
    Estimated total to be offered a bivalent booster: 26M
    In other words, even with 100% uptake of the booster, this leaves a potentially willing 28M adults without even the option of a booster from the NHS or privately.
    It is of course entirely right to start rollouts to the most vulnerable – and to expand those groups as widely as possible. The issue is that there is no option, other than travel to the US, for the majority of willing UK adults to receive a bivalent vaccine. It is clear that, all other things being equal, more population immunity is better. Waning immunity in millions of UK adults will reduce ongoing pandemic mitigation. That is particularly a problem when the winter burden of covid and flu on the NHS is expected to be large, impairing outcomes for treatment of other illness.
    I understand that there is no longer any shortage of vaccines; vaccine takeup in the UK has been lower than expected. Although it might be better for the NHS to provide boosters free of charge to all who want them, the Government has decided not to do that. But in any case, it would be strictly better for all concerned, including those who do not get a booster, if the number of covid cases were reduced by making it possible for people to pay a modest fee for a booster if they are not entitled to a free one.
    Pfizer has stated that it will be selling vaccines in the US from 2023 when the US Government stops providing them free of charge. In the UK flu vaccines are sold privately and are administered in pharmacies by trained staff members. Currently, there is no public argument or expectation that the bivalent vaccine should be available for purchase in the UK.
    However, there may be a solution. Based on confidential discussions with sources that I cannot name but who I believe ...
    7 min
  • EA - On retreats: nail the 'vibes' and venue by Vaidehi Agarwalla
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: On retreats: nail the 'vibes' and venue, published by Vaidehi Agarwalla on October 28, 2022 on The Effective Altruism Forum.
    This year I attended a few retreats with different goals and audiences (mostly for people already fairly involved in EA). This post lists some observations and lessons I took away from them about how I would want to run a retreat.
    I’ve probably written some suggestions with more confidence than is warranted - I have strong opinions that are weakly held. For brevity, I've not always delved into my reasoning for every specific point, but I'm happy to expand in the comments! I’ve shared early versions of this advice with a couple of people running retreats, and it seemed to be helpful.
    Not all of this advice will apply to any one retreat, I think the best way to read this is to take what makes sense given your goals and use those. I think most of these suggestions are useful for community retreats; and maybe ~30-60% are useful for professional retreats. I'd be really excited for more people to share their reflections on what's worked and what hasn't.
    Nail the 'vibes'
    Your goal is (probably) to help people make friends. For this reason, the vibes matter. Take time to observe how the attendees are interacting, and regularly ask the most perceptive or senior attendees how it’s going. It’s important to preserve participants’ intention, energy, and eagerness to participate proactively.
    Go light with the schedule & be flexible with content
    Creating a schedule is always difficult, but when in doubt I advocate for cutting things (ruthlessly). I think that attendees will ultimately care more about the overall flow and atmosphere than that extra session.
    Have a shorter ‘work’ day of (high quality) scheduled events to leave enough time for chilling and socializing (e.g., 10am-5pm). This will give participants time to make friends and prevent them getting too tired. That being said - make sure the content you do have is excellent.
    Be willing and able to pivot - don’t be scared to throw out the schedule and do something else if you feel participants aren't resonating with the content you have, or someone makes a suggestion that seems good.
    Factor in downtime for attendees to recover from travel.
    Make people feel comfortable and relaxed
    Try to avoid people coming late: It can feel bad to miss the first day. If people must come late, batch latecomers and have someone give them an orientation and introduce them to other participants. Consider introducing latecomers to someone beforehand, so they know at least one non-organizer when they arrive.
    Check in with folks: Organizers can identify and periodically check on (e.g.) the 30% of participants who are most likely to feel out of place: for example, newer EAs or people who are less well-connected.
    Look out for each other: The German funconference had ‘awareness facilitators’ who would support you in case of physical & mental health problems or if you felt like someone crossed your boundaries, and who would keep information confidential. I thought this was a really good thing to have.
    Help people get to know each other before the retreat: You can share a ‘names and faces’ deck of all the attendees for people to read, and also print them and put them out in the common areas.
    Consider activities which allow people to seek help and be vulnerable, if it makes sense: I have found Hamming Circles during retreats to be quite helpful and a positive experience (and others who've participated say the same). However, they can be very intense and it's likely that I had a positive experience because I only opted into them when I already trusted the people around me, felt open to the experience, and when the activity was being run by someone who was experienced in running these kinds of activities.
    Think about ways to facilitate...
    9 min
  • EA - The African Movement-building Summit by jwpieters
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: The African Movement-building Summit, published by jwpieters on October 28, 2022 on The Effective Altruism Forum.
    The African Movement-building Summit will be a chance to interact with people in the EA community from across the continent and work on growing the movement in Africa. You can apply here.
    Date and Location:
    Date: 1-5 December 2022
    Location: Devon Valley Hotel, Cape Town, South Africa
    About this retreat:
    The African EA Summit will be a ~30-person retreat primarily for members of the EA community in Africa who are interested in movement-building for EA or related cause areas. We also encourage experienced community members working on projects relevant to the African context to attend.
    We will cover room, board, and reasonable travel expenses (incl. visa applications) for all attendees. This is possible thanks to support from CEA's community events program.
    Goals & Agenda:
    The 3-day retreat will focus on both broad EA movement-building, as well as issues specific to the African context. We hope to:
    Establish better coordination between African EA leaders and movement-builders
    Form strong social connections amongst highly engaged EAs in Africa
    Help key international stakeholders better understand how the African EA community can be supported
    The summit will primarily focus on in-depth discussions and strengthening connections. There will also be workshops and talks run by the organisers and other attendees. We encourage suggestions for activities and discussion topics in the comments or in your application.
    Application:
    You can apply by filling out this application form by Sunday 6th November (midnight SAST). We highly recommend that applicants who need visas apply ASAP. We will try to fast-track decisions for these applicants to allow time for visa processing.
    Contact:
    I (Jordan Pieters) am the primary organiser for this event. If you have questions about the retreat, you can email me at [email protected] or leave a comment on this post.
    Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.
    3 min
  • EA - GiveWell should fund an SMC replication by Seth Ariel Green
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: GiveWell should fund an SMC replication, published by Seth Ariel Green on October 28, 2022 on The Effective Altruism Forum.
    Abstract: This essay argues that the evidence supporting GiveWell’s top cause area – Seasonal Malaria Chemoprevention, or SMC – is much weaker than it appears at first glance and would benefit from high-quality replication. Specifically, GiveWell’s assertion that every $5,000 spent on SMC saves a life is a stronger claim than the literature warrants on three grounds: 1) the effect size is small and imprecisely estimated; 2) co-interventions delivered simultaneously pose a threat to external validity; and 3) the research lacks the quality markers of the replication/credibility revolution. I conclude by arguing that any replication of SMC should meet the standards of rigor and transparency set by GiveDirectly, whose evaluations clearly demonstrate contemporary best practices in open science.
    1. Introduction: the evidence for Seasonal Malaria Chemoprevention
    GiveWell currently endorses four top charities, with first place going to the Malaria Consortium, a charity that delivers Seasonal Malaria Chemoprevention (SMC). GiveWell provides more context on its Malaria Consortium – Seasonal Malaria Chemoprevention page and its Seasonal Malaria Chemoprevention intervention report. That report is built around a Cochrane review of seven randomized controlled trials (Meremikwu et al. 2012). GiveWell discounts one of those studies (Dicko et al. 2008) for technical reasons and includes an additional trial published later (Tagbor et al. 2016) in its evidence base.
    No new research has been added since then, and GiveWell’s SMC report was last updated in 2018. It appears as though GiveWell treats the question of “does SMC work?” as effectively settled.
    I argue that GiveWell should revisit its conclusions about SMC and should fund and/or oversee a high-quality replication study on the subject. While there is very strong evidence that SMC prevents the majority of malaria episodes, “including severe episodes” (Meremikwu et al. 2012, p. 2), GiveWell’s estimate that every $5,000 of SMC saves a life in expectation is shaky on three grounds related to research quality: 1) the underlying effect size is small, relative to the sample size, and statistically imprecise; 2) SMC is often tested in places receiving other interventions, which threatens external validity because we don’t know which set of interventions bests maps onto the target population; and 3) the evidence comes from studies that are pre-credibility revolution, and therefore lack quality controls such as detailed pre-registration, open code and data, and sufficient statistical power.
    2. Three grounds for doubting the relationship between SMC and mortality
    2.1 The effect size is small and imprecisely estimated
    Across an N of 12,589, Meremikwu et al. record 10 deaths in the combined treatment groups and 16 in the combined control groups. Subtracting the one study that GiveWell discounts and including the one they supplement with, we arrive at 10 deaths for treatment and 15 for control. As the authors note, “the difference was not statistically significant” (p. 12), “and none of the trials were adequately powered to detect an effect on mortality.However, a reduction in death would be consistent with the high quality evidence of a reduction in severe malaria” (p. 4).
    Overall, the authors conclude, SMC “probably prevents some deaths,” but “[l]arger trials are necessary to have full confidence in this effect” (p. 4).
    GiveWell forthrightly acknowledges this on its SMC page, and provides reasons why it believes SMC reduces morality despite " limited evidence." This is laudably transparent, but the question is foundational to all of GiveWell's subsequent analyses of SMC. Especially given the organization's strong fundin...
    10 min
  • EA - EA-Aligned Political Activity in a US Congressional Primary: Concerns and Proposed Changes by Carolina EA
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: EA-Aligned Political Activity in a US Congressional Primary: Concerns and Proposed Changes, published by Carolina EA on October 28, 2022 on The Effective Altruism Forum.
    Summary
    This post shares my observations of the approach of the EA-aligned political action committee Protect our Future (POF) in a US Congressional Primary in North Carolina. Writing anonymously, I share my experience talking informally with roughly 20 individuals and observing the race through the media and as a citizen. I express concerns with POF's limited explanation of the rationale for their spending in this race, their limited explanation of why they selected their endorsed candidate, and advertisements that did not address their stated rationale for supporting the candidate. I explain my concern that the association of EA with large scale political spending and limited transparency could have harmful reputational effects. (This post is meant to provide constructive comments, and it should not be taken as a critique of POF's intentions or broader impact.)
    Finally, I propose a few changes that POF or other groups could make moving forward to mitigate possible harm, such as publishing the criteria used to evaluate candidates, providing a specific rationale for each endorsed candidate, and providing additional transparency on funding sources.
    Introduction
    I am writing this post to share my firsthand experience observing the Democratic primary in the 4th Congressional District of North Carolina this past spring. The political action committee (PAC) Protect our Future (POF), funded predominantly by prominent EA Sam Bankman-Fried, spent heavily (roughly $1 million) in this race to support the successful nomination of current NC State Senator Valerie Foushee. The 4th District is overwhelmingly Democratic, and Foushee has >99% odds of winning the seat in the general election on November 8th, according to FiveThirtyEight’s forecast as of October 26th.
    I decided to write this post, my first on the EA Forum, in order to share concerns that came out of my observation of this race. Given the highly anecdotal nature of my experiences, and the success of Foushee’s campaign so far, these critiques should be viewed with some skepticism. Nonetheless, as EAs become a larger force in politics, I hope that these concerns and the accompanying suggestions could be useful for EAs engaging in large scale political spending.
    I have no prior connection with or relationship to Sam Bankman-Fried or Protect our Future. I hope that these critiques and suggestions will be received as good faith concerns about specific practices, rather than any sort of personal attack or criticism of POF’s goals, intentions, or broader impact.
    Personal background
    I work in state government/politics in North Carolina. I’m worried that writing this post with my name attached to it could negatively impact my work, so I’ve decided to stay anonymous. I did not work or volunteer for any of the candidates in this race, though I am part of a group that endorsed one of the candidates (Nida Allam). I had no role in that endorsement. I was uncertain about who I'd vote for until a few weeks before the election, but I did end up voting for Nida Allam. Those factors could certainly influence how I view this situation.
    My experience of the election
    The concerns expressed in this post stem from conversations I had with political professionals and politically engaged but nonprofessional individuals in the run up to and the aftermath of this primary election. I also read a variety of local and national news articles that examined the role of money in the election, including the financial support from POF. In total, I talked casually with roughly 10 political professionals and 10 nonprofessionals about these topics. I did not necessarily discuss all aspects o...
    14 min
  • EA - Prizes for ML Safety Benchmark Ideas by Joshc
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Prizes for ML Safety Benchmark Ideas, published by Joshc on October 28, 2022 on The Effective Altruism Forum.
    “If you cannot measure it, you cannot improve it.” – Lord Kelvin (paraphrased)
    Website: benchmarking.mlsafety.org – receiving submissions until August 2023.
    ML Safety lacks good benchmarks, so the Center for AI Safety is offering $50,000 - $100,000 prizes for benchmark ideas (or full research papers). We will award at least $100,000 total and up to $500,000 depending on the quality of submissions.
    What kinds of ideas are you looking for?
    Ultimately, we will are looking for benchmark ideas that motivate or advance research that reduces existential risks from AI. To provide more guidance, we’ve outlined four research categories along with example ideas.
    Alignment: building models that represent and safely optimize difficult-to-specify human values.
    Monitoring: discovering unintended model functionality.
    Robustness: designing systems to be reliable in the face of adversaries and highly unusual situations.
    Safety Applications: using ML to address broader risks related to how ML systems are handled (e.g. for cybersecurity or forecasting).
    See Open Problems in AI X-Risk [PAIS #5] for example research directions in these categories and their relation to existential risk.
    What are the requirements for submissions?
    Datasets or implementations are not necessary, though empirical testing can make it easier for the judges to evaluate your idea. All that is required is a brief write-up (guidelines here). How the write-up is formatted isn’t very important as long as it effectively pitches the benchmark and concretely explains how it would be implemented. If you don’t have prior experience designing benchmarks, we recommend reading this document for generic tips.
    Who are the judges?
    Dan Hendrycks, Paul Christiano, and Collin Burns.
    If you have questions, they might be answered on the website, or you can post them here. We would also greatly appreciate it if you helped to spread the word about this opportunity.
    Thanks to Sidney Hough and Kevin Liu for helping to make this happen and to Collin Burns and Akash Wasil for feedback on the website. This project is supported by the Future Fund regranting program.
    Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.
    3 min