The Nonlinear Library

The Nonlinear Library

By The Nonlinear FundEducation
Download on the App Store

All The Nonlinear Library episodes

  • EA - Casting the Decisive Vote by Toby Ord
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Casting the Decisive Vote, published by Toby Ord on March 27, 2023 on The Effective Altruism Forum.
    The moral value of voting is a perennial topic in EA. This piece shows that in any election that isn't a forgone conclusion, the chance of your vote being decisive can't be much lower than 1 in the number of voters. So voting will be worth it around the point where the value your preferred candidate would bring to the average citizen exceeds the cost of you voting.
    What is the chance your vote changes the outcome of an election? We know it is low, but how low?
    In particular, how does it compare with an intuitive baseline of a 1 in n chance, where n is the number of voters? This baseline is an important landmark not only because it is so intuitive, but because it is roughly the threshold needed for voting to be justified in terms of the good it produces for the members of the community (since the total benefit is also going to be proportional to n).
    Some political scientists have tried to estimate it with simplified theoretical models involving random voting. Depending on their assumptions, this has suggested it is much higher than the baseline — roughly 1 in the square root of n (Banzhaf 1965) — or that it is extraordinarily lower — something like 1 in 10^2659 for a US presidential election (Brennan 2011).
    Statisticians have attempted to determine the chance of a vote being decisive for particular elections using detailed empirical modelling, with data from previous elections and contemporaneous polls. For example, Gelman et al (2010) use such a model to estimate that an average voter had a 1 in 60 million chance of changing the result of the 2008 US presidential election, which is about 3 times higher than the baseline.
    In contrast, I’ll give a simple method that depends on almost no assumptions or data, and provides a floor for how low this probability can be. It will calculate this using just two inputs: the number of voters, n, and the probability of the underdog winning, p_u.
    The method works for any two-candidate election that uses simple majority. So it wouldn’t work for the US presidential election, but would work for your chance of being decisive within your state, and could be combined with estimates that state is decisive nationally. It also applies for many minor ‘elections’ you may encounter, such as the chance of your vote being decisive on a committee.
    We start by considering a probability distribution over what share of the vote a candidate will get, from 0% to 100%. In theory, this distribution could have any shape, but in practice it will almost always have a single peak (which could be at one end, or somewhere in between). We will assume that the probability distribution over vote share has this shape (that it is ‘unimodal’) and this is the only substantive assumption we’ll make.
    We will treat this as the probability distribution of the votes a candidate gets before factoring in your own vote. If there is an even number of votes (before yours) then your vote matters only if the vote shares are tied. In that case, which way you vote decides the election. If there is an odd number of votes (before yours), it is a little more complex, but works out about the same: Before your vote, one candidate has one fewer vote. Your vote decides whether they lose or tie, so is worth half an election. But because there are two different ways the candidates could be one vote apart (candidate A has one fewer or candidate B has one fewer), you are about twice as likely to end up in this situation, so have the same expected impact. For ease of presentation I’ll assume there is an even number of voters other than you, but nothing turns on this.
    (In real elections, you may also have to worry about probabilistic recounts, but if you do the analysis, these don’t substantivel...
    15 min
  • LW - The salt in pasta water fallacy by Thomas Sepulchre
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: The salt in pasta water fallacy, published by Thomas Sepulchre on March 27, 2023 on LessWrong.
    There is a particular fallacy I encounter often, and I don't know if it has a name already. I call it the salt in pasta water fallacy. In this post I will present the fallacy, discuss why I believe it is a fallacy, and give a concrete example of it happening.
    Salt in pasta water
    The archetypical example of the fallacy is the following claim: Adding salt to pasta water reduces cooking time.
    The proof for this statement can be described as follows:
    When cooking pasta, the higher the temperature of the water, the lower the cooking time
    The temperature of the water is its boiling point
    Adding salt to water increases its boiling point
    (3+2) Adding salt to water increases the temperature
    (4+1) Adding salt to water decreases the cooking time
    "All of those points seem very reasonable", you may be wondering, "where is the fallacy?" Is there some sneaky chemistry phenomenon going on?
    AFAIK no, there is none. My objection goes a bit deeper.
    Let's do the math
    First of all, can we estimate how big the effect is? To do this, we need to know three things:
    How does the temperature affect cooking time? I found a paper from which I derive the following first-order approximation: each extra °C decreases the cooking time of 30 seconds.
    How does salt affect the temperature? I found another paper which claims that the boiling point is raised one degree for every mole of salt (i.e. for every 58g) per liter.
    How much salt does one typically add to pasta water? This is more personal, but it seems that about 5g/l is standard
    So, now we simply multiply all of this and obtain... 2.6s. By adding salt to pasta water, one saves 2.6 seconds of cooking time.
    Where is the fallacy again?
    You may be wondering where I am going. The initial statement seemed true, I just added numbers, so what? Where is the fallacy.
    The fallacy, I claim, is that 2.6s are nothing. Adding salt does not decrease cooking time. It doesn't increase it either, it just does nothing (wrt cooking time at least). This may seem weird to you, and I will try to detail it a bit more. The fallacy, I believe, lies in some play-on-word.
    Assume for example that my friend Bob and I are in front of a cooking pot, waiting for the pasta to be ready. And Bob says "adding salt will reduce cooking time". What does this sentence mean?
    Does it only mean that adding salt will reduce cooking time?
    Does it mean that we should add salt, because it will reduce cooking time (and we want to reduce cooking time because we are hungry/bored/something else)?
    I want to argue that, in this context, it has to mean (2). And, in this context, I want to claim that Bob is wrong, such a tiny reduction in cooking time doesn't justify doing anything, or even arguing about it.
    Now comes the weird part. Assume that I do the math in front of Bob, and we both agree that the math is correct and it won't have an effect greater than 3s. Bob could still claim "Ok, so adding salt does reduce cooking time, you are being stubborn". Bob is now claiming he meant (1) the whole time, the proposition adding salt reduces cooking time is true. And I'd like to claim that, no, he was using meaning (2), and just backed to meaning (1) because he was wrong.
    A concrete example
    This winter, in France (and, I assume everywhere else in Europe), the government wanted the whole country to save energy (this had to do with the war in Ukraine and the sudden drop in Russian gas supply). A whole strategy was devised to reduce energy consumption, one aspect of it was a set of recommendations, targeted to households, and broadcast through TV, radio, and other media. I will only list the two most iconic:
    Lower the temperature at home to 19°C
    Turn off the wifi during the night
    (1) is a very effective idea. The gov...
    6 min
  • LW - Don't take bad options away from people by Dumbledore's Army
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Don't take bad options away from people, published by Dumbledore's Army on March 26, 2023 on LessWrong.
    Short version: when people are in a bad situation and only have bad options, taking one of those options away is wrong and causes suffering. Not understanding this is a common failure mode among the general population and results in a lot of situations where governments are actively harming poor and desperate people.
    Why does this happen? I think it's a combination of fabricated options, typical-minding, and the usual political failure mode where activists care more about signalling their virtue than actually putting in the effort to understand what does and does not help people. It's also often easy to strawman the case for not taking people's options away.
    Example 1: selling kidneys
    Mrs Singh is an impoverished Indian woman who loves her children. One child has contracted tuberculosis and will die unless she can get money for antibiotics. Mrs Singh has three options:
    sell a kidney to get the money
    do something else desperate and maybe illegal to get money
    watch the child die
    I wish we lived in an ideal world where everyone had access to free health care and no one was desperate for money. But when you're truly desperate for money, at least you can sell a kidney. Oh wait, that's illegal. Because almost every government decided to take the only halfway-good option away from desperate people.
    (In a comparable situation where no money was involved - say a British Mrs Smith needed to donate one of her own kidneys to save a child with kidney failure, and the surgery was free on the National Health Service - no-one thinks that the desperate mother is exploited and should be saved from donating a kidney for her own good. I defy anyone who is anti-selling-kidneys to explain why Mrs Smith is not exploited but Mrs Singh is.)
    How can the world get this so wrong? My best guess is typical-mind fallacy. Activists and lawmakers tend to be reasonably well-off people who are unlikely to be so desperate that selling a kidney is their best option. So they are unable to put themselves in the mindset of someone genuinely desperate and assume that anyone who sells a kidney must be exploited in some way. And they make the lives of desperate people even harder while telling themselves how virtuous they are.
    Example 2: sex for rent
    The UK, where I live, bans people from having sex with their landlord as a form of rent, and recently had a debate on toughening the law even further. Note that the debate specifically references landlords with empty rooms who advertise for people who are willing to have sex in lieu of rent. This is not about landlords trying to change the terms of agreement with existing money-paying tenants, a situation where there really would be a risk of coercion.
    Insofar as I can pass the ITT of the proponents, I think it goes something like this: "Evil exploitative landlords demand sex from tenants. If we just stop them then the happy tenants could live exploitation-free."
    The problem is that the happy-tenants outcome is a fabricated option. What would actually happen is that the landlords who can't rent to tenants willing to have sex would just rent the room to someone who can pay a market rate in money instead. (And probably then use the cash to pay prostitutes - which is actually legal in the UK - since we've already established that these landlords are willing to give up cash to get sex.)
    From the point of view of the prospective tenants, the legal-sex-for-rent situation gives them three options:
    Pay a shockingly high rent for a room somewhere, which they may or may not be able to afford at all
    Agree to have sex with their landlord and pay nothing or a very discounted rent which they can afford
    Be homeless
    These are all bad options, and anyone who has to choose betwe...
    8 min
  • LW - Practical Pitfalls of Causal Scrubbing by Jérémy Scheurer
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Practical Pitfalls of Causal Scrubbing, published by Jérémy Scheurer on March 27, 2023 on LessWrong.
    TL;DR: We evaluate Causal Scrubbing (CaSc) on synthetic graphs with known ground truth to determine its reliability in confirming correct hypotheses and rejecting incorrect ones. First, we show that CaSc can accurately identify true hypotheses and quantify the degree to which a hypothesis is wrong. Second, we highlight some limitations of CaSc, in particular, that it cannot falsify all incorrect hypotheses. We provide concrete examples of false positive results with causal scrubbing. Our main finding is that false positives can occur when there is “cancellation”, i.e., CaSc causes the model to do better on some inputs and worse on others, such that, on average, the scrubbed model recovers the full loss. A second practical failure mode is that CaSc cannot detect whether a proposed hypothesis is specific enough, and it cannot distinguish between hypotheses that are extensionally equivalent.
    We thank Redwood Research for generously supporting this project by providing us with their Causal Scrubbing implementation, access to REMIX materials, and computational resources. We specifically thank Ansh Radhakrishnan, Buck Shlegeris, and Nicholas Goldowsdky-Dill for their feedback and advice. We thank the Long-Term Future Fund for financial support and Marcel Steimke and Lennart Heim for operational support. Finally, we thank Marius Hobbhahn, Adam Jermyn, and Erik Jenner for valuable discussions and feedback.
    Introduction
    Causal Scrubbing (CaSc) is a method to evaluate the accuracy of hypotheses about neural networks and provides a measure of the deviation of a hypothesis from the ground truth. However, CaSc does not guarantee to reject false or incomplete hypotheses. We thus believe that systematic evaluation of CaSc to investigate these limitations is valuable (in addition to evaluating its effectiveness in the wild, as done in most existing work). Hence, we evaluate CaSc to highlight its strengths and weaknesses and explore the reliability of CaSc in confirming correct hypotheses and rejecting incorrect ones.
    We evaluate the reliability of CaSc on synthetic graphs. While synthetic graphs are less realistic than trained neural networks, we get access to the known ground truth interpretation, which allows us to accurately evaluate our hypotheses. Since CaSc operates on general computational graphs, any results on synthetic graphs also apply to using CaSc on neural networks (although we don’t make any claim on how likely the situations we find are to occur in trained neural networks).
    Our evaluation is based on creating a synthetic graph that solves a specific problem (e.g., sorting a list) and creating an identical interpretation graph (the correct hypothesis). We then perturb the correct interpretation graph to make the hypothesis “worse”. Finally, we evaluate whether CaSc correctly determines the better hypothesis. Ideally, we want the scrubbed loss (the loss induced by applying CaSc) to correlate with the “correctness” of a hypothesis.
    To determine whether a hypothesis is “better” or “worse”, we introduce the concepts of extensional and intensional equivalence between functions. Extensional equivalent functions have the same input-output behavior; for example, Quicksort and Mergesort are extensionally equivalent as they both sort an input sequence. Intensional equivalent functions are implemented in the same way mechanistically. So once we zoom further in and compare Quicksort and Mergesort algorithmically, we see that they are not intensionally equivalent. This point is already made in the CaSc writeup, and our goal is merely to highlight that in the context of mechanistic interpretability, this is an important distinction that’s easy to overlook.
    In this post, we look deep...
    23 min
  • AF - Lessons from Convergent Evolution for AI Alignment by Jan Kulveit
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Lessons from Convergent Evolution for AI Alignment, published by Jan Kulveit on March 27, 2023 on The AI Alignment Forum.
    Prelude: sharks, aliens, and AI
    If you go back far enough, the ancestors of sharks and dolphins look really different:
    But modern day sharks and dolphins have very similar body shapes:
    This is a case of convergent evolution: the process by which organisms with different origins develop similar features. Both sharks and dolphins needed speed and energy efficiency when moving in an environment governed by the laws of hydrodynamics, and so they converged on a pretty similar body shape.
    For us, this isn’t very surprising, and doesn’t require much knowledge of evolution: we have a good intuitive understanding of how water works, and humans knew a lot of the underlying maths for the laws of hydrodynamics before they understood anything about evolution. Starting from these laws, it isn’t very surprising that sharks and dolphins ended up looking similar.
    But what if instead of starting with knowledge of hydrodynamics and then using that to explain the body shape of sharks and dolphins, we started with only knowledge of sharks’ and dolphins’ body shape, and tried to use that to explain underlying laws?
    Let’s pretend we’re alien scientists from an alternative universe, and for some weird reason we only have access to simplified 3D digital models of animals and some evolutionary history, but nothing about the laws of physics in the human/shark/dolphin universe. My guess is that these alien scientists would probably be able to uncover a decent amount of physics and a fair bit about the earth’s environment, just by looking at cases of convergent evolution.
    If I’m right about this guess, then this could be pretty good news for alignment research. When it comes to thinking about AI, we’re much closer to the epistemic position of the alien scientist: we either don't know the ‘physics’ of life and intelligence at all, or are only just in the process of uncovering it.
    But cases of convergent evolution might help us to deduce deep selection pressures which apply to AI systems as well as biological ones. And if they do, we might be able to say more about what future AI systems might look like, or, if we are lucky, even use some of the selection pressures to shape what systems we get.
    Introduction
    This post argues that we should use cases of convergent evolution to look for deep selection pressures which extend to advanced AI systems.
    Convergent evolution is a potentially big deal for AI alignment work:
    Finding deep selection pressures could help us predict what advanced AI systems will be like.
    It seems plausible that some of the properties people in the alignment space assume are convergent don’t actually extend to advanced AI.
    In this post, I’ll:
    Share some basics of convergent evolution,
    Argue that this is a big deal for alignment work, and then
    Respond to the objection that biology is super different from AI.
    The basics of convergent evolution
    The body shape of sharks and dolphins is just one of very many examples of convergent evolution in biology. For example:
    Visual organs arose “possibly hundreds of times”.
    Multicellularity evolved independently probably at least 11 times.
    Some form of higher-level intelligence evolved multiple times - in primates, apes, corvids, cetaceans, elephants - and possibly many other cases, depending on thresholds and definitions.
    We can think about convergent evolution in terms of:
    a basin of convergent evolution,
    an attractor state(s), and
    selection pressure(s).
    The basin of convergent evolution is the region of the abstract space in which, once an organism enters the basin, the pull of the selection pressure brings the organism closer to the attractor state.
    In the case of sharks and dolphins:
    The basin of convergent evolution is ...
    16 min
  • EA - EA is three radical ideas I want to protect by Peter Wildeford
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: EA is three radical ideas I want to protect, published by Peter Wildeford on March 27, 2023 on The Effective Altruism Forum.
    Context note: This is more of an emotional piece meant to capture feelings and reservations, rather than a logical piece meant to make a persuasive point. This is a very personal piece and only represents my own views, and does not necessarily represent the views of anyone else or any institution I may represent.
    It’s been a rough past five months for effective altruism.
    Because of this, understandably many people are questioning their connection and commitment to the movement and questioning whether “effective altruism” is still a brand or set of ideas worth promoting. I’ve heard some suggest it might be better to instead promote other brands and perhaps even abandon promotion of “effective altruism” altogether.
    I could see ways in which this is a good move. Ultimately I want to do whatever is most impactful. However, I worry that moving away from effective altruism could make us lose some of what I think makes the ideas and community so special and what drew me to the community more than ten years ago.
    Essentially, effective altruism contains three radical ideas that I don’t easily find in other communities. These three ideas are ideas I want to protect.
    Radical empathy
    Humanity has long had a fairly narrow moral circle. Radical Empathy is the idea that there are many groups of people, or other entities, that are worthy of moral concern even if they don't look or act like us. Moreover, it’s important to deliberately identify all entities worthy of moral concern so that we can ensure they are protected. I find effective altruism to be unique in extending moral concern to not just traditionally neglected farm animals and future humans (very important) – but also to invertebrates and potential digital minds. Effective altruists are also unique in trying to intentionally understand who might matter and why and actually incorporating this into the process of discovering how to best help the world. Asking the question "Who might matter that we currently neglect?" is a key question that is asked way too rarely.
    We understand that while it’s ok to have special concern for family and friends, we should generally aim to make altruistic decisions based on impartiality, not weighing people differently just because they are at a different level of geographic distance, a different level of temporal distance, a different species, or run cognition on a different substrate.
    I worry that if we were to promote individual subcomponents of effective altruism, like pandemic preparedness or AI risk, we might not end up promoting radical empathy and we might end up missing entire classes of entities that matter. For example, I worry that one more subtle form of misaligned AI might be an AI that treats humans ok but adopts common human views on nonhuman animal welfare and perpetuates factory farming or abuse of a massive number of digital minds. The fact that effective altruism has somehow created a lot of AI developers that avoid eating meat and care about nonhuman animals is a big and fairly unexpected win. I think only some weird movement that somehow combined factory farming prevention with AI risk prevention could’ve created that.
    Scope sensitivity
    I also really like that EAs are willing to “shut up and multiply”. We’re scope sensitive. We’re cause neutral. Nearly everyone else in the world is not. Many people pick ways to improve the world based on vibes or personal experience, rather than through a systematic search of how they can best use their resources. Effective altruism understands that resources are limited and that we have to make hard choices between potential interventions and help 100 people instead of 10, even if helping the 10 people feels as or m...
    6 min
  • EA - Introducing Focus Philanthropy by Focus Philanthropy
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Introducing Focus Philanthropy, published by Focus Philanthropy on March 27, 2023 on The Effective Altruism Forum.
    We’re excited to announce the launch of Focus Philanthropy, a new organization that connects philanthropists with outstanding giving opportunities to reduce the harms of factory farming.
    Focus Philanthropy aims to fill a gap in the effective animal advocacy space. The factory farming cause area is both fast-changing and complex in its variety of approaches and we aim to reduce barriers for new funders interested in entering the space. We offer thoughtful, tailored funding advice and engage with donors to increase their commitment and inspire them to support impactful giving opportunities within the cause area over the long term.
    Our core principles
    Impact: We help philanthropists maximize the impact of their donations, ensuring their funds go where they can do the most good.
    Evidence: Our recommendations are based on research, the current state of scientific evidence, and expert judgment.
    Independence: Our advice is always free of charge, allowing us to provide unbiased and impartial giving recommendations.
    The team
    Focus Philanthropy was founded by Leah Edgerton and Manja Gärtner.
    Leah has extensive experience in effective animal advocacy ranging from volunteering and direct work to having acted as a leader, mentor, and advisor. She previously worked as a philanthropic advisor, at Animal Charity Evaluators, and at ProVeg International.
    Manja has several years of experience as a researcher, grantmaker, and advisor in effective animal advocacy. She previously worked as a philanthropic advisor and at Animal Charity Evaluators. She holds a Ph.D. in economics.
    Please approach us directly with any feedback or questions.
    Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.
    2 min
  • LW - What can we learn from Lex Fridman’s interview with Sam Altman? by Karl von Wendt
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: What can we learn from Lex Fridman’s interview with Sam Altman?, published by Karl von Wendt on March 27, 2023 on LessWrong.
    These are my personal thougths about this interview.
    Epistemic status: I neither consider myself a machine-learning expert, nor am I an alignment expert. My focus is on outreach: explaining AI safety to the general public and professionals outside of the AI safety community. So an interview like this one is important material for me to both understand the situation myself and explain it to others. After watching it, I’m somewhat confused. There were bits in this talk that I liked and others that disturbed me. There seems to be a mix of humbleness and hubris, of openly acknowledging AI risks and downplaying some elements of them. I am unsure how open and honest Sam Altman really was. I don’t mean to criticize. I want to understand what OpenAI’s and Sam Altman’s stance towards AI safety really is.
    Below I list transcriptions of the parts that seemed most relevant for AI safety and my thoughts/questions about them. Maybe you can help me better understand this by commenting.
    [23:55] Altman: “Our degree of alignment increases faster than our rate of capability progress, and I think that will become more and more important over time.”
    I don’t really understand what this is supposed to mean. What’s a “degree of alignment”? How can you meaningfully compare it with “rate of capability progress”? To me, this sounds a lot like marketing: “We know we are dealing with dangerous stuff, so we are extra careful.” Then again, it’s probably hard to explain this in concrete terms in an interview.
    [24:40] Altman: “I do not think we have yet discovered a way to align a super powerful system. We have something that works for our current scale: RLHF.”
    I find this very open and honest. Obviously, he not only knows about the alignment problem, but openly admits that RLHF is not the solution to aligning an AGI. Good!
    [25:10] Altman: “It’s easy to talk about alignment and capability as of orthogonal vectors, they’re very close: better alignment techniques lead to better capabilities, and vice versa. There are cases that are different, important cases, but on the whole I think things that you could say like RLHF or interpretability that sound like alignment issues also help you make much more capable models and the division is just much fuzzier than people think.”
    This, I think, contains two messages: “Capabilities research and alignment research are intertwined” and “criticizing us for advancing capabilities so much is misguided, because we need to do that in order to align AI”. I understand the first one, but I don’t subscribe to the second one, see discussion below.
    [47:53] Fridman: “Do you think it’s possible that LLMs really is the way we build AGI?”Altman: “I think it’s part of the way. I think we need other super important things . For me, a system that cannot significantly add to the sum total of scientific knowledge we have access to – kind of discover, invent, whatever you want to call it – new, fundamental science, is not a superintelligence. . To do that really well, I think we need to expand on the GPT paradigm in pretty important ways that we’re still missing ideas for. I don’t know what those ideas are. We’re trying to find them.”
    This is pretty vague, which is understandable. However, it seems to indicate to me that the current, relatively safe, mostly myopic GPT approach will be augmented with elements that may make their approach much more dangerous, like maybe long term memory and dynamic learning. This is highly speculative, of course.
    [49:50] Altman: “The thing that I’m so excited about is not that it’s a system that kind of goes off and does its own thing but that it’s this tool that humans are using in this feedback loop . I’m excited about a world ...
    15 min
  • LW - GPT-4 Plugs In by Zvi
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: GPT-4 Plugs In, published by Zvi on March 27, 2023 on LessWrong.
    GPT-4 Right Out of the Box
    In some ways, the best and worst thing about GPT-4 was its cutoff date of September 2021. After that date it had no access to new information, and it had no ability to interact with the world beyond its chat window.
    As a practical matter, that meant that a wide range of use cases didn’t work. GPT-4 would lack the proper context. In terms of much of mundane utility, this would often destroy most or all of the value proposition. For many practical purposes, at least for now, I instead use Perplexity or Bard.
    That’s all going to change. GPT-4 is going to get a huge upgrade soon in the form of plug-ins(announcement).
    That means that GPT-4 is going to be able to browse the internet directly, and also use a variety of websites, with the first plug-ins coming from Expedia, FiscalNote, Instacart, KAYAK, Klarna, Milo, OpenTable, Shopify, Slack, Speak, Zapier and Wolfram. Wolfram means Can Do Math. There’s also a Python code interpreter.
    Already that’s some exciting stuff. Zapier gives it access to your data to put your stuff into context.
    Also there’s over 80 secret plug-ins already, that can be revealed by removing a specific parameter from an API call. And you can use them, there are only client-side checks stopping you. Sounds secure.
    We continue to build everything related to AI in Python, almost as if we want to die, get our data stolen and generally not notice that the code is bugged and full of errors. Also there’s that other little issue that happened recently. Might want to proceed with caution.
    (I happen to think they’re also downplaying the other risks even more, but hey.)
    Perhaps this wasn’t thought out as carefully as we might like. Which raises a question.
    So, About That Commitment To Safety and Moving Slowly And All That
    That’s all obviously super cool and super useful. Very exciting. What about safety?
    Well, we start off by having everyone to write their instructions in plain English and let the AIs figure it out, because in addition to being super fast, that’s the way to write secure code that does what everyone wants that is properly unit tested and fails gracefully and doesn’t lead to any doom loops whatsoever.
    (Gross’ link here.)
    Also one of the apps in the initial batch is Zapier, which essentially hooks GPT up to all your private information and accounts and lets it do whatever it wants. Sounds safe.
    No, no, that’s not fair, they are concerned about safety, look, concern, right here.
    At the same time, there’s a risk that plugins could increase safety challenges by taking harmful or unintended actions, increasing the capabilities of bad actors who would defraud, mislead, or abuse others.
    I mean, yes, you don’t say, having the ability to access the internet directly and interface with APIs does seem like the least safe possible option. I mean, I get why you’d do that, there’s tons of value here, but let’s not kid ourselves. So, what’s the deal?
    We’ve performed red-teaming exercises, both internally and with external collaborators, that have revealed a number of possible concerning scenarios. For example, our red teamers discovered ways for plugins—if released without safeguards—to perform sophisticated prompt injection, send fraudulent and spam emails, bypass safety restrictions, or misuse information sent to the plugin.
    We’re using these findings to inform safety-by-design mitigations that restrict risky plugin behaviors and improve transparency of how and when they’re operating as part of the user experience.
    This does not sound like ‘the red teams reported no problems,’ it sounds like ‘the red teams found tons of problems, while checking for the wrong kind of problem, and we’re trying to mitigate as best we can.’
    Better than nothing. Not super comforting. What ha...
    8 min
  • EA - What would a compute monitoring plan look like? [Linkpost] by Akash
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: What would a compute monitoring plan look like? [Linkpost], published by Akash on March 26, 2023 on The Effective Altruism Forum.
    Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.
    1 min