The Nonlinear Library

The Nonlinear Library

By The Nonlinear FundEducation
Download on the App Store

All The Nonlinear Library episodes

  • EA - Epistemics-Improving Activities for Groups and Friends by ChanaMessinger
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Epistemics-Improving Activities for Groups and Friends, published by ChanaMessinger on October 10, 2022 on The Effective Altruism Forum.
    A list I've accumulated from multiple places, so can't vouch for all of them, and definitely there's no empirical evidence I know of for these improving epistemics, but I wanted to collect the list anyway.
    The obvious place to do these is in student groups, but I think limiting them there is a mistake. I think we all can benefit from honing our skills, plus a bunch of these are just fun. If you play board games or do Puzzled Pint with your friends or coworkers, then these can just be part of that part of your life.
    Will be editing and adding to this over time - what would you recommend?
    Activities
    Thinking Physics questions
    Fermi estimates / Estimathons
    Sample estimathon questions
    Rules for estimathons from Berkeley, which probably came from Jane Street
    List of lists of Fermi problems
    Calibration training
    Sage Calibration Activity (my current favorite)
    Scout Mindset Calibration Questions
    Open Philanthropy's Calibration Questions
    I have more if you DM, don't want everyone to know all the good questions ahead of time :)
    Team Forecasting - really excited about this, watch this space, but in the meantime, nothing stopping you from just picking a question from Metaculus or Manifold and discussing together
    I had a great time recently pastcasting a question with a friend, which had the added benefits of getting both of us smoe knowledge about the current conflict in Ethiopia
    Games
    Wits and Wagers
    Murder, She Bet
    Betting Market Game Night
    Poker
    Talking
    Explain or draw how something works, notice where you falter, let someone pick up the slack from there
    Writing
    Fact posts
    Minimal trust investigations
    Learning by Writing
    Related: Putting Ideas Into Words
    Learning by writing in groups
    Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.
    3 min
  • EA - Book a corporate event for Giving Season by Jack Lewars
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Book a corporate event for Giving Season, published by Jack Lewars on October 11, 2022 on The Effective Altruism Forum.
    Tl;dr
    Tl;dr - following the success of previous corporate talks, One for the World, Giving What We Can and High Impact Professionals are collaborating to offer a range of corporate talks this giving season.
    Use our contact form to learn more or book a talk.
    Context
    In recent years, various organizations have done successful corporate outreach, most notably One for the World, Giving What We Can and High Impact Professionals. These efforts have raised at least US$1m for effective nonprofits (1, 2), the vast majority of it counterfactual. They have also anecdotally contributed to spreading EA ideas and getting people more engaged in EA (although we'd like to track this more closely/better).
    As different organizations enter this space, it's important to avoid unnecessary competition and make it easy for potential customers to find what they want. Accordingly, OFTW, GWWC and HIP are collaborating to offer different types of corporate talk this giving season. We're also pooling results and will analyse these to see if certain types of talk produce better outcomes (in terms of donations, recurring donations, email sign ups, further actions with EA etc.).
    Offers
    We are offering three types of corporate outreach for the 2022 giving season.
    Introduction to effective giving talks, from One of the World
    One for the World has run ~60 of these talks, raising $600k in one-off donations and ~$150k in recurring donations. The format is a 40 minute talk and 20 minute Q&A, introducing the main ideas of effective giving and promoting our 1% pledge to GiveWell's nonprofits. It's delivered by a One for the World member of staff, usually Executive Director Jack, and every attendee gets a $10 donation to any nonprofit of their choice as an incentive to attend. We can support you with advertising copy and graphics, a bespoke donation page and help getting internal sign off.
    Run a fundraising drive yourself
    High Impact Professionals supported 8 EAs to run fundraising drives at their workplace in 2021, raising $240k in counterfactual dollars. On an hourly basis, organizing those events proved to be as impactful as direct work (3), which is a great way for EA working professionals to multiply their impact. They have a full playbook for how to run a fundraising drive, together with other resources, and offer training and staff time to support you.
    A philanthropy workshop or Giving Game
    Giving What We Can are offering two products this year:
    A workshop on high-impact philanthropy (35-90 min):
    A new workshop format which aims to get people to reason about their approach to philanthropy and come up with their first steps, delivered by a GWWC member of staff.
    A giving game (40-90 min):
    GWWC sponsor some funds, give your group some information on different effective charities and the group votes on where to donate (again facilitated by a GWWC member of staff).
    They are particularly enthusiastic about doing more workshops as they're much more interactive than talks.
    Get in touch to discuss your options or book an event
    We have a contact form that can be used to select any of the above options, and we'll redirect to you the right organization.Organizing any of these events is a highly impactful way to raise counterfactual dollars, build career capital and spread the ideas of effective giving and EA - so please let us know if you can help!
    Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.
    4 min
  • EA - Counterproductive EA mental health advice (and what to say instead) by Ada-Maaria Hyvärinen
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Counterproductive EA mental health advice (and what to say instead), published by Ada-Maaria Hyvärinen on October 10, 2022 on The Effective Altruism Forum.
    They tell me: eat and drink! Be glad to be among the haves!But how can I eat and drink whenI take what I eat from the starving andThose who have died of thirst go without my glass of water?And yet I eat and drink.
    (Bertolt Brecht: An die Nachgeborenen)
    Many people in EA at least occasionally struggle with feelings of worthlessness and urgency or have the tendency to self-sacrifice to an unhealthy level. Many other people in EA would like to help them out by outlining how feeling worthless is not strategically good. However, in these discussions, people are often talking past each other, without making any progress on the actual issues.
    In this post, I point out why some seemingly motivational fixes on the feeling of worthlessness can actually be counterproductive. I also give some tips on how to better support your EA friends who are struggling with these issues, and say some words to those who are facing the issues themselves.
    Examples of counterproductive advice
    “It’s actually altruistic to be happy, because happiness makes you more productive”
    Most people are in fact more productive if they are feeling at least ok, so at face value, this statement is true. Despite this, it is not necessarily good advice for someone who is not feeling happy and would like to be more productive.
    Inside the not-happy person’s head there is a to-do list that looks something like this:
    do more EA! people are dying
    also read up more EA to know how to actually do EA
    I’m so tired but I must hold through and do my EA stuff because if I don’t it means I don’t actually care, and I really don’t want anyone to die
    So whenever you tell them to be more happy in order to be more productive, the to-do list starts to look like this:
    do more EA! people are dying
    also read up more EA to know how to actually do EA
    I’m so tired but I must hold through and do my EA stuff because if I don’t it means I don’t actually care, and I really don’t want anyone to die
    become happier! you cannot actually help when you are feeling like this, so stop it and get on with all the other stuff; people are dying, remember!
    Redefining happiness as an instrumental goal for becoming more productive is rarely going to make anyone more happy. It is actually likely to put additional pressure on the person who is already feeling guilty about not being productive enough, thus making them unhappier in the progress.
    “If you’ve donated $3400 (or $5000 or whatever the current estimate is) you clearly deserve to live”
    I haven’t seen this advice much online but several people around me have come up with it independently. This is probably because I occasionally have thoughts about not deserving to be alive and talking about them to people around me sometimes helps.
    In this advice, the advisor is trying to debunk a depressed misguided belief by reason and evidence (sort of). The concept is related to climate (or perhaps animal welfare) compensation: “if you think you are not worthy of living, just buy your way out of it by saving another person – tada, you have compensated for your life!”
    There are many dangers to this line of thought such as:
    how do I know that saving one person is a sufficient compensation amount?
    if I haven’t donated that much, does it mean I deserve to die?
    if I have donated that much, what does it actually matter that I continue to live – it won’t change anything about my previous donations anyway?
    if I ever lose my ability to produce impact in the future (for example because I need to stop working due to disability), does it mean I deserve to die then?
    do all people who have not donated x amount deserve to die?
    These questions are all good, because the life comp...
    10 min
  • EA - Some Carl Sagan quotations by finm
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Some Carl Sagan quotations, published by finm on October 10, 2022 on The Effective Altruism Forum.
    Carl Sagan (1934–1996) was an astronomer and science communicator. He organised the first physical messages to space (the Pioneer plaque and the Voyager Golden Record), presented the hugely popular TV series Cosmos (1980), and considered humanity’s long-term future in Pale Blue Dot (1994). He was also part of the team of researchers who first discovered the possibility of nuclear winter, and so became a leading voice of concern about the use of nuclear weapons.
    Sagan’s words were often prescient and always poetic. In particular, I think he captures many ideas related to longtermism and existential risk as powerfully as anyone writing today.
    I’ve tried collecting some quotations that stand out to me from Sagan’s work, though I’ve only read a minority of his published writing.
    You can find a slightly more comprehensive version here. The website for Toby Ord’s book The Precipice contains a list of quotations pertaining to existential risk, which I partially borrowed from here. Michael Nielsen has also written some fantastic 'working notes' on Cosmos.
    Cosmos: A Personal Voyage (1980)
    Note that Cosmos was co-written with Ann Druyan.
    Episode 1 — "The Shores of the Cosmic Ocean"
    The cosmos is all that is, or ever was, or ever will be. Our contemplations of the Cosmos stir us — there is a tingling in the spine, a catch in the voice, a faint sensation, as if a distant memory, of falling from a great height. We know we are approaching the greatest of mysteries. The size and age of the cosmos are beyond ordinary human understanding. Lost somewhere between immensity and eternity is our tiny planetary home, the Earth. For the first time we have the power to determine the fate of our planet, and ourselves. This is a time of great danger, but our species is young and curious and brave. It shows much promise. In the last few millennia we have made the most astonishing and unexpected discoveries about the cosmos, and our place within it. I believe our future depends powerfully on how we understand this cosmos; in which we float, like a mote of dust, in the morning sky.
    You can watch this opening scene here.
    The surface of the Earth is the shore of the cosmic ocean. On this shore, we’ve learned most of what we know. Recently, we’ve waded a little way out; maybe ankle-deep: and the water seems inviting. Some part of our being knows this is where we came from; we long to return — and we can, because the Cosmos is also within us: we are made of star stuff.
    We are the legacy of 15 billion years of cosmic evolution. We have a choice. We can enhance life and come to know the universe that made us, or we can squander our 15 billion year heritage in meaningless self-destruction. What happens in the first second of the next cosmic year depends on what we do — here and how — with our intelligence, and our knowledge of the cosmos.
    Episode 13 — "Who Speaks for Earth?"
    [Imagining human extinction] Maybe the reptiles will evolve intelligence once more. Perhaps, one day, there will be civilizations again on Earth. There will be life. There will be intelligence. But there will be no more humans. Not here, not on a billion worlds.
    [T]he world impoverishes itself by spending a trillion dollars a year on preparations for war. And by employing perhaps half the scientists and high technologists on the planet in military endeavors. How would we explain all this to a dispassionate extraterrestrial observer? What account would we give of our stewardship of the planet Earth? We have heard the rationales offered by the superpowers. We know who speaks for the nations. But who speaks for the human species?
    It's probably here. [Alexandria] that the word "cosmopolitan" realized its true meaning of a citizen, not just...
    18 min
  • EA - On absurdity by OllieBase
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: On absurdity, published by OllieBase on October 10, 2022 on The Effective Altruism Forum.
    Also posted on my blog. These are my views, and not the views of my employer, CEA.
    “Accepting the absurdity of everything around us is one step, a necessary experience: it should not become a dead end. It arouses a revolt that can become fruitful” ― Albert Camus
    Many people in the effective altruism community are ambitious. Absurdly ambitious.
    This is good. To achieve amazing things, you need to be absurdly ambitious. But, as we work towards our absurdly ambitious goals, we might end up forgetting how absurd it all is.
    I think that would be a shame. We can take comfort, even joy, in the absurdity of it all. It can help us stay grounded, in a strange, roundabout way. My claim here is not that people should be less ambitious, or give up. I want to offer an absurdist frame for thinking about our efforts that you can try on. My claim is that noticing the absurdity of our projects could be a healthy way for some of us to stay absurdly ambitious over the long run.
    All of us are weird
    Here are some of the things that people in the EA community do, framed absurdly:
    Some people might design systems on computers badly such that they do bad things so other people on other computers are trying to work out how to make the systems better but we don’t know how and we’re hoping that the other people don’t make the bad systems quicker than we can make them bad at being bad. How do I help with that? Well, I’m not building the better systems myself, I’m trying to find the people who might build the better systems and encouraging them to go and hang out with the better-systems people in the better-systems office so that they actually build the better systems. Also, I only drink chocolate milk.
    There’s this bad chemical which everyone thought was a fine chemical but it’s not fine, it’s bad, and that’s very bad because people put that chemical in paint. I can’t take the chemical out of the paint myself so I’m flying to Africa where they use that paint to tell people to tell other people to stop putting the bad chemical in the paint so that people don’t use the bad paint and then get sick from the bad chemical.
    The companies that kill chickens treat them really badly. I can’t stop them killing the chickens because people love eating them but we can make them treat the chickens a little less badly by letting them stay in bigger houses before they kill the chickens. We think that the chickens feel better about having bigger houses, but we don’t know how they feel about it and we don’t know how to ask them.
    I’m trying to get all the build-better-computer-system people, the take-the-chemical-out-the-paint people and the bigger-houses-for-chickens people and lots of other people all in one building in London (but also in Mexico and India and Sweden) so that they can talk to each other and make them all better at doing all the good things they’re doing. I don’t really know how they do this but they do seem to keep coming so that’s good.
    This is all absolutely nuts.
    This would all be fine if it wasn’t causing us any problems. But we take ourselves and our work very seriously and that causes us to burn out while we work on these crazy things and that is definitely bad.
    As a brief aside, I think it’s important to distinguish absurdity from hopelessness:
    Hopelessness says: what you’re working on doesn’t make sense. But the rest of the world makes sense, so you need to work this out or you will fail.
    Absurdity says: what you’re working on doesn’t make sense. But the rest of the world doesn’t make sense either, so that’s probably fine. Go have a margarita or just lie face down on the floor — that would make about as much sense but it’s a bit more relaxing.
    How noticing absurdity can help
    Creating distance between...
    6 min
  • LW - We can do better than argmax by Jan Kulveit
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: We can do better than argmax, published by Jan Kulveit on October 10, 2022 on LessWrong.
    Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.
    1 min
  • AF - QAPR 4: Inductive biases by Quintin Pope
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: QAPR 4: Inductive biases, published by Quintin Pope on October 10, 2022 on The AI Alignment Forum.
    Introduction
    This is week 4 of Quintin's Alignment Papers Roundup. The current focus is the inductive biases of stochastic gradient descent.
    For most datasets and labels, there are many possible models that reach good performance. "Inductive biases" refers to the various factors that incline a particular training process to find some types of models over others. When the data under-specify the learned model, a training process's inductive biases determine what sort of decision making process the model implements, and how the model generalizes beyond its training data.
    I'd intended to publish this last week, but it turns out that there's a lot of work on SGD's inductive biases, and it's very technical. I kept finding new papers that seemed relevant. That's why this roundup has 16 papers, in place of the usual ~9 or so.
    Papers
    Eigenspace Restructuring: a Principle of Space and Frequency in Neural Networks
    Understanding the fundamental principles behind the massive success of neural networks is one of the most important open questions in deep learning. However, due to the highly complex nature of the problem, progress has been relatively slow. In this note, through the lens of infinite-width networks, a.k.a. neural kernels, we present one such principle resulting from hierarchical localities. It is well-known that the eigenstructure of infinite-width multilayer perceptrons (MLPs) depends solely on the concept frequency, which measures the order of interactions. We show that the topologies from deep convolutional networks (CNNs) restructure the associated eigenspaces into finer subspaces. In addition to frequency, the new structure also depends on the concept space, which measures the spatial distance among nonlinear interaction terms.
    The resulting fine-grained eigenstructure dramatically improves the network's learnability, empowering them to simultaneously model a much richer class of interactions, including Long-Range-Low-Frequency interactions, Short-Range-High-Frequency interactions, and various interpolations and extrapolations in-between. Additionally, model scaling can improve the resolutions of interpolations and extrapolations and, therefore, the network's learnability. Finally, we prove a sharp characterization of the generalization error for infinite-width CNNs of any depth in the high-dimensional setting. Two corollaries follow: (1) infinite-width deep CNNs can break the curse of dimensionality without losing their expressivity, and (2) scaling improves performance in both the finite and infinite data regimes.
    My opinion:
    The NTK lets us directly compute the inductive biases of a neural network near a particular point in parameter space. The NTK's eigenfunctions give us possible behaviors, and its spectrum tells us how easy it is for the network to learn each eigenfunction.
    This paper uses the NTK to compare the architectural inductive biases of convolutional networks to those of multilayer perceptrons. It seems like a promising approach for better understanding what sorts of behaviors different architectures are inclined to learn. However, this paper makes two major simplifying assumptions:
    It assumes the infinite width limit, thereby ensuring the NTK remains constant through training (and also preventing any feature learning).
    It assumes that input data is uniformly distributed across a manifold formed as a product of hyperspheres.
    This paper's discussion of inductive biases focuses a lot on the frequency biases of neural networks, rather than things more related to alignment. It is very hard to use the NTK (or any mathematical formalism) to talk about inductive biases towards or away from "intentional" / high-level concepts, such as values, deception, corri...
    35 min
  • AF - Disentangling inner alignment failures by Erik Jenner
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Disentangling inner alignment failures, published by Erik Jenner on October 10, 2022 on The AI Alignment Forum.
    TL;DR: This is an attempt to disentangle some concepts that I used to conflate too much as just "inner alignment". This will be old news to some, but might be helpful for people who feel confused about how deception, distributional shift, and "sharp left turns" are related. I first discuss them as entirely separate threat models, and then talk about how they're all aspects of "capabilities are more robust than alignment".
    Here are three different threat models for how an AI system could very suddenly do catastrophic things:
    Deception: The AI becomes deceptively aligned at some point during training, and then does what we want only for instrumental reasons (because it wants to be deployed). Once we deploy, it starts pursuing its actual objective, which is catastrophic for humans.
    Distributional shift: The AI behaves well during training, perhaps using some messy set of heuristics and proxy objectives. We deploy, and there's distributional shift in the AI's inputs, which leads to the model's proxies no longer being aligned with human values. But it's still behaving capably, so we again get catastrophic outcomes.
    Capability gains/sharp left turn: At some point (while training or in deployment), the AI becomes much more capable, including at a bunch of things we didn't explicitly train for. This could happen quite suddenly, e.g. because it learns some crucial general skill in relatively few gradient steps, or because it starts learning from something other than gradients that's way faster. The properties of the AI that previously ensured alignment are too brittle and break during this transition.
    Note that these can be formulated as entirely distinct scenarios. For example, deception doesn't require a distributional shift nor capability gains; instead, the sudden change in model behavior occurs because the AI was "let out of the box" during deployment. Conversely, in the distributional shift scenario, the model might not be deceptive during training, etc. (One way to think about this is that they rely on changes along different axes of the training/deployment dichotomy).
    Examples
    I don't think we have any empirical examples of deception in AI systems, though there are thought experiments. We do see kind of similar phenomena in interactions between humans, basically whenever someone pretends to have a different goal than they actually do in order to gain influence.
    To be clear, here's one thing that is not an example of deception in the sense in which I'm using the word: an AI does things during training that only look good to humans even though they actually aren't, and then continues to do those things in deployment. To me, this seems like a totally different failure mode, but I've also seen this called "deception" (e.g. "Goodhart deception" in this post), thus the clarification.
    We do have experimental evidence for goal misgeneralization under distributional shift (the second scenario above). A well-known one is the CoinRun agent from Goal misgeneralization in Deep RL, and more recently, DeepMind published many more examples.
    A classic example for sudden capability gains is the history of human evolution. Relatively small changes in the human brain compared to other primates made cultural evolution feasible, which allowed humans to improve from a source other than biological evolutionary pressure. The consequence were extremely quick capability gains for humanity (compared to evolutionary time scales). This example contains both the "threshold mechanism", where a small change to cognitive architectures has big effects, and the "learning from another source mechanism", with the former enabling the latter.
    In ML, grokking might be an example for the "threshold ...
    8 min
  • EA - Is there any general advice you have? by lynettebye
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Is there any general advice you have?, published by lynettebye on October 10, 2022 on The Effective Altruism Forum.
    The Peek behind the Curtain interview series includes interviews with eleven people I thought were particularly successful, relatable, or productive. We cover topics ranging from productivity to career exploration to self-care.
    This ninth and final post covers general advice from my guests.
    You can view bios of my guests and other posts in the series here. This post is cross posted on my blog.
    Backchain
    We want to think, figure out some things to do, and then, if we do those things, the world will be better. An important part of that, obviously, is making sure that the things you think about, matter for the outcomes you want to cause to happen.
    In practice, it seems to me that what happens is people get into an area, look around, look at what other people are doing. They spend a few minutes, possibly hours thinking about, “Okay, why would they be doing this?” This seems great as a way to get started in a field. It's what I did.
    But then they just continue and stay on this path, basically, for years as far as I can tell, and they don't really update their models of "Okay, and this is how the work that I'm doing actually leads to the outcome." They don't try to look for flaws in that argument or see whether they're missing something else.
    Most of the time when I look at what a person is doing, I don't really see that. I just expect this is going to make a lot of their work orders of magnitude less useful than it could be.
    Rohin Shah
    Fail often
    One thing on my to-do list is to make sure I fail every month at something because then that means that you have been reaching your max. When I was applying for fellowships, scholarships, internships in policy and whatever, if I had looked at every single rejection as a failure rather than just expected, then I wouldn't be here.
    Abi Olvera
    Guilt is a symptom, not a motivator
    I think guilt is a feeling I have reasonably often, not too intensely, and not all the time. It doesn't generally cause me to be motivated. It's more usually a symptom of being in an unmotivated place. Or being in a place where there's not so many hours I can pour into my thing and I'm wandering in the desert. I'll often feel, less frequently now but still sometimes, I'll feel guilty and frustrated that I haven't moved on something yet, things haven't moved forward, I haven't put in a lot of hours.
    Guilt can get you to do some things, like call your parents or maybe even run an extra lap or something. The bottleneck for finding traction in research isn't something that I find I've been able to usefully address by feeling guilty about it. It's more like an effect of being in a place where I'm for whatever reason not that productive.
    Ajeya Cotra
    It’s okay to skill up before doing impactful work
    I made a mistake of feeling I needed to be working on some really impactful thing right away. I think that worked out okay, in my case. I got somewhat lucky in that regard, but I should have been totally happy to take at least a year or two and just try to find the place where I could skill up as well as possible.
    Daniel Ziegler
    You don’t need to be impactful immediately in your career
    Maybe don't stress out about the fact that you're not impactful now.
    Abi Olvera
    Competing with the very best takes a lot of time and effort
    It's really hard to be the top percent, to compete with people who are the 20 who get the national scholarship. I feel like if I would have eased up, it wouldn't have happened and then I would have had outcomes that are more typical of people from my socioeconomic background. I would say I actually wasn't unhappy but I had reached, probably, the max of what I could do, at least in terms of time.
    Abi Olvera
    Calibrate recommendations
    Maybe ...
    8 min
  • EA - When reporting AI timelines, be clear who you're (not) deferring to by Sam Clarke
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: When reporting AI timelines, be clear who you're (not) deferring to, published by Sam Clarke on October 10, 2022 on The Effective Altruism Forum.
    It's fashionable these days to ask people about their AI timelines. And it's fashionable to have things to say in response.
    But relative to the number of people who report their timelines, I suspect that only a small fraction have put in the effort to form independent impressions about them. And, when asked about their timelines, I don't often hear people also reporting how they arrived at their views.
    If this is true, then I suspect everyone is updating on everyone else's views as if they were independent impressions, when in fact all our knowledge about timelines stems from the same (e.g.) ten people.
    This could have several worrying effects:
    People's timelines being overconfident (i.e. too resilient), because they think they have more evidence than they actually do.
    In particular, people in this community could come to believe that we have the timelines question pretty worked out (when we don't), because they keep hearing the same views being reported.
    Weird subgroups forming where people who talk to each other most converge to similar timelines, without good reason.
    People using faulty deference processes. Deference is hard and confusing, and if you don't discuss how you’re deferring then you're not forced to check if your process makes sense.
    So: if (like most people) you don't have time to form your own views about AI timelines, then I suggest being clear who you're deferring to (and how), rather than just saying "median 2040" or something.
    And: if you’re asking someone about their timelines, also ask how they arrived at their views.
    (Of course, the arguments here apply more widely too. Whilst I think AI timelines is a particularly worrying case, being unclear if/how you're deferring is a generally poor way of communicating. Discussions about p(doom) are another case where I suspect we could benefit from being clearer about deference.)
    Finally: if you have 30 seconds and want to help work out who people do in fact defer to, take the timelines deference survey!
    Thanks to Daniel Kokotajlo and Rose Hadshar for conversation/feedback, and to Daniel for suggesting the survey.
    This sort of thing may not always be bad. There should be people doing serious work based on various different assumptions about timelines. And in practice, since people tend to work in groups, this will often mean groups doing serious work based on various different assumptions about timelines.
    Here are some things you might say, which exemplify clear communication about deference:
    "I plugged my own numbers into the bio anchors framework (after 30 minutes of reflection) and my median is 2030. I haven't engaged with the report enough to know if I buy all of its assumptions, though" - "I just defer to Ajeya's timelines because she seems to have thought the most about it" - "I don't have independent views and I honestly don't know who to defer to"
    Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.
    3 min