The Nonlinear Library

The Nonlinear Library

By The Nonlinear FundEducation
Download on the App Store

All The Nonlinear Library episodes

  • EA - High-Impact Psychology (HIPsy): Piloting a Global Network by Inga
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: High-Impact Psychology (HIPsy): Piloting a Global Network, published by Inga on September 29, 2022 on The Effective Altruism Forum.
    Engaged with psychology or mental health? This is for you.
    Impartial compassion. Rationality. Wellbeing. For a movement built on these values, EA likely underutilizes psychology professionals. Together with our supporters from the Global Priorities Institute, the Center for Effective Altruism, and the Happier Lives Institute, HIPsy aims to help people engaged with psychology or mental health maximize their impact.
    Vision
    We will follow in the footsteps of the EA Consulting Network, High-Impact Medicine, and High-Impact Athletes. Accordingly, the goal of HIPsy shall be to increase the likelihood of high-impact decisions, make collaboration and information processes more effective, and reduce the risk of value drift for people engaged in psychology or mental health.
    Relevant resources shall be available, easy to access, and use:
    up-to-date high-quality information,
    career and work advice,
    networking and collaboration opportunities.
    Psychological know-how shall be effectively acquired, shared, and used for EA. Psychology expertise is particularly needed in the fields of:
    mental health and well-being, both within EA and globally,
    community building, and outreach,
    management, HR, and operations,
    priorities research, and effectiveness research,
    x-risk-reduction and AI safety, e.g. awareness-building, and persuasion.
    Summary
    The goal for the next few months is to find out which of the many potential actions to prioritize, and how to address them most effectively. We will check what materials, events, and services are in-demand, and pilot some of them.
    You want to help?
    Let us know here. If you'd like to collaborate, fund us, or if you have got any of the following skills, we want to hear from you: online content creation, running mentorship programs, hosting events, web-dev, community-building, running surveys, research, and cost-effectiveness analyses.
    Opportunities
    Engaging with > 50 members of the EA community and their materials revealed three major opportunities:1. EA has skill bottlenecks that psychological professionals can help with
    Industry: management, entrepreneurship, operations, HR
    Science: at universities and EA mental health/wellbeing orgs (lack of seniors)
    Therapy: EA-informed psychotherapists and coaches for members of the EA community
    2. It’s hard for psychology professionals to enter EA even if they are likely good matches
    Existing materials and advice are difficult to find, scattered, contradictory, out of date, and selective (e.g. reading list, Effective Thesis) which likely discourages prospects
    Effectiveness and altruism: People largely enter psychology because they want to help others, and most universities have very high entrance requirements and thus select intelligent and ambitious individuals
    Mental health as a cause area faces large funding constraints that make it hard to enter, contribute and stick for people engaged in this field.
    3. There is unfulfilled potential for synergistic action and systematic exchangeThe impact of psychology-related EA orgs could be boosted by
    informing members of the community of techniques, knowledge, and best practices they can use to be more impactful, e.g. status quo of psychology research
    fostering collaboration and joint efforts between organizations that face similar challenges, e.g. mental health orgs that need to learn how to develop scalable and sustainable psychological interventions
    cooperating with leading psychology experts and EA-aligned individuals outside of EA that can help EA stay up to date with current industry, market, and research standards.
    Conclusion. It can be easier to access up-to-date high-quality information, advice, and networking opportunities. We imagine...
    10 min
  • EA - Why don't all EA-suggested organizations disclose salary in job descriptions? by Cassidy
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Why don't all EA-suggested organizations disclose salary in job descriptions?, published by Cassidy on September 28, 2022 on The Effective Altruism Forum.
    Summary
    I believe every organization that is part of the wider EA community , should disclose salary in the job description. Especially those listed on popular EA places, such as job boards, related forums or social media groups.
    Reason: Applying to jobs and processing those applications takes a lot of time. This time is essentially wasted if a certain (low) salary would prevent an applicant from being able to accept the position.
    Organizations who value using (applicant's) time efficiently should hence disclose salary up front.
    Request
    I didn't immediately find another post on the EA forum answering this question. There is this post outlining the same issue, but it doesn't mention the arguments. I would be grateful if somebody could point me to a list of reasons why that should not be the case. If there are great arguments to hide salaries, which outweigh the benefits of disclosing it, I'd be happy to change my view.
    If the general consensus is that all organizations should disclose salary, then why isn't this the case everywhere? For example, vacancy locations recommended by EA organizations could have salary as a listing criteria. Or at least strongly encourage it, as well as making the amount or non-disclosure visible right away.
    More information
    Applying to jobs takes a lot of time, if you include time for these examples:
    searching for open positions (e.g. browsing job board or Facebook groups)
    filtering jobs for requirements you meet
    learning more about the respective organizations
    reading job descriptions
    tailoring your CV to the job
    writing a customized Cover Letter
    If you actually get positive feedback, you will additionally spend time on:
    possibly talking to people working at these organizations
    one or multiple interviews
    potential trial tasks (plus perhaps test preparation)
    possibly required travel
    For most of your applications you might never hear back, or likely get rejections. There is already a great post with several time examples and which outlines the mental challenge coming with this (lack of) feedback.
    It also mentions that several EA organizations actually provide compensation for trial tasks, which is amazing! But of course you'll only get that if you reach a higher round. Regardless of you advancing, you will not be compensated for the initial application process steps.
    You might argue that if you just start applying to jobs, having the practice is beneficial, and hence even rejections are worth your time. However, after having sent out a dozen applicants you're unlikely to benefit sufficiently from extra practice to justify spending the time.
    On top of that, not getting any feedback with your rejections, doesn't even allow you to learn and improve your future applications. While I believe it would be incredibly helpful if organizations provide such feedback - where feasible - this is a whole different argument, since it requires significant resources.
    Compared to that, providing a salary range in the job description doesn't require as much time. This includes factoring in more initial effort, such as calculating different ranges and potential additional paragraphs about location etc. It could potentially even save resources from the organization, which would have been spent on reading CVs from applicants, who wouldn't be able to accept an offer either way.
    Further Notes
    Please note that I don't see an issue with the actual salary amount. It's understandable that many NGOs will have a lower budget than for-profit companies. And you likely also get the benefit of doing good directly with your job at an NGO.
    Giving a perfectly accurate number is not required either. I've seen many jobs offer a ...
    5 min
  • EA - I'm interviewing prolific AI safety researcher Richard Ngo (now at OpenAI and previously DeepMind). What should I ask him? by Robert Wiblin
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: I'm interviewing prolific AI safety researcher Richard Ngo (now at OpenAI and previously DeepMind). What should I ask him?, published by Robert Wiblin on September 29, 2022 on The Effective Altruism Forum.
    Next week I'm interviewing Richard Ngo, current AI (Safety) Governance Researcher at OpenAI and previous Research Engineer at DeepMind.
    Before that he was doing a PhD in the Philosophy of Machine Learning at Cambridge, on the topic of "to what extent is the development of artificial intelligence analogous to the biological and cultural evolution of human intelligence?"
    He is focused on making the development and deployment of AGI more likely to go well and less likely to go badly.
    Richard is also a highly prolific contributor to online discussion of AI safety in a range of places, for instance:
    Moral strategies at different capability levels on his blog Thinking Complete
    The alignment problem from a deep learning perspective on the EA Forum
    Some conceptual alignment research projects on the AI Alignment Forum
    Richard Ngo and Eliezer Yudkowsky policely debating AI Safety on Less Wrong
    The AGI Safety from First Principle education series
    And on his Twitter
    What should I ask him?
    Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.
    2 min
  • EA - Optimism, AI risk, and EA blind spots by Justis
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Optimism, AI risk, and EA blind spots, published by Justis on September 28, 2022 on The Effective Altruism Forum.
    Preface
    I'm going to start this post with a personal story, in part because people tend to enjoy writing that does that. If you don't want the gossip, just skip section one, the takeaway of which is: "EA has a strong cultural bias in favor of believing arbitrary problems are solvable".
    The gossip - and this takeaway - are not the only insight I'm trying to communicate. I don't mean for this post to be a "community" post overall, but rather one that is action relevant to doing good on the object level.
    N=1
    I had a two week work trial with a prominent EA org. There were some red flags. Nobody would tell me the projected salary, despite the job opportunity taking place across the country and in one of the most expensive cities on Earth. But whatever. I quit my job and flew over.
    It didn't work out. My best guess is that this was for cultural reasons. My would-be manager didn't think I'd been making fast enough progress understanding a technical framework, but the jobs I've had since have involved that framework, and I've received overwhelmingly positive feedback, working on products dramatically more complicated than the job opportunity called for. C'est la vie.
    Much later, I was told some of the things in my file for that organization. I was told by the organization's leader in a totally open way - nothing sneaky or "here's the dirt", just some feedback to help me improve. I appreciate this, and welcomed it. But here's the part relevant to the post:
    One of the negative things in my file was that someone had said I was "a bit of a downer". Much like with my technical competency, maybe so. But it's worth mentioning that in my day to day life, my coworkers generally think I'm weirdly positive, and often comment that my outlook is shockingly sanguine.
    I believe that both are true. I'm unusually optimistic. But professional EA culture is much, much more so.
    That's not a bad thing (he said, optimistically). But it's also not all good.
    (Why) is there an optimism bias?
    If you want to complete an ambitious project, it's extremely useful to presume that (almost) any challenge can be met. This is a big part of being "agentic", a much-celebrated and indeed valuable virtue within the EA community. (And also within elite circles more generally.) The high-end professional world has lots of upside opportunities and relatively little downside risk (you will probably always find a pretty great job as a fallback), so it's rational to make lots of bets on long odds and try to find holy grails.
    Therefore, people who are flagged as "ambitious", "impressive", "agentic", will both be selected for and encouraged to further cultivate a mindset where you never say a problem is insurmountable, merely challenging or, if you truly must, "not a top priority right now".
    But yeah. No odds are too long to be worth a shot!
    How is this action relevant?
    To avoid burying the lede, it's a major part of my reasoning to donate my 10% pledge to the Against Malaria Foundation, rather than x-risk reduction efforts. I'll trace out the argument, then pile on the caveats.
    On the 80,000 Hours Podcast, Will MacAskill put the odds of a misaligned AI takeover around 3%. Many community figures put the odds much higher, but I feel pretty comfortable anchoring on a combination of Will and Katja Grace, who put the odds at 7% that AI destroys the world. Low to mid single digits. Okay.
    So here's a valid argument, given its premises:
    Premise One: There is at least a 6% chance that AI destroys the world, or removes all humans from it.
    Premise Two: There exist interventions that can reliably reduce the risk we face by at least 1% (of the risk, not of the total - so 6% would turn into 5.94%, not 5%).
    Premise Three: W...
    15 min
  • LW - Petrov Day Retrospective: 2022 by Ruby
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Petrov Day Retrospective: 2022, published by Ruby on September 28, 2022 on LessWrong.
    This is a follow-up to the Petrov Day commemoration of 2022
    We died.
    Twice.
    Maybe three times.
    This is how it went down...
    September, 21, 5:43 PM Pacific Time: I write the code that determines whether a user is authorized to launch. I was in a rush because I leaving for EA Global DC.
    September 25, 8:00 PM: Petrov Day celebration commences. The button is on the frontpage. Any users with greater than 2,300 karma could press the button.
    September 25: 9:05 PM: I am reminded by others at LessWrong that I was supposed to publish the launch code in the comments. Without the code, no one can nuke the site. Understandably, no one spoke up to point out the absence of code – that would be hella suspicious. This reveals that incompetence on occasion might even save you from destruction. But don't bet on it.
    September 25, 10:55:04 PM: The frontpage is nuked after 2 hours and 55 minutes. The karma threshold is 2,100 and there are 290 users with that much karma or more.
    September 25, 11:51 PM: Oliver Habryka is suspicious that there is a bug in the code and he digs into it, uncovering that indeed, due to an error in the logic, any user with 0 karma (and exactly 0 karma) was already able to launch nukes and destroy the frontpage. An anonymity-preserving database check confirmed that it was a user with 0 karma who had launched the missiles.
    September 26, 12:36 AM: The decision is made to revive the frontpage and keep going with the Petrov Day commemoration.
    September 26, 10:11 AM: The entire LessWrong website (not just the front page) goes down and is returning 502s. This is because I pushed a bad commit to fix hiding the Petrov Opt-Out checkbox without properly testing it. The site is soon restored to functioning.
    September 26, 5:33:02 PM: At this time, the karma threshold for the ability to launch is 200, and 1,504 users were able to bring down the front page.
    With two hours and 27 minutes remaining, someone presses the initiation button, enters the launch code, and presses fire. The frontpage goes down.
    With the anonymity introduced in 2022, we do not know who pressed the button or why.
    September 26, 8:00:00 PM: The Petrov Day commemoration is concluded. The LessWrong frontpage is restored.
    What a day. What a day indeed.
    I will tell you the symbolic lessons I take from these events, though you are welcome to interpret things how you please.
    First, we very successfully commemorated how shoddy engineering by well-intentioned engineers can still lead to the end of the world. In Petrov's case, the shoddy engineering lead to a false alarm that was safely ignored. In our case, we weren't so lucky, and the bad code meant that the site did indeed go down. And in real life, you don't get to decide that wasn't fun and have a do-over.
    We could have decided to leave the site down, there would have been some symbolic purity in that, however it seemed more worthwhile to see what would happen under the primary design of the exercises – seeing how long it'd take for the site to go down, even if we were pretty sure that it would.
    Second, I want to claim that although the site went down, it in fact revealed that users on LessWrong with more than 300 karma (1,178 of them exist, 335 visited the site on Petrov Day) are not the kind to push buttons for one reason or another. Granted many of them would not have been checking the site, but still. This is a much better outcome than the Manifold Prediction market anticipated:
    You can see that for much of the day, the market thought that the site would survived for only ~30% of the day, i.e. 8 hours. In fact, the site survived 21.5 hours or 90% of the day.
    I feel pretty good about that! This is even true despite the fact that a button-pusher would remain anonymous this yea...
    7 min
  • EA - William MacAskill - The Daily Show by Tyner
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: William MacAskill - The Daily Show, published by Tyner on September 28, 2022 on The Effective Altruism Forum.
    Will appeared on The Daily Show with Trevor Noah yesterday (2022-09-27). This is, to my knowledge, the most prominent US television show to feature an EA guest. He primarily discusses ideas of most impactful charities, billionaire donors, and obligations to future people using the phrase "effective altruism" a bunch of times.
    It was neat to see these ideas in a different context and I thought it went really well!
    Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.
    1 min
  • AF - A Library and Tutorial for Factored Cognition with Language Models by Andreas Stuhlmüller
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: A Library and Tutorial for Factored Cognition with Language Models, published by Andreas Stuhlmüller on September 28, 2022 on The AI Alignment Forum.
    We want to advance process-based supervision for language models. To make it easier for others to contribute to that goal, we're sharing code for writing compositional language model programs, and a tutorial that explains how to get started:
    The Interactive Composition Explorer (ICE) is a library for writing and debugging compositional language model programs.
    The Factored Cognition Primer is a tutorial that explains using examples how to write such programs.
    We've been using ICE as part of our work on Elicit and have found it useful in practice.
    Interactive Composition Explorer (ICE)
    ICE is an open-source Python library for writing, debugging, and visualizing compositional language model programs. ICE makes it easy to:
    Run language model recipes in different modes: humans, human+LM, LM
    Inspect the execution traces in your browser for debugging
    Define and use new language model agents, e.g. chain-of-thought agents
    Run recipes quickly by parallelizing language model calls
    Reuse component recipes such as question-answering, ranking, and verification
    ICE looks like this:
    Factored Cognition Primer
    The Factored Cognition Primer is a tutorial that explains (among other things) how to:
    Implement basic versions of amplification and debate using ICE
    Reason about long texts by combining search and generation
    Run decompositions quickly by parallelizing language model calls
    Use verification of answers and reasoning steps to improve responses
    The Primer looks like this:
    If you end up using either, consider joining our Slack. We think that factored cognition research parallelizes unusually well and would like to collaborate with others who are working on recipes for cognitive tasks.
    Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.
    3 min
  • AF - AI Safety Endgame Stories by Ivan Vendrov
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: AI Safety Endgame Stories, published by Ivan Vendrov on September 28, 2022 on The AI Alignment Forum.
    Assume you are in the set of possible worlds where AI takeover happens by default. If you do nothing, then at some point in the 21st century the AI lab Magma develops a transformative AI system. Magma employees perform a number of safety checks, conclude the system is safe enough, and deploy it. They deploy it slowly and incrementally, with careful monitoring. But despite their efforts, the system turns out to be unsafe and the monitoring insufficient, triggering a cascade of events eventually leading to an existential catastrophe.
    I’ll refer to this sequence of events as the “baseline story” going forward.
    Assume further that you’re in the narrower set of worlds where this AI catastrophe is contingent on your actions. In other words, there exists a sequence of actions you (or your organization) can take that averts catastrophe, a decisive intervention. Not necessarily a pivotal act, an intervention that averts all existential risk from AI. Just an intervention that prevents this specific Magma catastrophe, giving humanity some breathing room, perhaps only a few months or years.
    Let’s try to understand what this decisive sequence of actions could look like. It’s tempting to start at the beginning of the sequence and think about what the first few actions look like. Unfortunately, the most probable starting actions are “meta” actions like thinking really hard, talking to experts, or recruiting more people to work on the problem. These are the same kinds of actions that any successful project starts with! So it doesn’t help us constrain the space of decisive interventions.
    Instead, it’s more helpful to start with the endgame: how, in the end, did your actions change the baseline story and avert catastrophe? And what were the last nodes in the causal chain leading up to the change?
    At the most abstract level, the baseline story has the following structure. A social process (Magma) instantiates a technological artifact (unsafe AI) which destroys the world. There are three objects here, and any change to the story requires changing the properties of at least one of them.
    This leads naturally to a 3-way categorization of endgame stories, though the real endgame story will probably involve some combination of the three:
    Changing the technology. You changed something about the technology that Magma had access to, which delayed the deployment or made it safe.
    Changing the decision maker. You changed something about Magma, or more broadly the social decision process that led to the unsafe deployment.
    Changing the broader world. You changed something about the broader world that made it resilient to Magma’s deployment decision.
    In the rest of this post I'll use this categorization to walk through a diverse array of endgame stories.
    Changing the Technology
    Differential Development of Safety
    Let’s start with the broad endgame story that most technical alignment research is aimed at, differential development of safety technology (a special case of differential technological development):
    You develop technology that makes AI safer, with mild competitiveness penalties. Because of your direct work, the technology is developed N years faster than it would have been by default. Magma’s deployment falls within that N-year window, so they use your technology, so their deployment does not lead to catastrophe.
    Can we make this story more concrete? How exactly does the new technology prevent catastrophe? The simplest path involves finding decisive safety augmentation, something like “add an adversarial robustness term to the training objective” - a conceptual breakthrough that Magma adopts immediately because it is relatively easy to implement and aligned with business objectives.
    Another s...
    20 min
  • EA - The sense of a start by Gavin
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: The sense of a start, published by Gavin on September 28, 2022 on The Effective Altruism Forum.
    Some attempts to feel the size of the future:
    ChronoZoom:
    [woosh]
    Will Macaskill:
    Imagine living, in order of birth, through the life of every human being who has ever lived... Your life lasts for almost four trillion years in total. For a tenth of that time, you’re a hunter-gatherer, and for 60 percent you’re an agriculturalist. You spend a full 20 percent of your life raising children, a further 20 percent farming, and almost 2 percent taking part in religious rituals. For over 1 percent of your life you are afflicted with malaria or smallpox. You spend 1.5 billion years having sex and 250 million giving birth. You drink forty-four trillion cups of coffee. As a colonizer, you invade new lands; as the colonized, you suffer your lands taken from you. You feel the rage of the abuser and the pain of the abused. For about 10 percent of your life you are a slaveholder; for about the same length of time, you are enslaved.
    But now imagine that you live all future lives, too. Your life, we hope, would be just beginning. Even if humanity lasts only as long as the typical mammalian species (one million years), and even if the world population falls to a tenth of its current size, 99.5 percent of your life would still be ahead of you. On the scale of a typical human life, you in the present would be just five months old. And if humanity survived longer than a typical mammalian species—for the hundreds of millions of years remaining until the earth is no longer habitable, or the tens of trillions remaining until the last stars burn out — your four trillion years of life would be like the first blinking seconds out of the womb.
    Carl Sagan:
    If history were a football field, all of human history would occupy an area the size of my hand.
    Bryan Magee:
    There are always some human beings who live to be a hundred. More do so today than ever before, but there have always been some... It comes as a shock to realise that the whole of civilisation has occurred within the successive lifetimes of sixty people -- which is the number of friends I squeeze into my living room when I have a drinks party. Twenty people take us back to Jesus, twenty-one to Julius Caesar.
    Samuel Arbesman:
    "Me (b. 1981)
    Harold C. Urey, American chemist, Nobel Prize laureate (b. 1893)
    Lucius Quintus Cincinnatus Lamar, U.S. Supreme Court justice (b. 1825)
    William Waldegrave, 1st Baron Radstock, Governor of Newfoundland (b. 1753)
    George Berkeley, Irish philosopher (b. 1685)
    King Charles II of England, Scotland and Ireland (b. 1630)
    Johannes Kepler, German astronomer (b. 1571)
    Shimazu Takahisa, Japanese samurai and warlord (b. 1514)
    Donato Bramante, Italian architect (b. 1444)
    Leonardo Bruni, Italian humanist (b. 1374)
    Petrarch, Italian poet (b. 1304)
    Emperor Go-Fukakusa of Japan (b. 1243)
    Hubert de Burgh, 1st Earl of Kent (b. circa 1160)
    Eric Jedvardsson, king of Sweden since 1156 (b. c. 1120)
    Gerard Thom (The Blessed Gerard), founder of the Knights Hospitaller (b. c. 1040)
    King Duncan I of Scotland (b. 1001)"
    Andri Snær Magnason:
    262 years. That’s the length of time you connect across. You’ll know the people who span this time. Your time is the time of the people you know and love, the time that molds you, and your time is the time of the people you will know and love, the time that you will shape. You can touch 262 years with your bare hands. Your great grandma taught you, you will teach your great granddaughter, you can have a direct impact on the future right up to the year 2186.
    Sydney Trent:
    The whipping post. The lynching tree. The wagon wheel. They were the stories of slavery, an inheritance of fear and dread, passed down from father to son. The boy, barely 5, would listen, awed, as his father spoke of life in Virgini...
    10 min
  • EA - Reasoning Transparency by Lizka
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Reasoning Transparency, published by Lizka on September 28, 2022 on The Effective Altruism Forum.
    I think “reasoning transparency” (and/or epistemic legibility) is a key value of effective altruism.
    As far as I can tell, the key piece of writing about it is this Open Philanthropy blog post by Luke Muehlhauser, which I’m cross-posting it to the Forum with permission. We also have a topic page for "reasoning transparency" — you can see some related posts there.
    Published: December 01, 2017 | by Luke Muehlhauser
    We at the Open Philanthropy Project value analyses which exhibit strong “reasoning transparency.” This document explains what we mean by “reasoning transparency,” and provides some tips for how to efficiently write documents that exhibit greater reasoning transparency than is standard in many domains.
    In short, our top recommendations are to:
    Open with a linked summary of key takeaways. [more]
    Throughout a document, indicate which considerations are most important to your key takeaways. [more]
    Throughout a document, indicate how confident you are in major claims, and what support you have for them. [more]
    1 Motivation
    When reading an analysis — e.g. a scientific paper, or some other collection of arguments and evidence for some conclusions — we want to know: “How should I update my view in response to this?” In particular, we want to know things like:
    Has the author presented a fair or biased presentation of evidence and arguments on this topic?
    How much expertise does the author have in this area?
    How trustworthy is the author in general? What are their biases and conflicts of interest?
    What was the research process that led to this analysis? What shortcuts were taken?
    What rough level of confidence does the author have in each of their substantive claims?
    What support does the author think they have for each of their substantive claims?
    What does the author think are the most important takeaways, and what could change the author’s mind about those takeaways?
    If the analysis includes some data analysis, how were the data collected, which analyses were done, and can I access the data myself?
    Many scientific communication norms are aimed at making it easier for a reader to answer questions like these, e.g. norms for ‘related literature’ sections and ‘methods’ sections, open data and code, reporting standards, pre-registration, conflict of interest statements, and so on.
    In other ways, typical scientific communication norms lack some aspects of reasoning transparency that we value. For example, many scientific papers say little about roughly how confident the authors are in different claims throughout the paper, or they might cite a series of papers (or even entire books!) in support of specific claims without giving any page numbers.
    Below, I (Luke Muehlhauser) offer some tips for how to write analyses that (I suspect, and in my experience) make it easier for the reader to answer the question, “How should I update my views in response to this?”
    2 Example of GiveWell charity reviews
    I’ll use a GiveWell charity review to illustrate a relatively “extreme” model of reasoning transparency, one that is probably more costly than it’s worth for most analysts. Later, I’ll give some tips for how to improve an analysis’ reasoning transparency without paying as high a cost for it as GiveWell does.
    Consider GiveWell’s review of Against Malaria Foundation (AMF). This review.
    .includes a summary of the most important points of the review, each linked to a longer section that elaborates those points and the evidence for them in some detail.
    .provides detailed responses to major questions that bear on the likely cost-effectiveness of marginal donations to AMF, e.g. “Are LLINs targeted at people who do not already have them?”, “Do LLINs reach intended destinations?”, “Is there roo...
    29 min