The Nonlinear Library

The Nonlinear Library

By The Nonlinear FundEducation
Download on the App Store

All The Nonlinear Library episodes

  • EA - Poorly considered throwaway statements harm the reputation of EA by Indahouse
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Poorly considered throwaway statements harm the reputation of EA, published by Indahouse on September 12, 2022 on The Effective Altruism Forum.
    Confidence: Spent 1 hour writing this, perhaps the write-up is rusty and missing some things, but I think the claims hold true
    Introduction
    Over the last few months, there’s been increasing attention drawn to EA. Iit seems likely that this may continue (although perhaps the recent launch of WWOTF represents a spike, I think there is a strong upward trend). In light of this, I think that it’s really important to make sure broad claims and statements are interrogated, especially if they might be distributed widely.
    Apologies for the brief and non-exhaustive post, but here I will quickly make a brief checklist of things to consider when making broad claims, using a worked example of a case study example in which I think this was done poorly. In a quick check, this rough framework seemed to work for other claims, although I imagine some modification and addition of criteria might be needed depending on the context
    Statement:
    Pre-checklist scoping
    Ask yourself three questions
    How bold is this claim?
    How widely disseminated will this claim be?
    How bad would the downside of this claim being false / misleading/ being misinterpreted be?
    Use these questions to guide how much time you spend on the below checklist
    Worked example
    This is quite a bold claim that I think will be shocking to many if true
    Although I imagine this interview was quite long and it was probably unclear at the time if this would make the article, I think we can assume that in expectation, this claim would have wide distribution
    If false or misinterpreted, this claim could undermine whether EA is academically and intellectually rigorous, and could also lead people to think the movement only cares about life expectancy over other instrumental health, income and social goals. In fact, in talking to many senior and intelligent people in government, policy and other sectors, they saw this claim and this is exactly what they thought. Therefore, the downside risk of making this claim seems quite high
    Conclusion: from this scoping, it seems worthwhile to spend at least a moderate amount of time on the checklist below
    Checklist
    1. Is this statement factually accurate and appropriately cited
    Some of the questions you might consider are
    Where is this statement from?
    Which context is it based on?
    What approach was used
    How confident am I in its conclusion?
    Worked example:
    In the FT statement, none of this is made obvious (caveat: perhaps Will made these, and these were dropped by the author; again, I am not trying to specifically target and criticise this statement itself, more use it as a case study)
    This statement appears to be from this source
    Based on the UK setting
    Used a modelling approach
    Low confidence
    Conclusion: Based on this, I might say: “I read a modelling study a few years ago that claimed that eradicating cancer would only reduce life expectancy by 2 years. I believe they were looking at a UK population, but this is certainly an interesting result regardless.”
    2. Make it clear within what ethical framework/ world view your statement has the most effect
    Make it clear what your values are and what world view your statement makes most sense within; this seems especially important for those working in the longtermist space.
    Worked example
    In a framework where we care a lot about the long term future of humanity, or we care about life extension, perhaps this is an appropriate statement. But in any other reasonably frameworks, it might not be
    Eradicating cancer health effects- 250 million DALYS per year, and 10 million deaths (Kocarnik 2022)
    Eradicating cancer income effects- highest burden of any disease, 1.16 trillion per year (WHO)
    Eradicating cancer social effects...
    6 min
  • EA - My closing talk at EAGxSingapore by Dion
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: My closing talk at EAGxSingapore, published by Dion on September 12, 2022 on The Effective Altruism Forum.
    I gave this talk at the end of EAGxSingapore, which I helped organize. It has been modified for clarity.
    Before I start the talk proper, I wanted to get a show of hands – if someone in this conference needs help, who would be willing to lend a helping hand, whether it's sending a message, reviewing a draft or hopping onto a call?
    (The majority, if not all, people raised their hands)
    For everyone new to EA conferences, take a look around! If there's someone that you wanted to reach out to but didn't send that message, check if they have their hands up. If they do, you should ask them for a call!
    I'll get back to why I asked you guys to raise your hands later.
    When preparing for this talk, the best advice I got was that I should be authentic, speak from my own experiences and not try to be someone I'm not. So I'm sharing a little about my first conference, what I took away from it, and what I hope you will take away from EAGxSingapore.
    My first EA conference was EAG London 2022. Like many of you, I watched Amy Labenz, Will MacAskill, and Benjamin Todd and I was so inspired. I wanted to be an effective altruist and improve the world. So, I applied for the conference and got accepted.
    A wave of fear hit me a few days before I was scheduled to fly. I have made an insane decision to meet a bunch of online strangers halfway across the world. It was something that I struggled to explain to my parents, and many of my non-EA friends had doubts about what I was doing. It all seemed like a terrible idea.
    It didn't stop there; scheduling 1-1s for someone introverted and new to the community was horrifyingly scary. All the people on Swapcard seemed so amazing and out of my league- researchers, specialists, and directors working on incredible projects. And scheduling 1-1s is only the first step because once you have a successful 1-1, you need to follow up. This means starting something, applying for a job, looking for collaborators or hiring someone. It also means that you'll constantly be stepping out of your comfort zone to grow, but it can also be challenging and nerve-wracking.
    So when people tell me that a common criticism of EA is that we are an emotionless bunch that only cares about data and numbers, I disagree because I associate EA with this fear and anxiety. However, beyond this fear and anxiety, there are two more things that I strongly feel when I'm around EAs.
    The first one is hope. Don't you think it is ridiculously and wondrously hopeful how so many of us think we can stop an existential risk? That something that can wipe out the whole of humanity can be mitigated by us all coming together and working on the world's most pressing issues. I think that that's the magical part of coming to conferences - that somehow so many of you have come together because we all believe we can do some good in the world. This makes me incredibly hopeful for the future.
    The second thing that I feel is warmth. The 1-1s were scary. However, it was also heartwarming how so many of these fantastic people made time for me, especially when many would not have gained value from it. People were willing to be mentors, guide me along my journey, and, even more importantly, I met lifelong friends. People who were there to check in on my mental health, gossip with me, and people who I hope will stay in my life far beyond my EA commitments. And this is something that I hope you guys will find as well.
    What I'm trying to say is that EA can be hard. It can be very demanding of you and your worldviews. And I feel kind of bad because I know it's difficult, but I'm asking you to take a leap of faith. Here are my three reasons why.
    The first reason is that I think you are worth it. EAGxSingapore was...
    8 min
  • EA - An experiment eliciting relative estimates for Open Philanthropy’s 2018 AI safety grants by NunoSempere
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: An experiment eliciting relative estimates for Open Philanthropy’s 2018 AI safety grants, published by NunoSempere on September 12, 2022 on The Effective Altruism Forum.
    Summary
    I present the design and results of an experiment eliciting relative values from six different researchers for the nine large AI safety grants Open Philanthropy made in 2018.
    The specific elicitation procedures I used might be usable for rapid evaluation setups, for going from zero to some evaluation, or for identifying disagreements. For weighty decisions, I would recommend more time-intensive approaches, like explicitly modelling the pathways to impact.
    Background and motivation
    This experiment follows up on past work around relative values (1, 2, 3) and more generally on work to better estimate values. The aim of this research direction is to explore a possibly scalable way of producing estimates and evaluations. If successful, this would bring utilitarianism and/or longtermism closer to producing practical guidance around more topics, which has been a recurring thread in my work in the last few years.
    Methodology
    My methodology was as follows:
    I selected a group of participants whose judgment I consider to be good.
    I selected a number of grants which I thought would be suitable for testing purposes.
    Participants familiarized themselves with the grants and with what exactly they ought to be estimating.
    Participants made their own initial estimates using two different methods:
    Method 1: Using a utility function extractor app.
    Method 2: Making a “hierarchical tree” of estimates.
    For each participant, I aggregated and/or showed their two estimates side by side, and asked them to make a best guess estimate.
    I took their best guess estimates, and held a discussion going through each grant, making participants discuss their viewpoints when they had some disagreements.
    After holding the discussion, I asked participants to make new estimates.
    Overall, the participants took about two to three hours each to complete this process, roughly divided as follows:
    10 to 30 mins to familiarize themselves with the estimation target and to re-familiarize themselves with the grants
    20 to 40 mins to do the two initial estimates
    5 to 30 mins to give their first best guess estimate after seeing the result of the two different methods
    1h to hold a discussion
    5 to 30 mins to give their resulting best guess estimate
    The rest of this section goes through these steps individually.
    Selection of participants
    I selected participants by asking friends or colleagues whose judgment I trust, and who had some expertise or knowledge of AI safety. In particular, I selected participants who would be somewhat familiar with Open Philanthropy grants, because otherwise the time required for research would have been too onerous.
    The participants were Gavin Leech, Misha Yagudin, Ozzie Gooen, Jaime Sevilla, Daniel Filan and another participant who prefers to remain anonymous. Note that one participant didn’t participate in all the rounds, which is why some summaries contain only five datapoints.
    Selection of grants
    The grants I selected were:
    AI Impacts — General Support (2018): $100,000
    Machine Intelligence Research Institute — AI Safety Retraining Program: $150,000
    Open Phil AI Fellowship — 2018 Class: $1,135,000
    Ought — General Support (2018): $525,000
    Oxford University — Research on the Global Politics of AI: $429,770
    Stanford University — Machine Learning Security Research Led by Dan Boneh and Florian Tramer: $100,000
    UC Berkeley — AI Safety Research (2018): $1,145,000
    Wilson Center — AI Policy Seminar Series: $400,000
    These are all the grants that Open Philanthropy made to reduce AI risk in 2018 above a threshold of $10k, according to their database. The year these grants were made is long enough ago that we have some information...
    23 min
  • AF - Ideological Inference Engines: Making Deontology Differentiable by Paul Bricman
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Ideological Inference Engines: Making Deontology Differentiable, published by Paul Bricman on September 12, 2022 on The AI Alignment Forum.
    Rather, making deontology play well with differentiable systems trained end-to-end.
    This post is part of my hypothesis subspace sequence, a living collection of proposals I'm exploring at Refine. Preceded by oversight leagues.
    TL;DR: An ideological inference engine is a mechanism for automatically refining a given propositional representation of human values (e.g. a normative charter, a debate stance) in an attempt to disambiguate and generalize it to novel situations. While the inference algorithm and the seed representation form the crux of the system, a multi-modal entailment verifier is employed to order possible agent behaviors based on their compatibility with the estimated ideology. This proposal then describes a way of instilling deontological drives in prosaic systems while maintaining the appeal of end-to-end differentiation. Ideological inference engines draw on ideas from traditional expert systems, but replace much of the clunky symbolic manipulation with contemporary LLMs and NLI models.
    Intro
    Ideological inference engines are a slightly more general framework than oversight leagues, in the sense they rely on several global assumptions, but each more concrete instance of the proposal requires new assumptions when designing the seed representation, the inference algorithm, and the entailment verifier. Here's a non-exhaustive list of global assumptions:
    Assumption 1, "Small Seed To Big Tree": Given a suitable inference algorithm and a finite propositional representation of human values, it is possible to estimate human values arbitrarily well given arbitrary amounts of compute. "Arbitrarily well" refers to there being an arbitrarily low error in the estimation. In the limit, the seed knowledge base would grow into the True Name of human values in propositional form.
    Assumption 2, "Linear Capability Ordering": Similar to the assumption invoked in oversight leagues, this states that a system composed of a fixed knowledge base and a fixed entailment verifier would eventually be gamed by an agent whose capability is constantly increasing. This is due to the more complex agent becoming able to exploit the inaccuracies of the knowledge base with respect to actual human values.
    Assumption 3, "Quantifiable Propositional-Behavioral Gap": The compatibility of a proposed behavioral sequence and a propositional representation of human values is computable. There is a fundamental relation between one's values and one's actions, and we can measure it. A close variant appears to be invoked in the CIRL literature (i.e. we can read one's values off of one's behavior) and in Vanessa Kosoy's protocol (i.e. we can narrow in on an agent's objective based on its behavioral history).
    Assumption 4, "Avoidable Consequentialist Frenzy": It's possible to prevent the agent-in-training from going on a rampage in terms of an outcome-based objective (e.g. get human upvotes) relative to a simultaneous process-based objective (i.e. the present deontological proposal). This might be achieved by means of myopia or impact measures, but we're not concerned with those details here — hence, an assumption.
    Together, those global assumptions allow for a mechanism for approaching the human values in propositional form, before employing the resulting representation to nudge an ML model towards being compatible with it.
    Proposal
    Ideological inference engines (IIE) are a modular framework for implementing such a mechanism. Each such system requires the following slots to be filled in with more concrete designs, each of which has attached assumptions related to whether it's actually suited to its role:
    Knowledge Base (KB): The way human values are supposed to be ...
    24 min
  • LW - AI Risk Intro 1: Advanced AI Might Be Very Bad by TheMcDouglas
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: AI Risk Intro 1: Advanced AI Might Be Very Bad, published by TheMcDouglas on September 11, 2022 on LessWrong.
    This is a submission for the AI Safety Public Materials contest.
    Introduction
    If human civilisation is destroyed this century, the most likely cause is advanced AI systems. This is perhaps a surprising conclusion, but one that many people who think about the topic keep coming to. While it is not easy to describe the case for risks from advanced AI in a single piece, here we make an effort that assumes no prior knowledge. Rather than try to argue from theory straight away, we approach it from the angle of what computers actually can and can’t do.
    The Story So Far
    (This section can be skipped if you understand how machine learning works and what it can and can’t do today)
    Let’s say you want a computer to do some complicated task, for example learning chess. The computer has no understanding of high-level things like “chess”, “board”, “piece”, “move”, or “win” - it only understands how to do a small set of things. Your task as the programmer is to break down the high-level goal of “beat me at chess” into simpler and simpler steps, until you arrive at a simple mechanistic description of what the computer needs to do. If the computer does beat you, it’s not because it had any new insight into the problem, but rather because you were clever enough to find some set of steps that, carried out blindly in sufficient speed and quantity, overwhelms whatever cleverness you yourself can apply during the game. This is how Deep Blue beat Kasparov, and more generally how most software and the so-called “Good Old-Fashioned AI” (GOFAI) paradigm works.
    Programs of this type can be powerful. In addition to beating humans at chess, they can calculate shortest routes on maps, prove maths theorems, mostly fly airplanes, and search all human knowledge. Programs of this type are responsible for the stereotypical impression of computers as logical, precise, uncreative, and brittle. They are essentially executable logic.
    Many people hoped that you could write programs to do “intelligent” things. These people were right - after all, ask almost anyone before Deep Blue won whether playing chess counts as “intelligence”, they’d have said yes. But “classical” programming hit limitations, in particular in doing “obvious” things like figuring out whether an image is of a cat or a dog, or being able to respond in English. This idea that abstract reasoning and logic are easy but humanly-intuitive tasks are hard for computers came to be known as Moravec’s paradox, and held back progress in AI for a long time.
    There is another way of programming - machine learning (ML) - going back to the 1950s, almost as far as classical programming itself. For a long time, it was held back by hardware limitations (along with some algorithmic and data limitations), but thanks to Moore’s law hardware has advanced enough for it to be useful for real problems.
    If classical programming is executable logic, ML is executable statistics. In ML, the programmer does not define how the system works. The programmer defines how the system learns from data.
    The “learning” part in “machine learning” makes it sound like something refined and sensible. This is a false impression. ML systems learn by going through a training process that looks like this:
    Step 1: you define a statistical model. This takes the form of some equation that has some unknown constants (“parameters”) in it, and some variables where you plug in input values. Together, the parameters and input variables define an output. (The equations in ML can be extremely large, for example with billions of parameters and millions of inputs, but they are very structured and almost stupidly simple.)
    Step 2: you don’t know what parameters to put in the equation, but you c...
    48 min
  • EA - Bring legal cases to me, I will donate 100% of my cut of the fee to a charity of your choice by Brad West
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Bring legal cases to me, I will donate 100% of my cut of the fee to a charity of your choice, published by Brad West on September 11, 2022 on The Effective Altruism Forum.
    I am a lawyer that practices near Chicago, Illinois and I also started a 501c3 charity called the Consumer Power Initiative. CPI is interested in using our economic power to benefit charities.
    To that end, I would like to donate 100% of my cut of any business I bring in to the charity of the choice to whoever brings in the business. I get a third of our firm's fee for business that I bring in. Thus, in a personal injury case, we would get a third of any money generated, so I would generate my cut (a ninth) to the charity of your choice. In an hourly case, I would get a third of the fees generated.
    Randall Wolff & Associates is the firm I work for and we do a great job for our clients; you can check out our high score on Google Reviews. We would love to take your personal injury, worker's compensation, bankruptcy, or divorce cases. We also would be interested in seeing if we can help you with commercial disputes.
    I would like EAs more generally to do stuff like this. Are you a professional where rainmaking could be valuable to your business and are interested in helping effective charities anyway? Make it explicitly known that you will direct your cut of the fees to a specific charity, or a charity of their choice. Earning to Give can be much more effective if you use the fact that you donate your earnings to generate business.
    If you would like to do this, please initiate any conversations about representation by either emailing [email protected] or calling 847-222-9465 and speaking with Brad West first. It needs to be clear that I generated the business for me (and thus your charity) to get the fee.
    If you're interested in learning more about the Consumer Power Initiative, check out our newsletter. Also check out BOAS, a company in the EU that sells sustainable baby products and directs all profits to charities.
    CPI Newsletter:%40gmail.com&usp=drive_fs
    BOAS website: boas.co
    Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.
    3 min
  • EA - My emotional reaction to the current funding situation by Sam Brown
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: My emotional reaction to the current funding situation, published by Sam Brown on September 11, 2022 on The Effective Altruism Forum.
    I’m allowed to spend two days a week at Trajan House, a building in Oxford which houses the Center for Effective Altruism (CEA), along with a few EA-related bodies. Two days is what I asked for, and what I received. The rest of the time I spend in the Bodleian Library of the University of Oxford (about £30/year, if you can demonstrate an acceptable “research need”), a desk at a coworking space in Ethical Property (which houses Refugee Welcome, among other non-EA bodies, for £200/month), Common Ground (a cafe/co-working space which I’ve recommended to people as a place where the staff explicitly explain, if you ask, that you don’t need to order anything to stay as long as you like), a large family house I’m friends with, and various cafes and restaurants where I can sit for hours while only drinking mint tea.
    I’m allowed to use the hot-desk space at Trajan House because I’m a recipient of an EA Long Term Future Fund grant, to research Alignment. (I call this “AI safety” to most people, and sometimes have to explain that AI stands for Artificial Intelligence.) I judged that 6 months of salary at the level of my previous startup job, with a small expenses budget, came to about £40,000. This is what I asked for, and what I received.
    At my previous job I thought I was having a measurable, meaningful impact on climate change. When I started there, I imagined that I’d go on to found my own startup. I promised myself it would be the last time I’d be employed.
    When I quit that startup job, I spent around a year doing nothing-much. I applied to Oxford’s Philosophy BPhil, unsuccessfully. I looked at startup incubators and accelerators. But mostly, I researched Alignment groups. I visited Conjecture, and talked to people from Deep Mind, and the Future of Humanity Institute. What I was trying to do, was to discern whether Alignment was “real” or not. Certainly, I decided, some of these people were cleverer than me, more hard-working than me, better-informed. Some seem deluded, but not all. At the very least, it’s not just a bunch of netizens from a particular online community, whose friend earned a crypto fortune.
    During the year I was unemployed, I lived very cheaply. I’m familiar with the lifestyle, and – if I’m honest – I like it. Whereas for my holidays while employed I’d hire or buy a motorbike, and go travelling abroad, or scuba dive, instead my holidays would be spent doing DIY at a friend’s holiday home for free board, or taking a bivi bag to sleep in the fields around Oxford.
    The exceptions to this thrift were both EA-related, and both fully-funded. In one, for which my nickname of “Huel and hot-tubs” never caught on, I was successfully reassured by someone I found very smart that my proposed Alignment research project was worthwhile. In the other, I and others were flown out to the San Francisco Bay Area for an all-expenses-paid retreat to learn how to better build communities. My hotel room had a nightly price written on the inside of the door: $500. Surely no one ever paid that. Shortly afterwards, I heard that the EA-adjacent community were buying the entire hotel.
    While at the first retreat, I submitted my application for funding. While in Berkeley for the second, I discovered my application was successful. (“I should hire a motorbike, while I’m here.” I didn’t have time, between networking opportunities.) I started calling myself an “independent alignment researcher” to anyone who would listen and let me into offices, workshops, or parties. I fit right in.
    At one point, people were writing plans on a whiteboard for how we could spend the effectively-infinite amount of money we could ask for. Somehow I couldn’t take it any more, so I ...
    9 min
  • EA - Aversion to Happiness Across Cultures: A Review of Where and Why People are Averse to Happiness by timfarkas
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Aversion to Happiness Across Cultures: A Review of Where and Why People are Averse to Happiness, published by timfarkas on September 11, 2022 on The Effective Altruism Forum.
    Context
    This is a paper I just randomly found again in my saved reading list that seems relevant to the EA Forum.
    It talks about the concept of 'aversion to happiness' which I have never heard of before in EA circles. This paper might thus be interesting to EAs who consider some form of happiness/good qualia intrinsically valuable and certain measures of happiness as their core unit of intrinsic value (of which there are many in my experience):
    Abstract
    "A common view in contemporary Western culture is that personal happiness is one of the most important values in life. For example, in American culture it is believed that failing to appear happy is cause for concern. These cultural notions are also echoed in contemporary Western psychology (including positive psychology and much of the research on subjective well-being). However, some important (often culturally-based) facts about happiness have tended to be overlooked in the psychological research on the topic. One of these cultural phenomena is that, for some individuals, happiness is not a supreme value. In fact, some individuals across cultures are averse to various kinds of happiness for several different reasons. This article presents the first review of the concept of aversion to happiness. Implications of the outcomes are discussed, as are directions for further research."
    Rough Review and Summary
    I have skimmed some of the paper, here are my thoughts:
    A lot of it focuses on the effects of outward expression of happiness on other people's happiness or the fact that embracing happiness might lead to more subsequent suffering, which is still compatible with happiness being a core unit of value/morality. I thus do not think these claims are very relevant to EA.
    However, the paper also claims that in some cultures, individuals do not consider being happy as morally desirable or even consider being happy as bad - which would be incompatible with EA notions of considering happiness intrinsically valuable:
    "People aren’t just averse to happiness because it might lead to [subsequent unhappiness], however; some individuals and some cultures tend to believe that happiness is worthy of aversion because being happy can make someone a worse person (both morally and otherwise). Again, we found evidence for this belief in both non-Western and Western cultures. First we discuss beliefs that happiness is worthy of aversion because it can make someone a morally worse person, and then we discuss beliefs that happiness is worthy of aversion because it can make someone less creative."
    It gives a few examples of that and later goes on to summarize and conclude:
    "It should be noted that [this paper] casts little doubt on the intrinsic value of most kinds of happiness. Indeed, while happiness and the pursuit of certain kinds of happiness are widely believed to have negative effects for some people in some cases, happiness is, in and of itself, still a positive experience for most people and according to most of the common conceptions of happiness.
    Nevertheless, it should not be in doubt that many individuals and cultures do tend to be averse to some forms of happiness, especially when taken to the extreme, for many different reasons.
    These considerations show that equating happiness with the supreme universal good is dangerous unless each culture (or individual!) were to create and be assessed by its own definition of happiness."
    It also makes a few interesting points about international comparisons of happiness across nations being potentially flawed due to response bias caused by aversion to happiness.
    Thanks for listening. To help us out with The Nonlinear L...
    4 min
  • EA - Join ASAP (AI Safety Accountability Programme) 🚀 by TheMcDouglas
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Join ASAP (AI Safety Accountability Programme) 🚀, published by TheMcDouglas on September 10, 2022 on The Effective Altruism Forum.
    Crossposted to LessWrong.
    TL;DR
    I'm creating a Slack group for people who are interested in working in AI safety at some point in the future, but who aren't working on it right now, and would like extra accountability and motivation while they pursue their goals.
    Join with this link!
    Why am I creating this?
    I just spent an awesome summer in Berkeley doing MLAB, surrounded by people who are really passionate about AI safety, and this definitely had a positive impact on my level of motivation. I think trying to recreate some (even much weaker) version of that would be really valuable. An accountability system is the most basic version of this, because making commitments to other people is a really nice way of motivating yourself to get shit done!
    I've spoken to a few people from MLAB, and several seem to agree (at least five participants have mentioned to me that they'd like to join a group like this).
    How will this work?
    (Note - this might all be changed depending on how many people join, and their suggestions & preferences. Hopefully by the end of next week, the group will be larger and we will have made many improvements to this basic design!)
    The core mechanism of the group will be everyone posting regular short updates (might be Slack message, or filling out a Google Form) (maybe once per 2 weeks) summarising what they've done over that period. For instance:
    Books you've read, or courses you've taken, or progress in structured self-study like this
    Blog posts you've written
    Projects you've done, or are doing, and your progress on them
    Companies or other opportunities you've applied for
    Dank AI memes you've designed
    There will also be optional extra commitment mechanisms like weekly Zoom calls. Also if enough people join (e.g. more than 6) then we'll probably divide people into smaller groups for personal check-ins, since larger groups tend to lead to individuals feeling less accountability. Progress reports will still be posted to the main Slack channel.
    Who is this group for?
    I expect it will be most useful for you if one or more of the following holds:
    You aren't yet contributing to AI alignment directly, but you think you might at some point in the future.
    You aren't necessarily surrounded by a group of people who are also working on AI safety.
    You have a particular idea for ways you want to skill up / projects you want to do / places you want to apply for, but lack the motivation to do it.
    You are interested in accountability systems, but don't have the ability to make regular time commitments.
    None of these are totally necessary, so feel free to join regardless if you think you'd benefit from this group.
    Why am I making this group, when there are other pre-existing groups that might work for this purpose?
    Three main reasons:
    I don't think size is necessarily an advantage in an accountability group. Larger groups can diminish the effectiveness, whereas smaller groups can often better supply accountability to each individual, and encourage a sense of community.
    I think having an entire Slack dedicated to this mechanism, rather than just one channel of a larger Slack group, has lots of benefits. For instance, AI Alignment Slack has a study buddies channel, but since this isn't the primary focus of this group I expect ASAP to have a comparative advantage in providing motivation and accountability.
    Many pre-existing AI safety related Slack / Discord groups have a much more specific topical focus (e.g. the Alignment Studies Slack group, which is mainly focused on the MIRI course list). I imagine most people would benefit from more flexibility, since everyone will probably be doing slightly different things.
    What is the end goal?
    I create...
    5 min
  • LW - A Pin and a Balloon: Anthropic Fragility Increases Chances of Runaway Global Warming by avturchin
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: A Pin and a Balloon: Anthropic Fragility Increases Chances of Runaway Global Warming, published by avturchin on September 11, 2022 on LessWrong.
    Alexey Turchin, [email protected]
    Anonymous coauthor 1
    Anonymous coauthor 2
    TL;DR: Because of survival bias we underestimate the location of climate tipping points and thus our world is more fragile to small anthropogenic actions. Therefore, human extinction because of runaway global warming is more probable.
    Abstract: Humanity may underestimate the rate of natural global catastrophes because of the survival bias (“anthropic shadow”). But the resulting reduction of the Earth’s future habitability duration is not very large in most plausible cases (1-2 orders of magnitude) and thus it looks like we still have at least millions of years.
    However, anthropic shadow implies anthropic fragility: we are more likely to live in a world where a sterilizing catastrophe is long overdue and could be triggered by unexpectedly small human actions. In the same way, an over-inflated toy balloon, which will soon burst, is very fragile.
    Anthropic fragility can manifest itself in the higher chances of runaway global warming. It has often been suggested that the Earth's atmosphere remained life-supporting for billions of years by sheer chance. Therefore, the survival bias can be strong. It is also known that Earth-like water worlds could experience transitions into deadly moisture greenhouse (mean T = 65C). This means that relatively small anthropogenic actions could put the climate above an unpredictable tipping point, which could lead to the moisture greenhouse. Thus, it is necessary to carry out urgent geoengineering studies and prepare to prevent an unexpected climate catastrophe.
    There are three main counterarguments against the existence of the anthropic shadow: self-indication assumption (SIA), past observers and the Gaia hypothesis; we show that they fail. It was proposed that SIA exactly compensates the anthropic shadow as an observer unlikely to find herself in a world with a strong anthropic shadow; however, there is a baseline level of the anthropic shadow for all habitable planets, similar to the rate of evolutionary transitions like abiogenesis. There are no “past observers” as qualified observers appeared only 50 years ago. Gaia hypothesis assumes existence of self-stabilizing feedback in climate, but new types of events like quick CO2 growth could override its coping ability.
    We present a list of other catastrophes that may have been underestimated because of the anthropic shadow, including collider catastrophes, nuclear war and even an alien invasion.
    We also hypothesised that human intelligence is more likely to emerge in an unstable world that is nearing its end and thus we get a new form of Doomsday argument.
    pdf with Appendix:
    Highlights:
    · The decrease of the expected habitability time of the Earth’s because of the survival
    bias (anthropic shadow) is not very large and thus it looks like that we still have at least millions of years.
    · But anthropic shadow implies anthropic fragility: we are more likely to live in a world where a catastrophe is long overdue and could be triggered by unexpectedly small human actions.
    · Catastrophic runaway global warming may be an example of such anthropic fragility, and we cannot predict where the tipping point is.
    · We should research urgent geoengineering to counter the unpredictable anthropic fragility of climate.
    · We also suggested a hypothesis that human intelligence is more likely to arise in an unstable world that is nearing its end.
    1. Introduction
    One formulation of the anthropic principle (Carter, 1974) is that observers can exist only in those worlds in which there are no conditions that prevent the appearance of the observers. Such conditions may take the form of fine-tun...
    1 hr 43 min