Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: 2-factor voting (karma, agreement) for EA forum?, published by david reinstein on June 25, 2022 on The Effective Altruism Forum.

As many of you know, on LessWrong there is now:

two axes on which you can vote on comments: the standard karma axis remains on the left, and the new axis on the right lets you show much you agree or disagree with the content of a comment.

I was thinking we should have this on EA Forum for the same reasons ... to avoid (i) agreement with the claim/position being confounded with (ii) liking the contribution to the discussion/community.

Reading the comments over there, it seems there are mixed reviews. Some key critiques:

Visual confusion and mental overload (maybe improvable with better formats)

It's often hard to discern what 'agree with the post' means.

My quick takes:

A. We might consider this for EAFo after LW works out the bugs (and probably the team is considering it)

B. Perhaps the 'agreement' axis should be something that the post author can add voluntarily, specifying what is the claim people can indicate agreement/disagreement with? (This might also work well with the metaculus prediction link that is in the works afaik).

What are your thoughts...? [1]

On two-factor voting for EA Forum overall

On "post author chooses what the agreement target ('central claim') is"

On whether the considerations here are different for EA Forum vs. LessWrong

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Worked Examples of Shapley Values, published by lalaithion on June 24, 2022 on LessWrong.

Three times in the past month, I've run across occasions where Shapley values were mentioned or would have been useful. There are a couple of good explainers of Shapley value already on the internet, but what most of them lack is a bunch of worked examples. I find that many times, the limiting factor in my ability to understand a concept is whether or not I've been exposed to multiple examples—so this post is where I'll be posting some of the examples I worked through to understand how Shapley value works.

I'll start with a brief overview of Shapley value, but first, a caveat. I think it's fine to skim most of the math in the overview. We won't really be using it that much, and I try to give an intuitive sense of what the equations mean before I present them. You could probably skip the entire overview and just go straight to the examples and not miss much.

Overview of Shapley Value

Suppose you have a group of people who work together to produce some sort of value (traditionally profit). How should you divide the credit for that value (or the actual profit) up among the individuals involved? One immediate suggestion might be "equally", but that doesn't necessarily satisfy intuitive notions of fairness. If a doctor performs life-saving surgery, should they receive equal credit for saving someone's life as the random person holding the door open for the doctor at the end of the surgery? If you're running a restaurant, and three servers spend all night running around serving people, and one server spends two hours on a smoke break, do all four deserve an equal amount of pay?

Shapley value provides a different way of computing what a fair division of value should be. Why use Shapley value? Well, like splitting things equally, it (a) divides all of the gains among all of the participants, (b) splits things equally between participants who contribute the same value, and (c) if there are two completely independent value-producing processes, then the assignment of value to each participant is equal to the sum of the value for that participant in each game. Unlike splitting things equally, it also (d) assigns no value to anyone who always contributes nothing, and in fact, it is the only assignment rule which satisfies all four of those constraints.

The formulas for computing the Shapley values can be found on its Wikipedia page.

The simplest way of computing Shapley value, the one we'll be using for most of this post, is to consider the synergy of a group. (Here we use synergy in the original sense, not in the buzzword-ified sense. If you've never heard it used as anything besides a buzzword, it means "the value of a group that is greater than the sum of its parts".) We'll follow Wikipedia's convention and use v(S) to notate the value produced by a set S, and w(S) to notate the synergy of some set of participants S. I'll break with Wikipedia's convention and use V(i) to indicate the Shapley value assigned to participant i (or subgroup R).

(Slightly obscure mathematical notation explainer footnote: )

The equation for synergy is

but I think in practice, it's easy for many questions to simply determine the synergy of a group by asking, "what value does this group provide that isn't already accounted for by smaller groups contained within this one?"

Once we have the synergy, we can find the Shapley value of each participant by considering the synergy of each group and dividing the synergy equally among participants. (In a way, we've gone one level up; instead of dividing the value produced equally, we divide the synergy equally.) Here it is, written in equation form

but I think that as we get into the examples, it will become much clearer.

Examples

Simple Factory Business

The most basic exam...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Linkpost: Robin Hanson - Why Not Wait On AI Risk?, published by Yair Halberstadt on June 24, 2022 on LessWrong.

Raises an interesting critique of AI Risk. I think he is wrong, but would be interested in how others would respond.

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Updated Deference is not a strong argument against the utility uncertainty approach to alignment, published by Ivan Vendrov on June 24, 2022 on The AI Alignment Forum.

Thesis: The problem of fully updated deference is not a strong argument against the viability of the assistance games / utility uncertainty approach to AI (outer) alignment.

Background: A proposed high-level approach to AI alignment is to have the AI maintain a probability distribution over possible human utility functions instead of optimizing for any particular fixed utility function. Variants of this approach were advocated by Stuart Russell in Human Compatible and by Hadfield-Menell et al in the CIRL paper. Adding utility uncertainty intuitively seems to provide a number of safety benefits relative to having a fixed objective, including:

Utility uncertainty gives the AI an incentive to adjust in response to a human operator's corrective actions.

Utility uncertainty weakens the AI's incentive to harm its human operators, since this might result in a permanent loss of utility-relevant information.

Utility uncertainty incentivizes the AI to avoid irreversible changes to the state of the world, since those might lead to permanently low utility.

Despite the high profile and intuitive appeal of utility uncertainty, almost none of the alignment researchers I know consider it a promising approach to AI alignment. The most common reason cited seems to be the problem of fully updated deference (e.g. Richard Ngo's alignment research exercises point to this as the reason for why CIRL doesn't solve the alignment problem).

In this post I will argue why fully updated deference should not be seen as a strong argument against utility uncertainty as approach to AI alignment. This is not meant as an argument in favor of the uncertainty approach; it may have other irresolvable difficulties which I discuss briefly in the conclusion.

Outline: The Arbital post that seems to be the canonical reference for updated deference contains many heuristic arguments and one concrete, worked-out example in the section Moral uncertainty and its relation to corrigibility. I will mostly engage with the example, and argue that

It conflates the problem of updated deference with the independent problem of prior mis-specification.

If we remove prior mis-specification, there is no problem in the limit of increasing AI capability.

The Problem of Updated Deference

The example in the post has an AI that is uncertain between three utility functions U1, U2, U3 whereas the human's true utility function is V. The AI believes that the utility that will be attained in each of the three possible worlds is

ui with AI assistance

vi if the human optimizes V without the AI's assistance (e.g. because the humans shut the AI down)

If the AI is much more powerful than humans, the argument goes, then uivi in any of the three worlds, so the AI will not let itself be shut down. The uncertainty doesn't help because the AI can choose to keep gathering information until it has fully updated. Since it's more powerful than the humans, it can gather that information more efficiently when it's not shut down, and therefore ignores the shutdown signal.

Factoring out prior mis-specification

The original example has the AI system assign probability 0 to the true human utility function V, presumably because its prior probability was 0. I think any advocate of the utility uncertainty approach would agree that assigning a nonzero prior to the true human utility function is critical for the approach to work. Describing such a prior abstractly is easy (just take the Solomonoff prior over programs), implementing a real CIRL agent that reasons with such a prior could be intractably difficult, but this is clearly a separate problem from "fully updated deference". So from now on w...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Feature request: voting buttons at the bottom?, published by Oliver Sourbut on June 24, 2022 on LessWrong.

This is a brief question/request regarding LessWrong and AlignmentForum UI and voting buttons.

Uncontroversial (I expect) factual statements

When I read a post or comment which fills more than one screen, I want to consume the whole content before providing any reinforcement or agreement signals.

But the vote buttons are rendered at the top of posts and comments, so I have to scroll back up again[1], and (especially if I'm reading through a thread of comments) it can therefore be tricky to keep track of my place.

A basic request

Could we have additional voting buttons rendered (perhaps optionally configurably?) at the bottom of posts and comments? This would make the above, presumably common, workflow that little bit more straightforward.

Or (more fiddly but maybe better), could the buttons float and follow me down the page, or some other fancy UI trick?

A more controversial normative claim

It seems like the factual statement of my reading/digesting/voting workflow above should be the default workflow. After all, how can you provide a reliable voting signal for something you haven't consumed?[2]

Would it make sense to always have the buttons exclusively at the bottom, to nudge[3] the 'correct' workflow to be marginally easier, and 'aberrant' workflows (like voting based on the author's name or the title or introduction)[4] to be marginally more difficult? What might the downsides be (beside the obvious developer opportunity cost)?

Even more controversial speculation

What if the karma/agreement tallies were also at the bottom? (i.e. unseen until after deliberate scroll)

It wouldn't in principle change my options as a reader, but it would make it easier to avoid anchoring and instead evaluate content on its merit (but I could still readily see the community consensus signal and adopt that information into my all-things-considered opinion).

This seems a lot easier to backfire, because often we filter what to even start reading based on more quickly-digested summaries (probably quite rightly), which include karma and the like, as well as recommendations. It might work better, for that reason, if at all, on comments than on top-level posts.

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Save the date: EAGxVirtual 2022, published by Alex Berezhnoi on June 24, 2022 on The Effective Altruism Forum.

Even with many in-person conferences happening this year, the demand for virtual interaction is still high. I’m excited to announce that a big virtual conference - EAGxVirtual 2022 - will take place on October 21-23. It will be hosted by EA Anywhere with the support of the CEA Events team.

Details on the conference

Our main goal is to help attendees identify the next steps to act based on EA principles wherever they are in the world and build stronger bonds within the community.

A lot of people recently engaged with the community through Intro Fellowships, Virtual Programs, and professional groups. But many of them have uncertainties about how to take action and don't have a good understanding of the EA landscape. We want to address that by facilitating valuable connections, highlighting relevant opportunities and resources, and inviting speakers who are working on concrete projects. There will be a range of talks, workshops, live Q&A sessions, Office Hours with experts, and facilitated networking.

Application process

The applications will be open in early September.

Sign-up here to be notified when applications open.

We expect attendees to be at least familiar with key EA ideas. If you’re a complete newcomer, consider signing up for the Introductory EA Program by July 24, such that you will have completed the Program in time for the event.

Content suggestions

We're putting together the program right now and we'd be happy to hear your suggestions. What kind of content would you like to see? As a virtual conference, we are not limited by the venue and have no need to mirror in-person conferences.

Feel free to put your suggestion in the comments or use this form.

We are looking forward to an inspiring conference with you!

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Cape Town Summer EA Office, published by jwpieters on June 24, 2022 on The Effective Altruism Forum.

TLDR:

We are exploring the feasibility of establishing an EA community office in Cape Town, South Africa. Building an EA hub office, in addition to employing full-time community builders and researchers in Africa, will allow for a major acceleration of community building on the continent. Additionally, an office in South Africa would allow EAs from northern hemisphere countries to work remotely in a much nicer environment with more sunlight during their winter months. If you’re interested in participating in this project, please fill in this form.

Background

There are a couple of projects within the EA community planning on opening more offices around the world, in the style of Trajan House in Oxford, and Lightcone and Constellation in Berkeley. These offices provide an optimized working environment for EA professionals, and occasionally for independent EAs moving transiently through those cities who are in need of a place to work. We believe that the case for co-working offices as impact multipliers is already quite strong. However, all of these offices are in the northern hemisphere, which is suboptimal because 1) it means there’s no EA co-working spaces for EAs in Africa, Asia, South America, and Australia and 2) when it's winter in the northern hemisphere, it is winter for everyone who works in an EA hub or office

The Plan

We want to run a pilot project/MVP to assess the potential value of this. There are a number of co-working office spaces available for short term rental in Cape Town. We can rent an office (for ~25 people) to use as a co-working space for EAs from around the world [prioritized if they live in 1) an LMIC without an EA hub city or office or 2) a cold dark place. physically]. Although creating a purpose-built space is more ideal in the long run, renting a co-working space for 6 months is a relatively cheap way to test the potential impact of this idea. We may also look into helping EAs find shared housing near the office.

If this project is successful, we believe that EAs from Africa and perhaps other LMICs will have access to a much more engaging community, thus increasing their feelings of cohesion with the movement. We also would like to see EAs from the northern hemisphere experience an improvement in quality of life during their stay. Since many EAs have the permanent opportunity to work remotely, we need not constrain ourselves to the dark and cold cities in which many EAs live between October and March.

We think it’s important that we don’t try to start new EA hubs arbitrarily. As such, we’d like to see very robust reasons for establishing something like this long term in Cape Town before putting significant resources towards it.

A vision for the future

It is 2035. There is a 75-100 person EA hub office in Cape Town CBD. The space is shared by full-time EAs, based in Cape Town, working on global health and wellbeing, animal welfare advocates, AI and biosecurity researchers, and community builders, including from EA for Muslims and EA for Christians. There is a large space in the office which is used for outreach and programming by the University of Cape Town and Stellenbosch University EA groups, and there are members of the greater African EA community working full time in the office for 6 months at a time on visiting fellowships. The office hosts regular African EA Summits, as well as talks by scholars. Between October and February every year, the office doubles in occupancy as EAs from the US, UK, and Europe come to work in Cape Town - some stay at the EA house down the road from the office.

Why Cape Town?

As anyone who’s been to Cape Town will attest, it is a top tier city. It has great weather and a good food scene with a strong plant-b...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Launching a donor circle for mental health, published by Joey on June 24, 2022 on The Effective Altruism Forum.

TLDR: The cause area of mental health is growing in popularity in the EA movement but it is also often limited by funding. As such, it seemed like a uniquely promising area to start a funding circle. This circle is roughly modeled on Farmed Animal Funders, and aims to connect funders to support projects in the space. If you would like to apply for funding or join the circle, you can connect via our website.

Why the area of mental health?

Mental health disorders are an important problem in the world. According to the Global Burden of Disease Study, mental disorders were the seventh leading cause of health burdens in the world in 2019; causing about 5% of all disability adjusted life years, and 14% of years of life lived with a disability.

According to the WHO, mental health is the most neglected health issue and should urgently receive more global investment. The Happier Lives Institute also argues that mental health disorders are particularly neglected, especially considering that they have some of the greatest impact on life satisfaction. Additionally, some initial estimates suggest a high cost-effectiveness and there seems to be shovel-ready charities with important gaps that we can fill.

What is a funding circle?

A funding circle is a collaboration between a number of funders who typically target a certain cause area. For example, Farmed Animal Funders (FAF) is a group of funders who are all keen to support the end of factory farming. Many of these funders have varying interests and they range fairly dramatically in size. FAF has a couple of staff who help the members of the network research and find opportunities, as well as generally coordinate between them. This reduces the risk of double-funding, or missing promising opportunities. Another example is Big Bang Philanthropy.

Funding circles can be a powerful way to leverage and coordinate multiple funders that share a common interest. Often a cause area will have a number of funders; but between closed application rounds, unclear standards, and disjointed networks, the funding is not organized as well as it could be. Having different funder views is really important for a good funding ecosystem, but a funding circle has the added, massive benefit of having a centralized point of contact. This means organizations can apply as if applying to a single large funder, but gain exposure to several potential funders. The structure also means that if one member of the network discovers a promising opportunity, but it falls outside of their scope, communication channels are open and ready for them to pass the project on to a grantmaker who is a better fit.

Testing the circle

If this funding circle works well, there may be similar groups set up in the future for other cause areas.

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Will faster economic growth make us happier? The relevance of the Easterlin Paradox to Progress Studies, published by MichaelPlant on June 24, 2022 on The Effective Altruism Forum.

This is a transcript of a talk I gave at the Moral Foundations of Progress Studies workshop at the University of Texas in March 2022. Or rather, it’s a re-recorded and edited version of the talk that was subsequently produced for a Global Priorities Institute reading group on ‘progress’ and then updated in light of many helpful comments from that seminar. The original slide deck can be viewed here.

1. Introduction

As I understand it, Progress Studies is a nascent intellectual field which starts by asking the question, “Since we seem to have gotten a lot of progress over the last couple of hundred years, where did this come from, and what can we do to get more of it?” (Vox, 2021).

Progress Studies has been popularised by academics such as Tyler Cowen and Steven Pinker. However, the Easterlin Paradox presents a real challenge to the claim that if we want more progress, we just need to improve the long-run growth rate - a view that Cowen argues for in his book Stubborn Attachments. This is a possible version of Progress Studies and the one I'm responding to.

So what is the Easterlin Paradox? Quoting Easterlin and O'Connor (2022), the Easterlin Paradox states:

At a point in time, happiness varies directly with income both among and within nations, but over time the long-term growth rates of happiness and income are not significantly related.

There is a common view that economic growth is going to make our lives better, but the Easterlin Paradox challenges this.

What's paradoxical is that at a given point in time, richer people are more satisfied than poorer people and richer countries are more satisfied than poorer countries, but over the course of time, countries which grow faster don't seem to get happier faster. In other words, if I get richer, that will be good for me, but if we all get richer, that won’t do anything for us collectively.

While subjective wellbeing (self-reported happiness and life satisfaction) has gone up in previous decades, the challenge of the Easterlin Paradox is that countries which grow faster do not seem to be getting happier faster; growth per se seems unrelated to average subjective wellbeing. If the paradox holds, the result would be striking and significant. It would suggest that, if we want to increase average wellbeing, we must not rely on growth, but go back to the drawing board and see what really works.

There's been quite a bit of debate over the nature and existence of the Paradox. The topic first emerged in 1974 when Richard Easterlin published a paper called, Does Economic Growth Improve the Human Lot? It's been particularly challenged by Stevenson and Wolfers (2008), who claim the paradox is an illusion and growth is making us happier. However, after looking into this myself, I actually think that Easterlin has the better half of the debate and the paradox does propose a real challenge to the idea that economic growth alone will make us happier.

My main purpose here is to explain what the Easterlin Paradox is and why - despite doubts - we need to take it seriously. My second purpose is to show that we can work out how to improve subjective wellbeing in society and make some tentative suggestions about this. However, this project is only starting to be taken seriously and there is lots more work to be done.

2. Evidence for the Paradox

So where does the Easterlin Paradox data come from? It's based on survey questions such as:

Taking all things together, how would you say things are these days?

Would you say you're very happy, pretty happy, or not too happy?[1]

All things considered, how satisfied are you with your life as a whole nowadays, from one, diss...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Half-baked AI Safety ideas thread, published by Aryeh Englander on June 23, 2022 on LessWrong.

[Cross-posted from the EA Forum. The EA Forum version of this post is for both half-baked EA ideas and half-baked AI Safety ideas, whereas this version of the post is for half-baked AI Safety ideas specifically.]

I keep having ideas related to AI safety, but I keep not having enough time available to really think through those ideas, let alone try to implement them. Practically, the alternatives for me are to either post something half-baked, or to not post at all. I don't want to spam the group with half-thought-through posts, but I also want to post these ideas, even in their current state, in case some of them do have merit and the post inspires someone to take up those ideas.

Originally I was going to start writing up some of these ideas in my Shortform, but I figured that if I have this dilemma then likely other people do as well. So to encourage others to at least post their half-baked ideas somewhere, I am putting up this post as a place where other people can post their own ideas without worrying about making sure they formulate those ideas to the point where they'd merit their own post.

If you have several ideas, please post them in separate comments so that people can consider each of them individually. Unless of course they're closely related to each other, in which case it might be best to post them together - use your best judgment.

[This post was also inspired by a suggestion from Zvi to create something similar to my AGI Safety FAQ / all-dumb-questions-allowed thread, but for ideas / potentially dumb solutions rather than questions.]

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.