Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: AI Training Should Allow Opt-Out, published by alyssavance on June 23, 2022 on LessWrong.

Last year, GitHub announced their Copilot system, an AI assistant for developers based on OpenAI's Codex model, as a free closed beta. Yesterday, they added that Copilot would now be available to everyone, but at a cost of $10 per month per user. Copilot is trained on all public GitHub repos, regardless of copyright, and various other data scraped from the Web (similar to Eleuther's Pile). Hence, GitHub is effectively using the work others made - for personal or non-commercial use, without having GitHub in mind, and without any way to say 'no' - to sell a product back to them, for their own profit. Many people are mad about this. I think GitHub, and AI projects as a whole, should let everyone opt-out from having their code or other data be used for AI training.

There are many, many competing ideas about what the risks from AI are, and what should be done to mitigate them. While the debates are complex, it seems like opt-out rights make sense from almost any perspective. Here are some arguments:

Argument from Simplicity

Mechanically, an opt-out would be very easy to implement in software. One could essentially just put a line saying:

(or the C++, Lua, etc. equivalent) into HuggingFace and other big AI frameworks. 'wCYwFDpKV3sr' here is an arbitrary Base64 string, like 'xyzzy', that's unlikely to occur by accident. Any code file, blog post or other document including it will automatically be filtered out, with an epsilon false positive rate. Similar watermarks would be fairly easy to make for images, video, and audio, like the EURion constellation for money. Google, Facebook, Microsoft, etc. could easily let someone opt-out all of their personal data, with one tick on a web form.

Argument from Competitiveness

An AI alignment "tax" is the idea that we expect AIs aligned with human needs to be slower or less capable than non-aligned AIs, since alignment adds complexity and takes time, just as it's easier to build bridges that fall down than reliable bridges. Depending on the particular idea, an alignment tax might vary from small to huge (an exponential or worse slowdown). Without strong global coordination around AI, a high alignment "tax" would be unworkable in practice, since someone else would build dangerous AI before you could build the safe one. This is especially true when it would be easy for one team to defect and disable a safety feature.

In this case, removing data makes the AI less capable, but there's definitely precedent that an opt-out tax would be low; in practice, people rarely bother to opt-out of things, even when there's a direct (but small) benefit. One obvious example is junk mail. No one likes junk mail, and in the US, it's easy to opt-out of getting junk mail and credit card offers, but most people don't. Likewise, there are tons of legally required privacy notices that give customers the chance to opt-out, but most don't. The same goes for arbitration opt-outs in contracts. Hence, a large majority of all data would probably still be available for AI use.

Argument from Ethics

It skeeves many people out that GitHub/Microsoft, or other companies, would take their work without permission, build a product on it, and then use it to make money off them, like academic publishers do. In the case of Google or Facebook, one might argue that, since the service is free, users have already agreed to "pay with their data" via AI analytics and targeted ads. (Although I think both services would be improved by a paid ad-free option, like YouTube Premium; and, it's questionable how much permission means with a quasi-monopoly.) However, GitHub isn't ad-supported, it's explicitly a freemium service that many teams pay for. And of course, people who write code or text for their...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Announcing the LessWrong Curated Podcast, published by Ben Pace on June 22, 2022 on LessWrong.

You can now listen to LessWrong Curated posts in podcast form on Spotify, Apple Podcasts, Audible, and Libsyn (which has an RSS feed, so it's available everywhere).

This is created and recorded by Solenoid Entity, who spent the last five years editing the SSC podcast, succeeded Jeremiah as narrator and publisher in 2020, and also makes the more recent Metaculus Journal Podcast. I reached out to him last week with an offer to do this work and he has quickly done some excellent recordings, which I'm very grateful for.

This is a new experiment and project, and so these 1-2 weeks are a great time to give me and Solenoid Entity feedback about what you like and dislike about the podcast, what would make it better for you, your experience as an author having your writing narrated, etc. You can leave comments here anytime, or talk to us via the intercom chat in the bottom right of the screen, or PM me personally via any channel.

Below are the 5 current available LessWrong Curated Podcasts.

Hat Tip to Tamera Lanham and Mattieu Putz for the suggestion at dinner!

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Air Conditioner Test Results & Discussion, published by johnswentworth on June 22, 2022 on LessWrong.

Background is in the preregistration post. This post will assume you’ve read that one.

First, the headline result: my main prediction was wrong. Paul’s predictions were also wrong. By the preregistered metric, the second hose improves performance by much less than either of us expected. That said, it looks like the experiment’s main metric mostly did not measure the thing it was intended to measure, which is why we were both so far off.

I did collect a bunch of data, which allows us to estimate the air conditioner’s performance in other ways. That analysis was not pre-registered and you should therefore be suspicious of it, but I nonetheless believe that it gives a more accurate view of the air conditioner’s performance. Main takeaway of that analysis is that the air conditioner performs about twice as well with two hoses as with one. Going by that analysis, both Paul's formula and my prediction were correct.

Experiment Setup

Here’s the air conditioner with the cardboard “second hose” attached:

I hung thermometers at each corner of the room, in the middle of each wall, and on the ceiling in the center of the room. I also placed one thermometer in the inlet (i.e. the hole in the window covering through which the cardboard hose draws air), the outlet (i.e. the hole in the window covering through which the other hose blows air), and outside on the balcony.

I ran a few different tests:

Air conditioner with and without the cardboard intake hose, on low fan

Air conditioner with and without the cardboard intake hose, on high fan

Air conditioner off (control)

For both the 1-hose and control tests, I left the inlet hole in the window covering open. (This was mainly to reduce infiltration from places other than outside.)

Assorted Notes:

I did try the experiment previously (about a month ago), and ran into issues which resulted in some minor changes to the experiment setup; info about that is here.

None of the experiment setup, thermometers, or the room were in direct sun.

The time for each test was mostly determined by when I had meetings scheduled, and when the temperatures seemed to stop changing.

Results

Data is here. Some numbers:

Outdoor temperature was 85-88°F (29.4 - 31.1°C) for most of the testing, though it dropped to 82°F (27.8°C) in the evening during the control test

Temperature difference between outdoor and indoor (higher is better), in each test:

Low fan: 20.6°F (11.4°C) with one hose, 22.7°F (12.6°C) with two hoses

High fan: 18.7°F (10.4°C) with one hose, 22.2°F (12.3°C) with two hoses

Control: 13.1°F (7.3°C)

Temperature variance across the room was fairly high (~2.5 - 3.0°F, or ~1.4 - 1.7°C), and consistent (i.e. measurements 30 minutes apart had similar relative temperature patterns)

Exhaust temperature was 98 - 100°F (36.7 - 37.8°C) with one hose, 112 - 119°F (44.4 - 48.3°C) with two

Analysis

My main prediction was that the outdoor-indoor temperature difference would be at least 50% greater with two hoses than with one hose, with my median estimate around 100% (i.e. a factor-of-two difference). That was definitely wrong: the actual number was 10% with fan on low, 19% with fan on high.

Paul’s prediction for the same number was 33-43%, though that was based on some very rough guesses about indoor, outdoor and exhaust temperatures. Using Paul’s formula with the actual indoor, outdoor and exhaust temperatures from the tests gives predictions anywhere from 75% to 180%, consistent with my own factor-of-two median guess. (The difference is mainly because the exhaust temperature was considerably lower than Paul had speculated. It really is a very shitty air conditioner.)

So experimentally, the difference came out way lower than anybody guessed. Why?

The result from ...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Mental support for EA partners?, published by Hedgehog on June 22, 2022 on The Effective Altruism Forum.

I am the significant other of someone who is heavily invested in EA-values and working full time at an EA-aligned organisation. I personally am interested in the work that the EA community is pursuing. And although a good amount of the values, considerations and motivations that are underpinning EA are overlapping with my personal values, as well as with motivations and questions that are fueling my professional field (which is not in EA), there are a few crucial EA-values that I personally don't subscribe to.

Increasingly I find myself in a position where I feel that I have to defend myself to my significant other for actions (from professional pursuits to small, family decisions) that are not necessarily the most tractable, quantifiable or having the most possible impact, according to EA-motivators.

These discussions and family-research-moments (ranging from 'what charity shall we donate to?' to 'what are your reasons for thinking that taking on a particular project - I am self-employed - will be the best use of your time?') are increasingly eating into my self-worth and feeding insecurities. Learning more about the EA movement via the forum, 80k website and podcasts is sometimes even causing substantial anxiety about my professional work as well as my personal motivations and values.

This made me think about the people near and dear to EA-people. How they are doing? How are they affected by it, if at all? And is there mental support for significant others that are supportive of, but not completely aligned to EA-values?

The only post in this context on the EA-forum I could find is 'It is ok to leave EA'. But I can't leave, as I am not in it personally. Any thoughts are appreciated!

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Results of a survey of international development professors on EA, published by ryancbriggs on June 22, 2022 on The Effective Altruism Forum.

I won an EA infrastructure fund grant a few months back that allowed me to survey all professors of international development in Canada, the UK, and the US on their views on effective altruism (among other things). I'm a professor of international development and I wanted to do this because my view is that this community basically agrees with EA precepts but is unaware of EA. I think this is detrimental to our research and (especially) our teaching. I also think that raising EA awareness among this group is potentially high impact, as professors influence a lot of people who can then go on to do great things in the world.

IDEV professors do not know about EA

My key question just asked the professors if "they knew enough about effective altruism to discuss it with a friend." The answers are below.

Familiar with EACanadaUnited StatesUnited KingdomYes125025No6012183

Among those who said "yes", most were unable to get within a factor of 10 of the most likely right answer to two questions testing EA knowledge. Q1 asked for the multiple by which the highest cost-effectiveness charity outperforms a median one (I went with 100x). Q2 asked the minimum cost to save the life of a child, and I assumed the GiveWell benchmark of about 5000 USD. I report both the median answers and the share within 10x of my best guess as to the right answer.

Familiar with EAQ1 median10x < Answer < 1000xQ2 median$500 < Answer < 50knYes4x0.2672.5 USD0.2287No3x0.1458.5 USD0.13264

Those who said that they were familiar with EA did better than those who were unfamiliar, but most people were far from the right answers and most answers mirrored the answers of the public at large (see also). This isn't surprising, as professors are quite specialized and most of us don't study these topics.

Most professors had not read EA-related books or heard of EA organizations.

My main takeaway here is just that professors don't know much about EA.

It should thus not be surprising that when I asked which topics a professor would devote at least 30 minutes of class time to in an introduction to international development course, the topics of EA and moral philosophy were the least likely to be covered, by a large margin.

Do IDEV professors already agree with (near-term) EA?

They basically do.

Almost 80% either strongly agree or agree that “From a moral perspective, people should care about the well-being of all human beings on the planet equally; they should not favour the well-being of people who are especially close to them either physically or emotionally” (Impartial beneficence).

The vast majority also agree that “when considering whether or not to devote time or money to a cause, one should think not merely about whether one will do good but also about how much good one will do.” (Magnitudes of good matter)

90% of professors gave to charity in the last year and 70% gave to a charity in a low or middle income country, though the most common reasons for giving were personal connections to the cause or specific charity.

Students and teaching

Finally, I asked the profs about students and teaching. The profs think that most students enrol in IDEV courses because they want to help the global poor. They think that students start out much too naive and optimistic about the prospects for global development, but that by the end of their program at least a substantial minority of students are overly pessimistic and jaded.

Conclusion

My main takeaway from all this is that IDEV profs agree with most (near-term) EA principles, but are unaware of EA. Even those that kind of have heard of EA don't actually know that much about it. These same professors often think that students start their pr...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Product Managers: the EA Forum Needs You, published by Ben West on June 22, 2022 on The Effective Altruism Forum.

This forum has undergone huge growth recently:

This growth is exciting, but also brings new problems. We are no longer in a world where even the most dedicated EA's can keep up with everything that's written, and the large influx of new people will stretch our ability to create and share common knowledge. EA seems poised to influence the next generation of thinkers and leaders, and this site is a major way we communicate with each other, from top journalists to CEOs of billion-dollar companies.

The EA Forum team is considering:

Introducing Reddit-style federated sub-forums, to better direct people to the content they find useful

Diving deep into key user journeys, to understand where people bounce and how we can prevent that. For instance, trying to better understand the experience of a newcomer who gets sent a post by their group organizer for the first time.

Improving recommendations and personalization such that newcomers don't bounce from confusion but old-timers don't bounce from boredom

This work requires both product management expertise and a deep understanding of the EA network. The intended role will be the 'product owner', prioritizing projects (like those listed above) and executing them.

The CEA Online team grew from three people last year, to 10 as of this writing, and is expected to be around 15 by the end of the year.

If you want to join a rapidly growing team of engineers, designers, and content creators, working on a product that users love, that has a big positive impact on the world, then we would love to hear from you.

If you know someone else who might fit this description, please send this post to them!

Apply Here

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Reflection Mechanisms as an Alignment target: A survey, published by Marius Hobbhahn on June 22, 2022 on The AI Alignment Forum.

This is a product of the 2022 AI Safety Camp. The project has been done by Marius Hobbhahn and Eric Landgrebe under the supervision of Beth Barnes. We would like to thank Jacy Reese Anthis and Tyna Eloundou for detailed feedback.

You can find the google doc for this post here. Posts to other sections of the text automatically link to the google doc. Feel free to add comments there.

Abstract

We surveyed ~1000 US-based Mechanical Turk workers (selected and quality tested by Positly) on their attitudes to moral questions, conditions under which they would change their moral beliefs, and approval towards different mechanisms for society to resolve moral disagreements.

Unsurprisingly, our sample disagreed strongly on questions such as whether abortion is immoral. In addition, a substantial fraction of people reported that these beliefs wouldn’t change even if they came to different beliefs about factors we view as morally relevant such as whether the fetus was conscious in the case of abortion.

However, people were generally favorable to the idea of society deciding policies by some means of reflection - such as democracy, a debate between well-intentioned experts, or thinking for a long time.

In a hypothetical idealized setting for reflection (a future society where people were more educated, informed, well-intentioned e.t.c.), people were favorable to using the results of the top reflection mechanisms to decide policy. This held even when respondents were asked to assume that the results came to the opposite conclusion as them on strongly-held moral beliefs such as views on abortion.

This suggests that ordinary Americans may be willing to defer to an idealized reflection mechanism, even when they have strong object-level moral disagreements. This indicates that people would likely support aligning AIs to the results of some reflection mechanism, rather than people’s current moral beliefs.

Introduction

Optimistically, a solution to the technical alignment problem will allow us to align an AI to “human values.” This naturally raises the question of what we mean by this phrase. For many object-level moral questions (e.g. “is abortion immoral?”), there is no consensus that we could call a “human value.” When lacking moral clarity we, as humans, resort to a variety of different procedures to resolve conflicts both with each other (democracy/voting, debate) and within ourselves (read books on the topic, talk with our family/religious community). In this way, although we may not be able to gain agreement at the object level, we may be able to come to a consensus by agreeing at the meta level (“whatever democracy decides will determine the policy when there are disagreements”); this is the distinction between normative ethics and meta-ethics in philosophy. We see the meta question of value choice as being relevant to strategic decisions around AI safety for a few reasons.

First, understanding people’s preferred conflict resolution mechanisms could be relevant for strategic decisions around AI governance and governance in general. Specifically, we want to avoid arms race conditions so that responsible labs have adequate time to ensure the safety of human-level AI. We feel that the ability to avoid race conditions could be strongly influenced by people’s broad ability to agree on metaethics. In particular, people should be less likely to fight for control of AI if they trust that AI builders will produce a system that reflects their (potentially-meta level) values. Understanding people’s expressed meta-ethics will therefore hopefully inform what types of governance structures are most likely to avoid race conditions.

Additionally, understanding people's...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: EA-break, EA-slow, published by Rémi T on June 22, 2022 on The Effective Altruism Forum.

See also: It's ok to leave EA

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: A Quick List of Some Problems in AI Alignment As A Field, published by NicholasKross on June 21, 2022 on LessWrong.

1. MIRI as central point of failure for... a few things...

For the past decade or more, if you read an article saying "AI safety is important", and you thought, "I need to donate or apply to work somewhere", MIRI was the default option. If you looked at FLI or FHI or similar groups, you'd say "they seem helpful, but they're not focused solely on AI safety/alignment, so I should go to MIRI for the best impact."

2. MIRI as central point of failure for learning and secrecy.

MIRI's secrecy (understandable) and their intelligent and creatively-thinking staff (good) have combined into a weird situation: for some research areas, nobody really knows what they've tried and failed/succeeded at, nor the details of how that came to be. Yudkowsky did link some corrigibility papers he labels as failed, but neither he nor MIRI have done similar (or more in-depth) autopsies of their approaches, to my knowledge.

As a result, nobody else can double-check that or learn from MIRI's mistakes. Sure, MIRI people write up their meta-mistakes, but that has limited usefulness, and people still (understandably) disbelieve their approaches anyway. This leads either to making the same meta-mistakes (bad), or to blindly trusting MIRI's approach/meta-approach (bad because...)

3. We need more uncorrelated ("diverse") approaches to alignment.

MIRI was the central point for anyone with any alignment approach, for a very long time. Recently-started alignment groups (Redwood, ARC, Anthropic, Ought, etc.) are different from MIRI, but their approaches are correlated with each other. They all relate to things like corrigibility, the current ML paradigm, IDA, and other approaches that e.g. Paul Christiano would be interested in.

I'm not saying these approaches are guaranteed to fail (or work). I am saying that surviving worlds would have, if not way more alignment groups, definitely way more uncorrelated approaches to alignment. This need not lead to extra risk as long as the approaches are theoretical in nature. Think early-1900s physics gedankenexperiments, and how diverse they may have been.

Or, if you want more hope and less hope at the same time, look at how many wildly incompatible theories have been proposed to explain quantum mechanics. A surviving world would have at least this much of a Cambrian explosion in theories, and would also be better at handling this than we are in real-life handling the actual list of quantum theories (in absence of better experimental evidence).

Simply put, if evidence is dangerous to collect, and every existing theoretical approach is deeply flawed along some axis, then let schools proliferate with little evidence, dammit! This isn't psych, where stuff fails to replicate and people keep doing it. AI alignment is somewhat better coordinated than other theoretical fields... we just overcorrected to putting all our eggs in a few approach baskets.

(Note: if MIRI is willing and able, it could continue being a/the central group for AI alignment, given the points in (1), but it would need to proliferate many schools of thought internally, as per (5) below.)

One problem with this [1], is that the AI alignment field as a whole may not have the resources (or the time) to pursue this hits-based strategy. In that case, AI alignment would appear to be bottlenecked on funding, rather than talent directly. That's... news to me. In either case, this requires either more fundraising, and/or more money-efficient ways to get similar effects to what I'm talking about. (If we're too talent-constrained to pursue a hits-based approach strategy, it's even more imperative to fix the talent constraints first, as per (4) below.)

Another problem is whether the "winning" approach mi...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Disruptive climate protests in the UK didn’t lead to a loss of public support for climate policies, published by James Ozden on June 20, 2022 on The Effective Altruism Forum.

Note: This is our more accessible report, with a more technical one intended for academic publication, with additional analysis, being released soon. Subscribe to our newsletter if you want to read the technical write-up at a later date. If you want to skim this analysis, we recommend you read the summary, key results and discussion.

Summary

Social Change Lab commissioned YouGov to conduct three nationally representative surveys of approximately 2,000 people each in the United Kingdom, focusing on their views on climate change, their willingness to engage in environmental activism, and their opinion of the climate activist group Just Stop Oil. The surveys were conducted before any significant protest activity occurred (on March 29th), during the Just Stop Oil and Extinction Rebellion protests (on the 9th of April), and after most (but not all) of the protest activity had occurred (on the 19th of April). Despite disruptive protests, there was no loss of support for climate policies, providing some evidence against the notion that disruptive protests tend to cause a negative public reaction. Furthermore, our findings suggest that the protests had a marginally statistically significant effect (p = 0.09) on respondents’ self-perceived likelihood of participating in environmental activism.

Our data suggests that the number of people saying that they were willing to take part in some form of climate activism increased from 8.7% to 11.3% of the UK population, equivalent to approximately 1.7 million additional people. However, due to some unforeseen circumstances, we’re unsure on exactly how to interpret the strength of these findings on willingness to take climate action (explained below).

Key Results:

Despite disruptive protests, there was no loss of support for climate policies. We believe this provides some evidence against the notion that disruptive protests tend to cause a negative public reaction towards the goals of the disruptive movement.

There was a marginally statistically significant change (p = 0.09) in the self-reported likelihood of participants taking part in environmental activism from before the protests to after the protests. Whilst this is slightly outside standard scientific margins of error, we think it’s still worth reporting weak evidence which readers can interpret for themselves (see more here).

Our effect size for likelihood of taking climate action was smaller than we had anticipated, which we believe led to our result being marginally significant (p < 0.10) rather than clearly statistically significant (p < 0.05).

The main reasons we believe our effect size was smaller than we anticipated was due to crowding out of media appearances by the Russian invasion of Ukraine, as well as diminishing marginal returns to protest in certain contexts. We explore these factors in greater detail in our Discussion.

Respondents were also asked questions about their concern about the climate crisis, their awareness of the impact of fossil fuels on climate change, their support for the goals of Just Stop Oil, their discontent with government environmental policy, and their support for more fossil fuel extraction. There were no statistically significant changes in any of these variables.

Most people claimed to be aware of Just Stop Oil and aware that the protests were taking place, with 63% of respondents saying that they were aware of the Just Stop Oil protests after most of the protests had occurred. Most respondents did not support the actions of Just Stop Oil, with only 18% of respondents saying that they supported the protests.

However, due to existing high levels of climate concern in th...