Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Beware safety-washing, published by Lizka on January 13, 2023 on The Effective Altruism Forum.

Tl;dr: don’t be fooled into thinking that some groups working on AI are taking “safety” concerns seriously (enough).

Outline

Two non-AI examples

Greenwashing

Humanewashing

Definition of safety-washing

What are the harms?

What can (and should) we do about this?

Note: I’m posting this in my personal capacity. All views expressed here are my own. I am also not (at all) an expert on the topic.

Two non-AI examples

Greenwashing

Companies “greenwash” when they mislead people into incorrectly thinking that their products or practices are climate and environment-friendly (or that the company focuses on climate-friendly work).

Investopedia explains:

Greenwashing is an attempt to capitalize on the growing demand for environmentally sound products.

The term originated in the 1960s, when the hotel industry devised one of the most blatant examples of greenwashing. They placed notices in hotel rooms asking guests to reuse their towels to save the environment. The hotels enjoyed the benefit of lower laundry costs.

Wikipedia: “[Jay Westerveld, the originator of the term] concluded that often the real objective was increased profit, and labeled this and other profitable-but-ineffective ‘environmentally-conscientious’ acts as greenwashing.” (Wikipedia also provides a long list of examples of the practice.)

I enjoy some of the parody/art (responding to things like this) that comes out of noticing the hypocrisy of the practice.

Humanewashing

A similar phenomenon is the “humanewashing” of animal products. There’s a Vox article that explains this phenomenon (as it happens in the US):

A carton of “all natural” eggs might bear an illustration of a rustic farm; packages of chicken meat are touted as “humanely raised."

In a few cases, these sunny depictions are accurate. But far too often they mask the industrial conditions under which these animals were raised and slaughtered.

Animal welfare and consumer protection advocates have a name for such misleading labeling: “humanewashing.” And research suggests it’s having precisely the effect that meat producers intend it to. A recent national survey by C.O.nxt, a food marketing firm, found that animal welfare and “natural” claims on meat, dairy, and egg packaging increased the intent to purchase for over half of consumers.

...rather than engaging in the costly endeavor of actually changing their farming practices, far too many major meat producers are attempting to assuage consumer concerns by merely changing their packaging and advertising with claims of sustainable farms and humane treatment. These efforts mislead consumers, and undermine the small sliver of farmers who have put in the hard work to actually improve animal treatment.

If you want a resource on what food labels actually mean, here are some: one, two, three (these are most useful in the US). (If you know of a better one, please let me know. I’d especially love a resource that lists the estimated relative value of things like “free-range” vs. “cage-free,” etc., according to cited and reasonable sources.)

Definition of safety-washing

In brief, “safety-washing” is misleading people into thinking that some products or practices are “safe” or that safety is a big priority for a given company, when this is not the case.

An increasing number of people believe that developing powerful AI systems is very dangerous, so companies might want to show that they are being “safe” in their work on AI.

Being safe with AI is hard and potentially costly, so if you’re a company working on AI capabilities, you might want to overstate the extent to which you focus on “safety.”

So you might:

Pick a safety paradigm that is convenient for you, and focus on that

Talk about “safety” when you really mean other kinds o...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Disentangling Shard Theory into Atomic Claims, published by Leon Lang on January 13, 2023 on The AI Alignment Forum.

Introduction

Produced as part of the SERI ML Alignment Theory Scholars Program - Winter 2022 Cohort. Thanks to Magdalena Wache for giving feedback on a recent version, and to Alex Turner for giving feedback on an early version of this article.

When thinking about shard theory, I noticed that my brain wanted to answer questions such as "What does shard theory predict?" or "Do I agree with shard theory?". Additionally, I observed other people engage in similar thinking. I now think this is confused, and that it is better to view shard theory as a bag of related claims that should be reasoned about and evaluated separately.

Therefore, In this post I want to explain the following:

What do I currently consider the main claims of shard theory?

What is my own stance on these claims?

This distillation is in spirit very similar to LawrenceC’s Shard Theory in Nine Theses. Note, however, that I did not read that distillation in order to produce a more independent explanation of shard theory.

My one-sentence categorization is that shard theory is both a theory for human value formation and also a paradigm/frame for thinking about alignment. It might also become a theory of value formation in RL agents, but it’s not quite there yet since it doesn’t make enough concrete and formalized empirical predictions yet.

Instead of splitting this distillation into “claims about humans and claims about trained RL agents” I decided to make a different split. Namely, I will start with a section on claims in the shard theory meme-space which do not involve shards at all, and later on claims that actually make use of the concept of a shard. I hope that this separation makes it easier for people to agree or disagree with very specific claims instead of accepting or rejecting the theory as a whole.

Please let me know if there are other important claims that I forgot, and any other relevant feedback on this post.

After each claim, I will indicate my level of agreement as follows:

✓ : Agree

(✓) : tentative agree

? : Neither agree nor disagree

Shard Theory claims without any shards

The following are claims from shard theory that can mostly be formulated without even talking about shards. Shards are often in the background of much of the thinking, but they don't need to be mentioned explicitly.

Humans get their values from within-lifetime learning

The Claim

There is neuroscientific evidence showing that humans get most of their complex values from within-lifetime learning. In other words, human values are "learned from scratch" and not "hardcoded". One core argument is that we value many things that didn’t even exist in our evolutionary environment, like photographs of our past or specific regional traditions. Other arguments are made in Human values & biases are inaccessible to the genome.

There are hardcoded reward circuits in human brains, mostly coming from the brain stem, that provide reinforcement signals that the brain uses to develop its values, but the resulting values do not coincide with this reward.

The claim is used as supporting evidence in humans provide an untapped wealth of evidence about alignment and the shard theory of human values.

My Opinion: (✓)

I tentatively agree with this claim after reading much of the above-mentioned supporting evidence. The reason that my agreement is only tentative is that I'm not even remotely a neuroscientist. More concretely, while it seems reasonable that our actual values emerge in our lifetimes, it seems conceivable to me that the evolutionarily formed inductive biases of our brain contain many "hacks" that are hard to reproduce in machine learning systems.

To achieve alignment of ML systems, we should learn more about how humans get ...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: AGISF adaptation for in-person groups, published by Sam Marks on January 13, 2023 on The AI Alignment Forum.

This past semester, HAIST and MAIA (the Harvard and MIT AI safety student groups) ran an adapted version of Richard Ngo's AGI Safety Fundamentals alignment curriculum. This adaptation – which consists of eight 2-hour long meetings, with all readings done during the meeting – is now available on the AGISF website.

In this post, we discuss the adapted curriculum and its intended use, and we recommend that other in-person reading groups following AGISF use this adaptation.

The adapted curriculum and its intended use

The adapted curriculum was made by refining a slightly rustier first adaptation, with significant help from Richard Ngo and feedback from participants. The key differences between the adapted curriculum and the mainline AGISF alignment curriculum are:

Participants do all the core readings during the meeting; no reading is required in between meetings.

Participants meet for 2 hours per week instead of 1.5.

Readings, including further readings, tend to be more bite-sized (usually not longer than 20 minutes).

There are no projects, and certain topics are omitted (e.g. governance and inverse reinforcement learning).

The way that HAIST and MAIA used this curriculum, and the way we recommend other groups use it, is:

Alternate between silent reading and discussion.

So a typical meeting might look like: people arrive, everyone does reading 1, everyone discusses reading 1, everyone does reading 2, everyone discusses reading 2, etc.

With certain longer or more difficult readings (e.g. Toy models of superposition), it could be reasonable to occasionally pause for discussion in the middle of the reading.

Encourage faster readers to take a look at the further readings while they wait for others to catch up.

We found that reading speeds varied significantly, with slower readers taking ~1.5x as long to finish as faster readers.

This works especially well if the readings are printed (which we recommend doing).

We note that this format introduces some new challenges, especially when there are slower readers.

Facilitators need to manage discussion timing since discussions that go too long cut into time for reading and discussing other material.

Planning out how long to spend discussing each core reading ahead of time can be very useful.

Facilitators should feel comfortable cutting off discussions to make sure there’s time to read and discuss all the core readings. (On the other hand, if a discussion is very productive, it may be worth skipping certain readings; this is a judgment call that facilitators will need to make.)

Different reading speeds need to be managed.

At HAIST, we typically found it feasible to wait for the slowest reader to finish reading. We printed copies of the further readings for faster readers to peruse while they waited for others to finish.

On the other hand, this might not work well for groups with especially slow readers. In these cases, you may need to begin discussions before everyone is done reading and, going forward, encourage slower readers to take a look at the core readings ahead of future meetings.

To help with some of these challenges, Sam prepared a guide for HAIST and MAIA facilitators that included recommended discussion times, points of discussion, and advice about which readings to cut if necessary. That facilitator guide was for an outdated version of the curriculum, but we hope to have an updated facilitator guide in the next few weeks. We don’t want to make these public, but feel free to reach out to [email protected] if you’re running a reading group and are interested in seeing the old or forthcoming facilitator guides.

Why we recommend the adapted curriculum

Sam and Xander generally felt that the in-sessions reading ...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: ea.domains - Domains Free to a Good Home, published by plex on January 12, 2023 on The Effective Altruism Forum.

Tl;dr: I’ve set up a database of domains at ea.domains which are free to a good home, to prevent them being squatted and blocked for use. You can add domains you control to it using this form.

Since my well received post on setting up an Anti-squatted AI x-risk domains index, I’ve been picking up more and more domains, talking to other domain holders, and building an interface for viewing them. I’ve also put a few of them to good use already!

Also a big thanks to Ben West for sharing 215 of CEA's parked domains, which EA projects are welcome to request use of by emailing [email protected]. He also offered for CEA to be the custodian of the domains I bought, with the condition that they will be pointed to nameservers I control and that they will return ownership of them for EA-aligned use on request, unless they’re in active use by another EA project. This will save me from having to pay upkeep, which will help make this more sustainable. He's open to extending this offer to holders of other relevant domains.

If you'd like to use one of these domains, then message the contact person specified. They each have different policies for handing over the domain, but a good standard is that they'll point the domain towards your servers on request, and hand it over for free if and when you have built something useful at that location.

Here’s the top 40 domains I’m most excited about, but go check the full list:

DomainPossible useContactaisafety.globalAI Safety [email protected] quality explanation?tech@centreforeffectivealtruism.orgontological.techNew [email protected] [email protected] [email protected] of [email protected] [email protected] [email protected] [email protected] [email protected] [email protected] entrepreneurs or impact investing [email protected] [email protected] Safety PhD [email protected] said they were happy for others to join the careers advice [email protected] specific funding [email protected] said they were happy for others to join the careers advice

[email protected] funding [email protected] to do a coordinated push for cause [email protected] to do a coordinated push for cause [email protected] to do a coordinated push for cause [email protected] to do a coordinated push for cause [email protected] to do a coordinated push for cause [email protected] to do a coordinated push for cause areas? [email protected] to do a coordinated push for cause [email protected] Euler for [email protected] org which specializes in coaching AI safety [email protected] [email protected] on news on the AI safety [email protected] [email protected]

[email protected] [email protected] study groups for skilling up [email protected] tech@centreforeffectivealtruism.orgbountiedrationality.orgWebsite to pair with the BR facebook [email protected]...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Victoria Krakovna on AGI Ruin, The Sharp Left Turn and Paradigms of AI Alignment, published by Michaël Trazzi on January 12, 2023 on LessWrong.

Victoria Krakovna is a Research Scientist at DeepMind working on AGI safety and a co-founder of the Future of Life Institute, a non-profit organization working to mitigate technological risks to humanity and increase the chances of a positive future.

In this interview we discuss three of her recent LW posts, namely DeepMind Alignment Team Opinions On AGI Ruin Arguments, Refining The Sharp Left Turn Threat Model and Paradigms of AI Alignment.

This conversation presents Victoria's personal views and does not represent the views of DeepMind as a whole.

Below are some highlighted quotes from our conversation (available on Youtube, Spotify, Google Podcast, Apple Podcast). For the full context for each of these quotes, you can find the accompanying transcript.

The intelligence threshold for planning to take over the world isn't low

Michaël: “Do you mostly agree that the AI will have the kind of plans to disempower humanity in its training data, or does that require generalization?”

Victoria: “I don't think that the internet has a lot of particularly effective plans to disempower humanity. I think it's not that easy to come up with a plan that actually works. I think coming up with a plan that gets past the defenses of human society requires thinking differently from humans. I would expect there would need to be generalization from the kind of things people come up with when they're thinking about how an AI might take over the world and something that would actually work. Maybe one analogy here is how, for example, AlphaGo had to generalize in order to come up with Move 37, which no humans have thought of before. [...]

The same capabilities that give us probably creative and interesting solutions to problems that, like Move 37, could also produce really undesirable creative solutions to problems that we wouldn't want the AI to solve. I think that's one argument that I think is also on the AGI Ruin list that I would largely agree with, that it's hard to turn off the ability to come up with undesirable creative solutions without also turning off the ability to generally solve problems that we one day want AI to solve. For example, if we want the AI to be able to, for example, cure cancer or solve various coordination problems among humans and so on, then a lot of the capabilities that would come with that could also lead to bad outcomes if the system is not aligned.” (full context)

Why Refine The Sharp Left Turn Threat Model

(On the motivations for writing Refining The Sharp Left Turn Threat Model, a Lesswrong post distilling the claims in the sharp left turn thread model as described in Nate Soares’ post)

“Part of the reason that I wrote a kind of distillation of the threat model or a summary how we understand it is that I think the original threat model seems a bit vague or it wasn’t very clear exactly what claims it’s making. It sounds kind of concerning, but we weren’t sure how to interpret it. And then when we were talking about it with within the team, then people seem to be interpreting it differently. It just seemed useful to kind of arrive at a more precise consensus view of what this threat model actually is and what implications does it have. Because if we decide that the sharp left turn is sufficiently likely, that we would want our research to be more directed towards overcoming and dealing with the sharp left turn scenario. That implies maybe different things to focus on. It’s one thing that I was wondering about. To what extent do we agree that this is one of the most important problems to solve and what the implications actually are in particular. [...]

The first claim is that you get this rapid phase transition in cap...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: I am tired., published by MeganNelson on January 12, 2023 on The Effective Altruism Forum.

Hello! It’s me, a small-scale part-time EA community builder. I read The Life You Can Save in 2009 and figured that in addition to being a vegan and a social worker, I should donate 10%-plus of my income to highly effective causes. Then I connected with my local effective altruism community in 2016 and figured that I should also spend a not-insignificant portion of my waking hours encouraging and connecting other people who want to make the world a better place.

I am cheerful. I work hard. I volunteer at EAGs. I show up for the people around me.

Why? Because I think it’s the right thing to do.

But folks, I am TIRED.

I am tired of having a few people put on pedestals because they are very smart - or very good at self-promotion. I am tired of listening to arguments about who can have the think-iest thoughts. I am tired of drama, scandals, and PR. I am tired of being in a position where I have to apologize for sexism, racism, and other toxic ideologies within this movement. I am tired of convening calls with other community builders where we try to figure out how to best react to the latest Thing That Happened. I am tired of billionaires. And I am really, really tired of seeing people publicly defend bad behavior as good epistemics.

I’m just here because I want the world to be a better, kinder, softer place. I know I’m not the only one. I’m not quitting. But I am tired.

Maybe you are tired, too.

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: I Support Bostrom, published by 𝕮𝖎𝖓𝖊𝖗𝖆 on January 12, 2023 on The Effective Altruism Forum.

Epistemic Status

Expressing this opinion because I get the sense the current zeitgeist on the forum underweights it, so staking it out feels somewhat valuable.

Personal Context

For context, I'm black (Nigerian who migrated to the UK last year as a student), currently upskilling to work in AI safety and joined EA via osmosis from LessWrong/the rationalist community.

I've been a rationalist since 2017, and EA-adjacent since 2019-ish? I began overtly identifying as an EA last year.

I'm concerned about the longterm flourishing of humanity, and I want to do what I can to help create a radically brighter future.

I'm just going to express my honest opinions here:

The events of the last 48 hours (slightly[1]) raised my opinion of Nick Bostrom. I was very relieved that Bostrom did not compromise his epistemic integrity by expressing more socially palatable views that are contrary to those he actually holds.

I think it would be quite tragic to compromise honestly/accurately reporting our beliefs when the situation calls for it to fit in better. I'm very glad Bostrom did not do that.

Beyond just general epistemic integrity that I think we should uphold, to the extent that one thinks that Bostrom is an especially important thinker re: humanity's longterm flourishing, then it's even more important that he strongly adheres to epistemic integrity.

I think accurately reporting our beliefs and being honest even when society would reproach us for it is especially valuable for people thinking about "grand strategy for humanity".

I think it would be very tragic if Bostrom were to face professional censure because of this. I don't think an environment that punishes epistemic integrity is particularly productive with respect to working on humanity's most pressing problems.

As for the contents of the email itself, while very untasteful, they were sent in a particular context to be deliberately offensive and Bostrom did regret it and apologise for it at the time. I don't think it's useful/valuable to judge him on the basis of an email he sent a few decades ago as a student. The Bostrom that sent the email did not reflectively endorse its contents, and current Bostrom does not either.

I'm not interested in a discussion on race & IQ, so I deliberately avoided addressing that.

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Announcing the 2023 PIBBSS Summer Research Fellowship, published by Nora Ammann on January 12, 2023 on The AI Alignment Forum.

What can we learn from biological and social systems about designing beneficial AI systems?

We are, once again, pleased to announce the PIBBSS Research Fellowship program.

Principles of Intelligent Behavior in Biological and Social Systems (PIBBSS) aims to facilitate research on parallels between intelligent behaviour in natural and artificial systems and to leverage these insights towards the goal of building safe and aligned AI.

Last year, we welcomed 20 fellows from fields as diverse as computational neuroscience, evolutionary biology, complex systems studies and the humanities. Over the course of three months, supported by two research retreats in Oxford and Prague, shared office space for the cohort, and mentorship from experienced AI alignment researchers, fellows worked on a range of projects with the purpose of bringing interdisciplinary expertise to AI safety. We were overall pleased with the results and thus decided to organize a second iteration of the program.

If you’re interested in the program, learn more and apply here.

Application deadline: Feb 5th, 2023, 23:59 CET

We are running to information sessions if you want to learn more or want to ask us questions:

1st information session: 28th of January, 17:00 UTC (09:00 PST, 12:00 EST, 18:00 CET, 01:00 [29th of Jan] Singapore) - Zoom Link

2nd information session: 29th of January, 11:00 UTC (03:00 PST, 06:00 EST, 12:00 CET, 19:00 Singapore) - Zoom Link

We are grateful for any help to pass on the word to people who might be interested in applying. At this link you can find all the information about the program in several forms (Twitter length, short description, medium description), so you can copy-paste the version from there you find most useful.

For more information, see:

Our website: PIBBSS.ai

A general introduction to PIBBSS, see: Introducing the Principles of Intelligent Behaviour in Biological and Social Systems (PIBBSS) Fellowship

A discusison of the epistemic bet PIBBSS is aiming to foster, see: AI alignment as “navigating the space of intelligent behaviour” (see section 3 in particular)

A brief retrospective on last year’s fellowship program

... or follow us on Twitter:

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: A personal response to Nick Bostrom's "Apology for an Old Email", published by Habiba on January 12, 2023 on The Effective Altruism Forum.

On 9th January 2023, Nick Bostrom posted this apology for an email he sent on the Extropians listserv in the 90s. On 11th January 2023, Anders Sandberg linked to it on Bostrom's behalf in this twitter thread.

I recommend you read those first as I don't summarise or explain the contents below.

This is my personal response to reading Bostrom's apology and email and is (bar some minor changes) a cross-post of my tweet thread.

As a meta-point I would like to flag that I do find discussion of the topic to be incredibly stressful. I have almost never posted here on the forum about even straightforward things. And debating race and IQ is something I find exceptionally emotionally tough. So I don't plan to participate in any extensive debates in the comments, hope that you understand why.

My thoughts

In my view, Bostrom's email would have been offensive in the 90s and it is offensive now, for good reason. His apology fails badly to fully take responsibility or display an understanding of the harm the views expressed represent.

I think that being deliberately offensive to make a point is gross. When people in positions of privilege use or mention slurs lightly they are able to do so because they are blinkered to the lived experience of others and disengaged from empathy with those different to them.

Note that I’m not generally in the business of picking people apart for small one-off past infractions. But I do think it would be virtuous to apologise for and to truly take responsibility for one’s past actions.

Bostrom’s apology is defensively couched - emphasising the age of the email, what others wrote on the listserv, that it would be best forgotten, that fear that people might smear him. I think that is cowardly and shows a disappointing lack of ownership of his actions.

But I don’t just care about the inclusion of a slur in the email. I am deeply uncomfortable with a discussion of race and intelligence failing to acknowledge the historical context of the ideas’ origin and the harm they can and have caused.

To be clear, I think the view Bostrom expressed was wrong, and wrong in a harmful and reckless way.

When you argue a point like this without addressing the context of how those ideas came about you will likely be missing something important we should learn from history and be badly wrong.

When you are willfully disengaged from the empathy that underlies common decency you will have a massive blindspot in your reasoning and you will likely be badly wrong.

I do not think that there is only one acceptable way to express thoughts about this issue, nor do I think this issue could never be discussed sensitively. And I do think it is okay for people to sometimes say things online that I think are plain wrong.

But we all know that this issue is high stakes - ideas about racial superiority in the UK, America, and Germany led to some of the worst atrocities of the 20th century. And eugenists historically espoused utterly wrong views on race and intelligence wearing the guise of science.

There were c.60,000 sterilisations in US eugenics programmes - focused on women of colour. Including those who were young, poor, victims of sexual abuse, labelled “feeble minded” (allegedly inherited via a recessive gene), and had their fates decided by committees of white men.

Hitler was a fan of eugenicist Madison Grant. And at the Nuremberg trials, the Nazi defendants entered Grant’s book - The Passing of the Great Race - in their defence tracing the lineage of their genocidal ideas to a popular American author.

And we know now there are very good reasons to think that scores on IQ tests are affected by cultural factors, that global IQ databases are poor sour...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: CEA statement on Nick Bostrom's email, published by Shakeel Hashim on January 12, 2023 on The Effective Altruism Forum.

Effective altruism is based on the core belief that all people count equally. We unequivocally condemn Nick Bostrom’s recklessly flawed and reprehensible words. We reject this unacceptable racist language, and the callous discussion of ideas that can and have harmed Black people. It is fundamentally inconsistent with our mission of building an inclusive and welcoming community.

The Centre for Effective Altruism

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.