Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: We must be very clear: fraud in the service of effective altruism is unacceptable, published by evhub on November 10, 2022 on The Effective Altruism Forum.

I care deeply about the future of humanity—more so than I care about anything else in the world. And I believe that Sam and others at FTX shared that care for the world.

Nevertheless, if some hypothetical person had come to me several years ago and asked “Is it worth it to engage in fraud to send billions of dollars to effective causes?”, I would have said unequivocally no.

At this stage, it is quite unclear just from public information exactly what happened to FTX, and I don't want to accuse anyone of anything that they didn't do. However, I think it is starting to look increasingly likely that, even if FTX's handling of its customer's money was not technically legally fraudulent, it seems likely to have been fraudulent in spirit.

And regardless of whether FTX's business was in fact fraudulent, it is clear that many people—customers and employees—have been deeply hurt by FTX's collapse. People's life savings and careers were very rapidly wiped out. I think that compassion and support for those people is very important. In addition, I think there's another thing that we as a community have an obligation to do right now as well.

Assuming FTX's business was in fact fraudulent, I think that we—as people who unknowingly benefitted from it and whose work for the world was potentially used to whitewash it—have an obligation to condemn it in no uncertain terms. This is especially true for public figures who supported or were associated with FTX or its endeavors.

I don't want a witch hunt, I don't think anyone should start pulling out pitchforks, and so I think we should avoid a focus on any individual people here. We likely won't know for a long time exactly who was responsible for what, nor do I think it really matters—what's done is done, and what's important now is making very clear where EA stands with regards to fraudulent activity, not throwing any individual people under the bus.

Right now, I think the best course of action is for us—and I mean all of us, anyone who has any sort of a public platform—to make clear that we don't support fraud done in the service of effective altruism. Regardless of what FTX did or did not do, I think that is a statement that should be clearly and unambiguously defensible and that we should be happy to stand by regardless of what comes out. And I think it is an important statement for us to make: outside observers will be looking to see what EA has to say about all of this, and I think we need to be very clear that fraud is not something that we ever support.

In that spirit, I think it's worth us carefully confronting the moral question here: is fraud in the service of raising money for effective causes wrong? I think it is, quite clearly—and that's as someone who is about as hardcore of a total utilitarian as it is possible to be.

When we, as humans, consider whether or not it makes sense to break the rules for our own benefit, we are running on corrupted hardware: we are very good at justifying to ourselves that seizing money and power for own benefit is really for the good of everyone. If I found myself in a situation where it seemed to me like seizing power for myself was net good, I would worry that in fact I was fooling myself—and even if I was pretty sure I wasn't fooling myself, I would still worry that I was falling prey to the unilateralist's curse if it wasn't very clearly a good idea to others as well.

Additionally, if you're familiar with decision theory, you'll know that credibly pre-committing to follow certain principles—such as never engaging in fraud—is extremely advantageous, as it makes clear to other agents that you are a trustworthy actor who can be relied upon....

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Some comments on recent FTX-related events, published by Holden Karnofsky on November 10, 2022 on The Effective Altruism Forum.

It appears that FTX, whose principals support the FTX Foundation, is in serious trouble. We’ve been getting a lot of questions related to these events.

I’ve made an attempt to get some basic points out quickly that might be helpful to people, but the situation appears to be developing quickly and I have little understanding of what’s going on, so this post will necessarily be incomplete and nonauthoritative.

One thing I’d like to say up front (more on how this relates to FTX below) is that Open Philanthropy remains committed to our longtermist focus areas and still expects to spend billions of dollars on them over the coming decades. We will raise the bar for our giving, and we don’t know how many existing projects that will affect, but we still expect longtermist projects to grow in terms of their impact and output.

Are the funds directed by Open Philanthropy invested in or otherwise exposed to FTX or related entities?

No.

The FTX Foundation has quickly become a major funder of many longtermist and effective altruist organizations. If it stops (or greatly reduces) funding them, how might that affect Open Philanthropy’s funding practices?

If the FTX Foundation stops (or greatly reduces) funding such people and organizations, then Open Philanthropy will have to consider a substantially larger set of funding opportunities than we were considering before.

In this case, we will have to raise our bar for longtermist grantmaking: with more funding opportunities that we’re choosing between, we’ll have to fund a lower percentage of them. This means grants that we would’ve made before might no longer be made, and/or we might want to provide smaller amounts of money to projects we previously would have supported more generously.

Does Open Philanthropy also need to raise its bar in light of general market movements (particularly the fall in META stock) and other factors?

Yes:

Our available capital has fallen over the last year for these reasons. That said, as of now, public reports of Dustin Moskovitz and Cari Tuna’s net worth give a substantially understated picture of our available resources. That’s because, among other issues, they don’t include resources that are already in foundations. (I also note that META stock is not as large a part of their portfolio as some seem to assume.) Dustin and Cari still expect to spend nearly all of their resources in their lifetimes on philanthropy that aims to accomplish as much good per dollar as possible.

Additionally, the longtermist community has been growing; our rate of spending has been going up; and we expect both of these trends to continue. This further contributes to the need to raise our bar.

As stated above, we remain committed to our focus areas and still expect to spend billions of dollars on them over the coming decades.

So how much might Open Philanthropy raise its bar for longtermist grantmaking, and what does this mean for today’s potential grantees?

We don’t know yet — the news about FTX was sudden, and we’re working to figure things out.

It’s a priority for us to think through how much to raise the bar for longtermist grantmaking, and therefore what kinds of giving opportunities to fund. We hope to gain some clarity on this in the next month or so, but right now we’re dealing with major new information and don’t have a lot to say about what it means. It could mean reducing support for a lot of projects, or for relatively few.

(We don’t have a crisp formalization of “the bar”; instead we have general guidance to grantmakers on what sorts of requests should be generously funded vs. carefully considered vs. rejected. We need to rethink and revise this guidance.)

Because of this, we are pausing mo...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: What it's like to dissect a cadaver, published by OldManNick on November 10, 2022 on LessWrong.

Why

I never thought I was a bio person. But then I overheard Viv talking about MAOIs at a party. I asked her:

> - What are MAOIs? > - monoamine oxidase inhibitor

> - What does that mean? > - It prevents reuptake of neurotransmitters. > - But what is a neurotransmitter? What does reuptake actually mean? > - ... > - So life uses chiral properties of space to implement things...

Viv had the most important trait of a teacher: patience. I asked the most naive questions and they answered them. They walked with me, all the way down to the very beginning, rebuilding my understanding. It was amazing. I wanted to know

more. Roadblock: finding lifeforms to study.

I wondered if non-medical students could watch dissections. You can’t get more information about an object than by directly interacting with it. The concrete world contains the abstract one. I even asked my doctor at a physical if she knew of any, and she said to look at community colleges.

After some searching, I found this: Bio

848NV. Forget viewing the dissection, you’re doing the dissection. 5 hour dissection for $60, free if you just watch. The only bureaucratic hangup is that you must pay by check.

This is why I love the Bay Area: there’s stuff like this and you can just do it. yes it’s weird no they can’t stop you. The boundary between scientist and serial killer is paper thin sometimes.

Takeaways

I’ve done this a few times now. Turns out that there’s way way way too much information to understand it all in one 5 hour session. Each time, we pick out areas and focus on them.

Seeing how everything fits together ‒and how big it is‒ makes understanding at different scales much easier.

There’s a common template to life. Seeing it in you hits different.

Brain has interesting connections to fractals and graph theory.

Maybe pan-psychism isn’t totally wrong.

What & how & why

I tell my friend Leah and she says “This is the most appealing activity that

I’ve ever seen you do”. Dunno whom that says more about.

We arrive and there are 5 people around 3 cadavers. We get aprons and lab coats and start syringing what’s mostly Downy fabric softener with a syringe. It prevents decay and smells sickly sweet.

Corpses can last a long time. One of the corpses had been dead for 5 years.

Many random observations

There’s a crazy amount of connective tissue, and it makes a creepy wireframe surrounding your skeleton. Even the space between the folds of the brain has it.

If you exercise, we’ll know. Their insides just look different. “It’s who you are inside that matters” is a much creepier sentence now.

Veins, arteries, and nerves all travel together, wound around each other by a bunch of connective tissue. Mnemonic: VAN.

Cancer can turn your guts and lungs green, and it’s this horrible bright moldy green. Metastasized tissue is hard but ultimately crumbly like overcooked chicken liver. The stomach and intestines have textures reminiscent of damp cardboard, but they’re dry to the touch.

I finally saw a lymph node. The body has a lot of drainage into the lymphatic system.

There’s a bunch of tiny nerves and you can’t feasibly preserve all.

The etymology of the word patience is “capacity for suffering”. This is apt. You can’t rush the process, and believe me, it is a process. Exposing the VANs requires reflecting away the skin, and this takes a long time and much more physical effort than you’d think. You’re basically scraping it off, and the best tool overall is your hands. Skin in particular is much tougher than it looks, and I ended up locking a pair of forceps against a shoulder and just leaning back to pull it taut.

Speaking of the shoulder, I spent 2 hours working up through one. There are a lot of fiddly bits. I knew, but didn’t understand, ...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Community support given FTX situation, published by Julia Wise on November 10, 2022 on The Effective Altruism Forum.

A lot of us are looking at the news about FTX’s financial troubles and wondering what it will mean.

For some people, that will be because they had investments held within FTX. For others, it will be because they were working on projects funded by the FTX Foundation. And for many, it will be because of the broader effects of this change on the world.

I know people want answers, and I wish CEA had information to share.

If you’re personally affected by what’s happening, the community health team wants to be here for you. We’ve already heard from people who are feeling worried, angry, or sad. We don’t have answers, but if you want to chat or vent, we’re available.

Here’s the best place to reach us if you’d like to talk. I know a form isn’t the warmest, but a real person will get back to you soon.

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: I Converted Book I of The Sequences Into A Zoomer-Readable Format, published by dkirmani on November 10, 2022 on LessWrong.

If I (a 19 year old male) texted "www.readthesequences.com" to my roommate, the probable outcome is that he would skim the site for under a minute, text back something like "seems interesting, I'll def check it out sometime", and then proceed to never read another word. I have another friend, one that I would consider a smart guy. He would consistently rank above me in our high school's math team, and he scored in the 1500's (≥3SD) on his SATs. The same dude did not read a single book during the entirety of his high school career.[1]

Attention is one's scarcest resource, and actually reading something longer than a paragraph is a trivial inconvenience, especially for my generation.

What, then, does manage to hold the fickle eyeballs of zoomers like me? Well, TikTok, mostly. However, there is one (very popular) genre of TikTok video worth investigating. In this genre of video, a Reddit post is broken into sub-paragraph chunks of text, and these chunks are sequentially rendered onscreen while a text-to-speech program reads them to the user. The text is overlaid upon a background video, which is either gameplay from the mobile game Subway Surfers, or parkour footage from Minecraft. The background gameplay provides engaging novelty to the user's visual cortex, while the synthetic voice ensures that the user doesn't have to go through the hard work of translating symbols into sounds. Really, it's all quite hypnotizing.

The fact that these videos are often recommended by TikTok's algorithm imply that they are among the most-engaging videos that our civilization produces. Therefore, to reduce the effort-cost of reading the sequences, I gave the TikTok treatment to Book I ("Map and Territory") of Rationality: From AI to Zombies.

Predictably Wrong

What Do I Mean By “Rationality”?

Feeling Rational

Why Truth? And.

. What’s a Bias, Again?

Availability

Burdensome Details

Planning Fallacy

Illusion of Transparency: Why No One Understands You

Expecting Short Inferential Distances

The Lens That Sees Its Own Flaws

Fake Beliefs

Making Beliefs Pay Rent (in Anticipated Experiences)

A Fable of Science and Politics

Belief in Belief

Bayesian Judo

Pretending to be Wise

Religion’s Claim to be Non-Disprovable

Professing and Cheering

Belief as Attire

Applause Lights

Noticing Confusion

Focus Your Uncertainty

What Is Evidence?

Scientific Evidence, Legal Evidence, Rational Evidence

How Much Evidence Does It Take?

Einstein’s Arrogance

Occam’s Razor

Your Strength as a Rationalist

Absence of Evidence Is Evidence of Absence

Conservation of Expected Evidence

Hindsight Devalues Science

Mysterious Answers

Fake Explanations

Guessing the Teacher’s Password

Science as Attire

Fake Causality

Semantic Stopsigns

Mysterious Answers to Mysterious Questions

The Futility of Emergence

Say Not “Complexity”

Positive Bias: Look into the Dark

Lawful Uncertainty

My Wild and Reckless Youth

Failing to Learn from History

Making History Available

Explain/Worship/Ignore?

“Science” as Curiosity-Stopper

Truly Part of You

Interlude: The Simple Truth

Do whatever you want with these videos. I may or may not convert the other 5 books of R:AZ, and I may or may not upload them to TikTok. If you want another work of text converted to video, please pitch it to me in the comments, or DM me.

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: What should I ask Joe Carlsmith — Open Phil researcher, philosopher and blogger?, published by Robert Wiblin on November 9, 2022 on The Effective Altruism Forum.

Next week for The 80,000 Hours Podcast I'll be interviewing Joe Carlsmith, Senior Research Analyst at Open Philanthropy.

Joe's did a BPhil in philosophy at Oxford University and is a prolific writer on topics both philosophical and practical (until recently his blog was called 'Hands and Cities' but it's all now collected on his personal site.

What should I ask him?

Some things Joe has written which we could talk about include:

Is Power-Seeking AI an Existential Risk?

Actually possible: thoughts on Utopia

On infinite ethics —XIV. The death of a utilitarian dream

Anthropics: Learning from the fact that you exist

Against neutrality about creating happy lives

Wholehearted choices and “morality as taxes”

On clinging

Can you control the past?

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: AI Safety groups should imitate career development clubs, published by Joshc on November 9, 2022 on The Effective Altruism Forum.

If you want to get people to do things (like learn about AI Safety) you have to offer them something valuable.

Here’s one of the posters we used when I was in charge of marketing for the Columbia EA group:

It’s a pretty graphic, but what valuable thing is it offering?

The message is “scan this link to talk about AI.” To be fair, people like talking about AI. We had applicants.

But we didn’t attract talented ML students.

If you want to attract talented people, you have to know what they want. Serious and ambitious people probably don’t want to sit around having philosophical discussions. They want to build their careers.

Enter ML @ Berkeley, a thriving group of 50 ML students who put 15 hours per week into projects and courses to become better at ML. No one gets paid – not even the organizers. And they are very selective. Only around 7% get in.

Why is this group successful? For starters, they offer career capital. They give students projects that often turn into real published papers. They also concentrate talent. Ambitious people want to work with other ambitious people.

AI safety student groups should consider imitating ML @ Berkeley.

I’m not saying that we should eliminate philosophical discussions and replace them with resume boosting factories. We still want people to think AI Safety and X-risk are important. But discussions don’t need to be the primary selling point.

Maybe for cultivating conceptual researchers, it makes more sense for discussions to be central. But conceptual and empirical AI Safety research are very different. ML students are probably more interested in projects and skill-building.

More rigorous programming could also make it easier to identify talent.

Talking about AI is fun, but top ML researchers work extremely hard. Rigorous technical curricula can filter out the ones that are driven.

There is nothing like a trial by fire. Instead of trying to predict in advance who will be good at research, why not have lots of people try it and invest in those that do well?

USC field builders are experimenting with a curriculum that, in addition to introducing X-risk, is packed-full with technical projects. In their first semester, they attracted 30 students who all have strong ML backgrounds. I’m interested in seeing how this goes and would be excited about more AI Safety groups running experiments on these lines.

People could also try:

checking whether grad students are willing to supervise group research projects.

running deep learning courses and training programs (like Redwood’s MLAB)

running an in-person section of intro to ML Safety (a technical course that covers safety topics).

Conclusion

As far as I can tell, no one has AI safety university field-building all figured out. Rather than copying the same old discussion group model, people should experiment with new approaches. A good start could be to imitate career development clubs like ML @ Berkeley that have been highly successful.

Thanks to Nat Li and Oliver Zhang for thoughts and feedback and to Dan Hendrycks for conversations that inspired this post.

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: A caveat to the Orthogonality Thesis, published by Wuschel Schulz on November 9, 2022 on LessWrong.

This post relies on an understanding of two concepts: The Orthogonality thesis and the sharp left turn. If you already know what they are, skip to the main text.

The orthogonality thesis states that an agent can have any combination of intelligence and goals. It is one of the core assumptions of alignment research.

The sharp left turn is a hypothesized event, where the capabilities of an AI suddenly generalize to new domains without its alignment capabilities generalizing. This process is sometimes described as “hitting the core of intelligence” and is considered to be the crucial point of alignment by some, as AIs after a sharp left turn might be more capable than humans. So we have to have AI alignment figured out before an AI takes the sharp left turn.

Main text

While I do think the orthogonality thesis is mostly correct, I have a small caveat:

For an AI to maximize/steer towards x, x must be either part of its sensory input or its world model.

Imagine a really simple “AI”: a thermostat that keeps the temperature. If you take a naive view of the orthogonality thesis, you would have to believe that there is a system as intelligent as a thermostat, that maximizes paperclips. I do not think there is such a system, because it would have no idea what a paperclip is. It doesn't even have a model of the outside world, it can only directly act on its sensory input.

Even for systems that have a world model, they can only maximize things that are represented in their world model. If that world model only includes objects in a room, but the AI does not have the concept of what a person is, and what intentions are, it can optimize towards “put all the red squares in one line” but it can not optimize towards “do what the person intends”.

So a more realistic view of the orthogonality thesis is the following:

In this plot, I have lumped intelligence and world model complexity together. Usually, these concepts go together, because the more intelligent an AI is, the more complex its world model gets. If we found a way to make an AI have an arbitrarily complex world model while still being arbitrarily dumb, this graph would no longer hold true.

Now, this is where the sharp left turn comes into play. Assuming that there is a sharp left turn, there is some maximum intelligence that a system can have, before “hitting the core of intelligence” and suddenly becoming stronger than humans.

The important question is, whether a system hits this core of intelligence just by having a more and more sophisticated world model. If this is the case, then there is some maximum amount of world model complexity that a system can have before taking the sharp left turn. And that would mean in turn, that there is a cap on the goals that we can give an AI in practice.

There might be Goals that are so complex, that any AI intelligent enough to understand them, would have hit the sharp left turn.

There are follow up questions, that I have no answers to, and would be grateful for insightful comments:

Is it true that a more and more sophisticated world model is enough to “hit the core of intelligence”?

Are human values in the reachable or the unreachable goals?

Is corrigibility in the reachable or the unreachable goals?

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Does the US public support radical action against factory farming in the name of animal welfare?, published by Neil Dullaghan on November 9, 2022 on The Effective Altruism Forum.

Summary

Surveys from Sentience Institute (2020, 2017) and Norwood & Murray (2018) showed substantial levels of support in the US for banning slaughterhouses (~39-43% support when including those who chose no opinion/don't know). Evidence of this considerable public support for radical action has been suggested as a reason for animal advocates to push stronger messages and bolder proposals against animal agriculture.

A preregistered study that we present here casts doubt on how substantial support for such radical action against factory farming actually is. In an experiment of 700 US survey respondents, we found 7.85% support (95% CI [4.3% - 14%]), when arguments framed around animal welfare for and against are presented, and respondents are asked to explain their reasoning. We also found 20.41% support (95% CI [11%-34.7%]) in the control condition when respondents were not asked to explain their reasoning.

In the second survey of 2,698 US respondents, the weighted results show 15.7% (95% CI [13%-18.8%]) support for a policy banning slaughterhouses, when arguments framed around animal welfare for and against are presented, and respondents are asked to explain their reasoning.

The attitudes expressed by poll respondents in response to broad questions may not be reliable indicators of actual support for specific policies or messages. It would be better to test people's responses to more detailed messages and policy proposals, paying special attention to how radical messages compare to counterfactual moderate messages.

Results: Attitudes towards a proposal to ban slaughterhouses

In July and August of 2022, we ran two online surveys, one with an experimental design. In both, respondents were presented with a proposal that included a definition of slaughterhouses and arguments framed around animal welfare for and against the proposal. In the survey experiment, the treatment condition asked respondents to explain their reasoning, with this prompt removed in the control condition. In the second survey, only the treatment condition was presented. The question wording was as follows:

Supporters of this policy say that slaughterhouses should be banned because it is wrong to kill animals. There is no way to kill animals for their meat which is “humane,” so this should be banned.

Opponents of this policy say that people have a right to eat meat if they choose. The practices in place are humane and produce quality meat for consumers at an affordable price. It is possible to prevent animals being killed inhumanely without banning slaughterhouses.

Do you support or oppose this proposed policy? [Support/Oppose/Don’t Know]

[Treatment condition] Please explain your reasons for the answer you gave above

In the survey experiment, with a sample of 700 US respondents:

We found much lower levels of support in both control (20.41%, 95% CI [11%-34.7%]) and intervention (7.85%, 95% CI [4.3% - 14%]) conditions compared to the Sentience Institute (2020, 2017) and Norwood & Murray (2018) studies (43% when including “No Opinion” responses). (More results in the appendix.)

In both the unweighted and weighted analyses, support was lower in the intervention condition than in the control condition. The share of “Don’t Know” respondents increased in the intervention in the unweighted analysis, while in the weighted analysis, the lost support seems to come directly from people choosing to oppose instead.

Weighted results for a proposal banning slaughterhouses (control and treatment)

The second survey had a sample of 2,698 US respondents (after filtering based on an honesty check and a basic attention/comprehension check). We pr...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Money Stuff: FTX Had a Death Spiral, published by Elliot Temple on November 9, 2022 on The Effective Altruism Forum.

Matt Levine is a financial writer I like and he explains (the latest information about) what happened with FTX.

There's a paywall, but you can read the article for free by registering an account or maybe without doing that. I got it to load for free by opening it in a private browsing window. The newsletter is also free by email which is how I receive it. The free email signup is at but that won't help you with past issues.

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.