Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: 2020 PhilPapers Survey Results, published by Rob Bensinger on November 2, 2021 on LessWrong.

In 2009, David Bourget and David Chalmers ran the PhilPapers Survey (results, paper), sending questions to "all regular faculty members" at top "Ph.D.-granting [philosophy] departments in English-speaking countries" plus ten other philosophy departments deemed to have "strength in analytic philosophy comparable to the other 89 departments".

Bourget and Chalmers now have a new PhilPapers Survey out, run in 2020 (results, paper). I'll use this post to pick out some findings I found interesting, and say opinionated stuff about them. Keep in mind that I'm focusing on topics and results that dovetail with things I'm curious about (e.g., 'why do academic decision theorists and LW decision theorists disagree so much?'), not giving a neutral overview of the whole 100-question survey.

The new survey's target population consists of:

(1) in Australia, Canada, Ireland, New Zealand, the UK, and the US: all regular faculty members (tenuretrack or permanent) in BA-granting philosophy departments with four or more members (according to the PhilPeople database); and (2) in all other countries: English-publishing philosophers in BA-granting philosophy departments with four or more English-publishing faculty members.

In order to make comparisons to the 2009 results, the 2020 survey also looked at a "2009-comparable departments" list selected using similar criteria to the 2009 survey:

It should be noted that the “2009-comparable department” group differs systematically from the broader target population in a number of respects. Demographically, it includes a higher proportion of UK-based philosophers and analytic-tradition philosophers than the target population. Philosophically, it includes a lower proportion of theists, along with many other differences evident in comparing 2020 results in table 1 (all departments) to table 9 (2009-comparable departments).

Based on this description, I expect the "2009-comparable departments" in the 2020 survey to be more elite, influential, and reasonable than the 2020 "target group", so I mostly focus on 2009-comparable departments below. In the tables below, if the row doesn't say "Target" (i.e., target group), the population is "2009-comparable departments".

Note that in the 2020 survey (unlike 2009), respondents could endorse multiple answers.

1. Decision theory

Newcomb's problem: The following groups (with n noting their size, and skipping people who skipped the question or said they weren't sufficiently familiar with it) endorsed the following options in the 2020 survey:

5% of decision theorists said they "accept a combination of views", and 9% said they were "agnostic/undecided".

I think decision theorists are astonishingly wrong here, so I was curious to see if other philosophy fields did better.

I looked at every field where enough surveyed people gave their views on Newcomb's problem. Here they are in order of 'how much more likely are they to two-box than to one-box':

(Note that many of these groups are small-n. Since philosophers of computing and information were an especially small and weird group, and I expect LWers to be extra interested in this group, I also looked at the target-group version for this field.)

Every field did much better than decision theory (by the "getting more utility in Newcomb's problem" metric). However, the only fields that favored one-boxing over two-boxing was ancient Greek and Roman philosophy, and aesthetics.

After those two fields, the best fields were philosophy of cognitive science, applied ethics, metaphilosophy, philosophy of mathematics, and 17th/18th century philosophy (only 4-5% more likely to two-box than one-box), followed by philosophy of religion, normative ethics, and metaphysics.

My quick post-hoc, low-c...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Effective Evil, published by lsusr on November 2, 2021 on LessWrong.

Many years ago, a blogger made a post advocating for an evil Y-Combinator which subsidized the opposite of Effective Altruism. Everyone (including the blogger) thought the post was a joke except the supervillains. The organization they founded celebrated its 10th anniversary this year. An attendee leaked to me a partial transcript from one of its board meetings.

Director: Historically, public unhealth has caused the most harm per dollar invested. How is is the Center for Disease Proliferation doing?

CDP Division Chief: Gain-of-function research remains—in principle—incredibly cheap. All you have to do is infect ferrets with the flu and let them spread it to one another. We focus on maximizing transmission first and then, once we have a highly-transmissible disease, select for lethality (ideally after a long asymptomatic infectious period).

CFO: You say gain of function research is cheap but my numbers say you're spending billions of dollars on gain-of-function research. Where is all that money going?

CDP Division Chief: Volcano lairs, mostly. We don't want an artificial pandemic to escape our labs by accident.

Director: Point of order. Did the CDP anything to do with COVID-19?

CDP Division Chief: I wish. COVID-19 was a work of art. Dangerous enough to kill millions of people and yet not dangerous enough to get most world governments to take it seriously. After we lost smallpox and polio I thought any lethal disease for which there was an effective vaccine would be eradicated within a year but COVID-19 looks like it would have turned endemic even the zoonotic vectors. We have six superbugs more lethal than COVID-19 sitting around in various island fortresses. We had planned to launch them this year but with all the COVID-19 data coming in I'm questioning whether that's really the right way to go. Primum non boni. If we release a disease too deadly, governments will stamp it down immediately. We'll kill few people while also training governments in how to stop pandemics. It'd be like vaccinating the planet. We don't want a repeat of SARS.

Director: Good job being quick to change your mind in the face of evidence. What's the status of our AI misalignment program?

Master Roboticist: For several years we've been working on mind control algorithms, but we cancelled that initiative in the face of competition with Facebook. I don't like Facebook. They're not optimally evil. There are many ways they could be worse for the world. But their monopoly is unassailable. The network effects are too great.

Director: Where are our AI misalignment funds going instead?

Master Roboticist: For a while our research was going into autonomous weapons. Autonomous weapons make it easier to start a war since leaders can attack deep within enemy territory without risking the lives of their own soldiers. Predator drones also cause lots of collateral damage. A drone strike by the Biden administration killed seven children.

Director: If the program's going so well then why stop it?

Master Roboticist: Competition. China is revamping its military around autonomous warfare. We have extraordinary resources, but even we can't compete with the world's biggest economy.

Director: Perhaps we can co-opt their work. Is there no way to start a nuclear WWIII?

Strategos: Starting a nuclear war is easy. It's so easy that our efforts are actually dedicated to postponing nuclear war by reducing accidents. There's more to evil than just causing the greatest harm. We must cause the greatest harm to the greatest number. Our policy is that we shouldn't start a nuclear war while the world population is still increasing.

Master Roboticist: Moreover, a nuclear war would stop us from summoning Roko's basilisk.

Director: How's the AGI project going by the ...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Effective Evil, published by lsusr on November 2, 2021 on LessWrong.

Many years ago, a blogger made a post advocating for an evil Y-Combinator which subsidized the opposite of Effective Altruism. Everyone (including the blogger) thought the post was a joke except the supervillains. The organization they founded celebrated its 10th anniversary this year. An attendee leaked to me a partial transcript from one of its board meetings.

Director: Historically, public unhealth has caused the most harm per dollar invested. How is is the Center for Disease Proliferation doing?

CDP Division Chief: Gain-of-function research remains—in principle—incredibly cheap. All you have to do is infect ferrets with the flu and let them spread it to one another. We focus on maximizing transmission first and then, once we have a highly-transmissible disease, select for lethality (ideally after a long asymptomatic infectious period).

CFO: You say gain of function research is cheap but my numbers say you're spending billions of dollars on gain-of-function research. Where is all that money going?

CDP Division Chief: Volcano lairs, mostly. We don't want an artificial pandemic to escape our labs by accident.

Director: Point of order. Did the CDP anything to do with COVID-19?

CDP Division Chief: I wish. COVID-19 was a work of art. Dangerous enough to kill millions of people and yet not dangerous enough to get most world governments to take it seriously. After we lost smallpox and polio I thought any lethal disease for which there was an effective vaccine would be eradicated within a year but COVID-19 looks like it would have turned endemic even the zoonotic vectors. We have six superbugs more lethal than COVID-19 sitting around in various island fortresses. We had planned to launch them this year but with all the COVID-19 data coming in I'm questioning whether that's really the right way to go. Primum non boni. If we release a disease too deadly, governments will stamp it down immediately. We'll kill few people while also training governments in how to stop pandemics. It'd be like vaccinating the planet. We don't want a repeat of SARS.

Director: Good job being quick to change your mind in the face of evidence. What's the status of our AI misalignment program?

Master Roboticist: For several years we've been working on mind control algorithms, but we cancelled that initiative in the face of competition with Facebook. I don't like Facebook. They're not optimally evil. There are many ways they could be worse for the world. But their monopoly is unassailable. The network effects are too great.

Director: Where are our AI misalignment funds going instead?

Master Roboticist: For a while our research was going into autonomous weapons. Autonomous weapons make it easier to start a war since leaders can attack deep within enemy territory without risking the lives of their own soldiers. Predator drones also cause lots of collateral damage. A drone strike by the Biden administration killed seven children.

Director: If the program's going so well then why stop it?

Master Roboticist: Competition. China is revamping its military around autonomous warfare. We have extraordinary resources, but even we can't compete with the world's biggest economy.

Director: Perhaps we can co-opt their work. Is there no way to start a nuclear WWIII?

Strategos: Starting a nuclear war is easy. It's so easy that our efforts are actually dedicated to postponing nuclear war by reducing accidents. There's more to evil than just causing the greatest harm. We must cause the greatest harm to the greatest number. Our policy is that we shouldn't start a nuclear war while the world population is still increasing.

Master Roboticist: Moreover, a nuclear war would stop us from summoning Roko's basilisk.

Director: How's the AGI project going by the ...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: [Book Review] "The Bell Curve" by Charles Murray, published by lsusr on November 2, 2021 on LessWrong.

Factor analysis is a mathematical method of inferring simple correlations between observations. It's the foundation of the Big Five personality traits. It's also behind how we define intelligence.

A person's ability to perform one cognitive task is positively correlated with basically every other cognitive task. If you collect a variety of cognitive measures you can use linear algebra to extract a single measure which we call g. Intelligence quotient (IQ) is a test specifically designed to measure g. IQ isn't a perfect measure of g but it's convenient and robust.

Here are six conclusions regarding tests of cognitive ability, drawn from the classical tradition, that are now beyond significant technical dispute:

There is such thing as a general factor of cognitive ability on which human beings differ.

All standardized tests of academic aptitude or achievement measure this general factor to some degree, but IQ tests expressly designed for that purpose measure it most accurately.

IQ scores match, to a first degree, whatever it is that ordinary people mean when they use the word intelligent or smart in ordinary language.

IQ scores are stable, although not perfectly so, over much of a person's life.

Properly administered IQ tests are not demonstrably biased against social, economic, ethnic, or racial groups.

Cognitive ability is substantially heritable, apparently no less than 40 percent and no more than 80 percent.

Charles Murray doesn't bother proving the above points. These facts are well established among scientists. Instead, The Bell Curve: Intelligence and Class Structire in American Life is about what g means to American society.

Stratification

Educational Stratification

Smarter people have always had an advantage. The people who go to college have always been smarter than average. The correlation between college and intelligence increased after WWII. Charles Murray argues that the competitive advantage of intelligence is magnified in a technological society. I agree that this has been the case so far and that the trend has continued between 1994 when Murray published his book and 2021 when I am writing this review.

SAT scores can be mapped to IQ. The entering class of Harvard in 1926 had a mean IQ of about 117. IQ is defined to have an average of 100 and a standard deviation of 15. Harvard in 1926 thus hovered around the 88th percentile of the nation's youths. Other colleges got similar scores. The average Pennsylvania college was lower with an IQ of 107 (68th percentile). Elite Pennsylvania colleges had students between the 75th and 90th percentiles.

By 1964, the average student of a Pennsylvania college had an IQ in the 89th percentile. Elite colleges' average freshmen were in the 99th percentile.

Charles Murray uses a measure called median overlap to quantify social stratification. Median overlap indicates what proportion of IQ scores the lower-scoring group matched or exceeded the median score in the higher-scoring group. Two identical groups would have a median overlap of 50%.

College graduates are not representative of the population. If most of your social circle is (or will be) a college graduate then your social circle is smarter than the population mean.

The national percentage of 18-year-olds with the ability to get a score of 700 or above on the SAT-Verbal test is in the vicinity of one in three hundred. Think about the consequences when about half of these students are going to universities in which 17 percent of their classmates also had SAT-Vs in the 700s and another 48 percent had scores in the 600s. It is difficult to exaggerate how different the elite college population is from the population at large—first in its level of intellectual talent, an...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: [Book Review] "The Bell Curve" by Charles Murray, published by lsusr on November 2, 2021 on LessWrong.

Factor analysis is a mathematical method of inferring simple correlations between observations. It's the foundation of the Big Five personality traits. It's also behind how we define intelligence.

A person's ability to perform one cognitive task is positively correlated with basically every other cognitive task. If you collect a variety of cognitive measures you can use linear algebra to extract a single measure which we call g. Intelligence quotient (IQ) is a test specifically designed to measure g. IQ isn't a perfect measure of g but it's convenient and robust.

Here are six conclusions regarding tests of cognitive ability, drawn from the classical tradition, that are now beyond significant technical dispute:

There is such thing as a general factor of cognitive ability on which human beings differ.

All standardized tests of academic aptitude or achievement measure this general factor to some degree, but IQ tests expressly designed for that purpose measure it most accurately.

IQ scores match, to a first degree, whatever it is that ordinary people mean when they use the word intelligent or smart in ordinary language.

IQ scores are stable, although not perfectly so, over much of a person's life.

Properly administered IQ tests are not demonstrably biased against social, economic, ethnic, or racial groups.

Cognitive ability is substantially heritable, apparently no less than 40 percent and no more than 80 percent.

Charles Murray doesn't bother proving the above points. These facts are well established among scientists. Instead, The Bell Curve: Intelligence and Class Structire in American Life is about what g means to American society.

Stratification

Educational Stratification

Smarter people have always had an advantage. The people who go to college have always been smarter than average. The correlation between college and intelligence increased after WWII. Charles Murray argues that the competitive advantage of intelligence is magnified in a technological society. I agree that this has been the case so far and that the trend has continued between 1994 when Murray published his book and 2021 when I am writing this review.

SAT scores can be mapped to IQ. The entering class of Harvard in 1926 had a mean IQ of about 117. IQ is defined to have an average of 100 and a standard deviation of 15. Harvard in 1926 thus hovered around the 88th percentile of the nation's youths. Other colleges got similar scores. The average Pennsylvania college was lower with an IQ of 107 (68th percentile). Elite Pennsylvania colleges had students between the 75th and 90th percentiles.

By 1964, the average student of a Pennsylvania college had an IQ in the 89th percentile. Elite colleges' average freshmen were in the 99th percentile.

Charles Murray uses a measure called median overlap to quantify social stratification. Median overlap indicates what proportion of IQ scores the lower-scoring group matched or exceeded the median score in the higher-scoring group. Two identical groups would have a median overlap of 50%.

College graduates are not representative of the population. If most of your social circle is (or will be) a college graduate then your social circle is smarter than the population mean.

The national percentage of 18-year-olds with the ability to get a score of 700 or above on the SAT-Verbal test is in the vicinity of one in three hundred. Think about the consequences when about half of these students are going to universities in which 17 percent of their classmates also had SAT-Vs in the 700s and another 48 percent had scores in the 600s. It is difficult to exaggerate how different the elite college population is from the population at large—first in its level of intellectual talent, an...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Stanley the Uncertain [Creative Writing Contest], published by Akash on October 28, 2021 on The Effective Altruism Forum.

“Of course you should pull the lever; you’re saving four people!”

“But if you pull the lever, you’re actively choosing to cause the death of someone else. You would be violating a core ethical principle!”

“Ugh, you don’t get it. Stanley, what do you think?”

Stanley stared back at his classmates. After a few moments of silence, he responded:

“Huh. I’m not sure. I want to think about this more. Can you ask me next week?”

His classmates looked displeased, but they reluctantly agreed.

Next week, after much contemplation, Stanley had an answer for his classmates.

“At first, I was 70% confident that pulling the lever would be wrong. But after thinking about it some more, and reading some essays, I’ve changed my mind. I’m 65% confident that pulling the lever is right, though I still have high uncertainty and a few unresolved questions.”

His classmates stared back.

“Jeez, Stanley,” one of them responded. “I can’t imagine what it’s like being inside your head.”

“How do you function with all of that uncertainty?”

Stanley simply shrugged, and the three continued their walk home.

At home, Stanley was studying for an upcoming philosophy exam when he noticed a splat. His fish, Percy, had plopped out of his bowl and onto the ground. Percy was floundering, deprived of water.

Stanley found himself overwhelmed with thoughts. Are fish conscious? I haven’t examined this yet. and if they are, what does that mean about how we eat them? We also eat chickens, which probably have a higher likelihood of being conscious. Also, even if Percy is conscious, is he even living a net positive life? Is a life that’s trapped in a bowl really a life worth living? And if I don’t act, am I morally responsibli--?

Suddenly, Stanley’s mother rushed into the room, grabbed Percy, and put him back into his bowl.

“Stanley! What were you thinking?” she cried.

“I. I just. I wasn’t sure what to do.”

“Sometimes, Stanley, it doesn’t matter if you’re sure. You still have to act.”

Years later, Stanley became a pilot. As he was flying, he noticed something strange.

The fuel gauge showed that the plane still had 75% of its fuel. But that was impossible; Stanley was confident that he had used more fuel. Stanley started to consider possible explanations: maybe his prediction was off, maybe it was an inconsequential fluke. Or maybe it signaled that there was something wrong with the dashboard-- or worse, the monitoring system itself.

“This is your captain aboard.” Stanley’s voice boomed over the speakers, “Please fasten your seatbelts; we’re going to be engaging in an emergency landing.”

Hours later, after the flight had landed, the airline noticed a fatal error in the plane’s power system. If it had gone unaddressed for the length of the original flight, the plane almost certainly would have crashed.

One of the fellow crew members asked, “Stanley, how did you know?”

Stanley responded, “I didn’t know. I was only 10% sure.”

“10%? You broke airline policy to initiate an emergency landing, and you were only 10% sure? Are you crazy?”

“I think I see your concern, and I agree. If I had been under less stress, my reasoning would have been better. Given all of the information I had, I should have been able to conclude that there was an 18% chance of failure.”

The crewmember stared at Stanley in disbelief.

“But the thing is, as soon as I was confident that the risk was at least 10%, I decided to stop calculating.” Stanley added.

"At that point, it didn’t matter if the risk was 10%, 18%, or 75%. My decision would have been the same.”

“Besides,” Stanley added, “It’s taken me a lot of practice. But I’ve learned that sometimes, even when you’re not sure, you still have to act.”

“Do you take Stanley to be your husban...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Stanley the Uncertain [Creative Writing Contest], published by Akash on October 28, 2021 on The Effective Altruism Forum.

“Of course you should pull the lever; you’re saving four people!”

“But if you pull the lever, you’re actively choosing to cause the death of someone else. You would be violating a core ethical principle!”

“Ugh, you don’t get it. Stanley, what do you think?”

Stanley stared back at his classmates. After a few moments of silence, he responded:

“Huh. I’m not sure. I want to think about this more. Can you ask me next week?”

His classmates looked displeased, but they reluctantly agreed.

Next week, after much contemplation, Stanley had an answer for his classmates.

“At first, I was 70% confident that pulling the lever would be wrong. But after thinking about it some more, and reading some essays, I’ve changed my mind. I’m 65% confident that pulling the lever is right, though I still have high uncertainty and a few unresolved questions.”

His classmates stared back.

“Jeez, Stanley,” one of them responded. “I can’t imagine what it’s like being inside your head.”

“How do you function with all of that uncertainty?”

Stanley simply shrugged, and the three continued their walk home.

At home, Stanley was studying for an upcoming philosophy exam when he noticed a splat. His fish, Percy, had plopped out of his bowl and onto the ground. Percy was floundering, deprived of water.

Stanley found himself overwhelmed with thoughts. Are fish conscious? I haven’t examined this yet. and if they are, what does that mean about how we eat them? We also eat chickens, which probably have a higher likelihood of being conscious. Also, even if Percy is conscious, is he even living a net positive life? Is a life that’s trapped in a bowl really a life worth living? And if I don’t act, am I morally responsibli--?

Suddenly, Stanley’s mother rushed into the room, grabbed Percy, and put him back into his bowl.

“Stanley! What were you thinking?” she cried.

“I. I just. I wasn’t sure what to do.”

“Sometimes, Stanley, it doesn’t matter if you’re sure. You still have to act.”

Years later, Stanley became a pilot. As he was flying, he noticed something strange.

The fuel gauge showed that the plane still had 75% of its fuel. But that was impossible; Stanley was confident that he had used more fuel. Stanley started to consider possible explanations: maybe his prediction was off, maybe it was an inconsequential fluke. Or maybe it signaled that there was something wrong with the dashboard-- or worse, the monitoring system itself.

“This is your captain aboard.” Stanley’s voice boomed over the speakers, “Please fasten your seatbelts; we’re going to be engaging in an emergency landing.”

Hours later, after the flight had landed, the airline noticed a fatal error in the plane’s power system. If it had gone unaddressed for the length of the original flight, the plane almost certainly would have crashed.

One of the fellow crew members asked, “Stanley, how did you know?”

Stanley responded, “I didn’t know. I was only 10% sure.”

“10%? You broke airline policy to initiate an emergency landing, and you were only 10% sure? Are you crazy?”

“I think I see your concern, and I agree. If I had been under less stress, my reasoning would have been better. Given all of the information I had, I should have been able to conclude that there was an 18% chance of failure.”

The crewmember stared at Stanley in disbelief.

“But the thing is, as soon as I was confident that the risk was at least 10%, I decided to stop calculating.” Stanley added.

"At that point, it didn’t matter if the risk was 10%, 18%, or 75%. My decision would have been the same.”

“Besides,” Stanley added, “It’s taken me a lot of practice. But I’ve learned that sometimes, even when you’re not sure, you still have to act.”

“Do you take Stanley to be your husban...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Can EA leverage an Elon-vs-world-hunger news cycle?, published by Jackson Wagner on November 2, 2021 on The Effective Altruism Forum.

Summary: Elon Musk promises to donate six billion dollars if the UN can explain how this would truly solve world hunger (it would probably be much more expensive). Regardless of whether the donation happens or not, a major news cycle about the cost-effectiveness of international charitable donations seems like a great opportunity to raise the public profile of effective altruism.

Details of The Billionare-Bashing Drama

US senators are currently debating a large bill that will probably include some form of tax increase on the rich. Elon Musk, now the world's richest man, voiced his opposition to a proposed tax on unrealized capital gains. He framed his oppositon specifically in terms of government inefficiency, saying:

"Who is best at capital allocation -- government or entrepeneurs -- is indeed what it comes down to."

More recently, a recent CNN headline asserted that just 2% of Elon Musk's ~$300B net worth could "solve world hunger" by feeding the 42 million people who suffer from malnutrition -- people who are otherwise "literally going to die".

Inevitably, this claim turns out to be somewhat hyperbolic/innumerate -- 6 billion dollars divided by 42 million people is around $140 per person, which would be a lot lower than givewell's most effective interventions (around ~$5000 per life saved). Maybe this billionare-bashing CNN interview is revealing an astounding, hithero unknown charitable opportunity. But more likely, most of the people are not literally going to die and/or the effort to alleviate the problem would cost much more than $6 billion (at the very least, if we need to keep giving people food each year, the real cost would be $6 billion repeating annually). I haven't yet looked into the details of the situation too closely.

Now, Elon has offered to indeed donate $6 billion, on the (presumably impossible) condition that the UN provide a realistic plan for how the problem of world hunger could legitimately be solved on that budget. For scale, six billion dollars devoted to EA cause areas would represent more than a 10% increase on the ~$42B total funds currently committed to the movement. Right now, EA organizations are spending spend about $0.2 billion on GiveWell-style global health charities each year.

This Seems Like A Good Time For EA To Shine

This conversation is already distinct from most billionare-related discorse for its focus on cost-effectiveness and international aid for the world's poorest, rather than the usual arguing over the fairness of allowing rich people to exist at all and the desire to increase taxes in order to fund more social services in the developed world.

In short, for a brief moment in time, a major news cycle is focused on how one can do the most good to save the most lives per dollar. This obviously seems like a great time to introduce the ideas of effective altruism to more people. I can only imagine that Kelsey Piper is already busily drafting up an article about this for Future Perfect. But what else can EA do to capitalize on this news cycle? Should Givewell try to outline how they would attempt to spend six billion dollars? Surely their current top charities would run out of room-for-more-funding? Would it be wiser to stay on-message with a relatively simple theme, like promoting Givewell's expertise in cost-effective global health and development spending? Or should we try to fire off a bunch of thinkpieces climbing the counterintuitiveness ladder from typical disaster aid to growth economics, and from there onwards to longtermism, x-risk reduction, etc?

What should EA's general strategy be around these news cycles -- the movement generally tries to avoid political polarization, but surel...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Can EA leverage an Elon-vs-world-hunger news cycle?, published by Jackson Wagner on November 2, 2021 on The Effective Altruism Forum.

Summary: Elon Musk promises to donate six billion dollars if the UN can explain how this would truly solve world hunger (it would probably be much more expensive). Regardless of whether the donation happens or not, a major news cycle about the cost-effectiveness of international charitable donations seems like a great opportunity to raise the public profile of effective altruism.

Details of The Billionare-Bashing Drama

US senators are currently debating a large bill that will probably include some form of tax increase on the rich. Elon Musk, now the world's richest man, voiced his opposition to a proposed tax on unrealized capital gains. He framed his oppositon specifically in terms of government inefficiency, saying:

"Who is best at capital allocation -- government or entrepeneurs -- is indeed what it comes down to."

More recently, a recent CNN headline asserted that just 2% of Elon Musk's ~$300B net worth could "solve world hunger" by feeding the 42 million people who suffer from malnutrition -- people who are otherwise "literally going to die".

Inevitably, this claim turns out to be somewhat hyperbolic/innumerate -- 6 billion dollars divided by 42 million people is around $140 per person, which would be a lot lower than givewell's most effective interventions (around ~$5000 per life saved). Maybe this billionare-bashing CNN interview is revealing an astounding, hithero unknown charitable opportunity. But more likely, most of the people are not literally going to die and/or the effort to alleviate the problem would cost much more than $6 billion (at the very least, if we need to keep giving people food each year, the real cost would be $6 billion repeating annually). I haven't yet looked into the details of the situation too closely.

Now, Elon has offered to indeed donate $6 billion, on the (presumably impossible) condition that the UN provide a realistic plan for how the problem of world hunger could legitimately be solved on that budget. For scale, six billion dollars devoted to EA cause areas would represent more than a 10% increase on the ~$42B total funds currently committed to the movement. Right now, EA organizations are spending spend about $0.2 billion on GiveWell-style global health charities each year.

This Seems Like A Good Time For EA To Shine

This conversation is already distinct from most billionare-related discorse for its focus on cost-effectiveness and international aid for the world's poorest, rather than the usual arguing over the fairness of allowing rich people to exist at all and the desire to increase taxes in order to fund more social services in the developed world.

In short, for a brief moment in time, a major news cycle is focused on how one can do the most good to save the most lives per dollar. This obviously seems like a great time to introduce the ideas of effective altruism to more people. I can only imagine that Kelsey Piper is already busily drafting up an article about this for Future Perfect. But what else can EA do to capitalize on this news cycle? Should Givewell try to outline how they would attempt to spend six billion dollars? Surely their current top charities would run out of room-for-more-funding? Would it be wiser to stay on-message with a relatively simple theme, like promoting Givewell's expertise in cost-effective global health and development spending? Or should we try to fire off a bunch of thinkpieces climbing the counterintuitiveness ladder from typical disaster aid to growth economics, and from there onwards to longtermism, x-risk reduction, etc?

What should EA's general strategy be around these news cycles -- the movement generally tries to avoid political polarization, but surel...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Models Modeling Models, published by Charlie Steiner on November 2, 2021 on The AI Alignment Forum.

I - Meanings

Now that we have some more concrete thinking under our belt, it's time to circle back on Goodhart's law for value learners. What sorts of bad behavior are we imagining from future value-learning AI? What makes those behaviors plausible, and what makes them bad?

Let's start with that last point first. Judgments of goodness or badness get contextualized by models, so our framing of Goodhart's law depends on what models of humans we tolerate. When I say "I like dancing," this is a different use of the word 'like,' backed by a different model of myself, than when I say "I like tasting sugar." The model that omces to mind for dancing treats it as one of the chunks of my day, like "playing computer games" or "taking the bus." I can know what state I'm in (the inference function of the model) based on seeing and hearing short scenes. Meanwhile, my model that has the taste of sugar in it has states like "feeling sandpaper" or "stretching my back." States are more like short-term sensations, and the described world is tightly focused on my body and the things touching it.

The meta-model that talks about me having preferences in both of these models is the framing of competent preferences. If someone or something is observing humans, it looks for human preferences by seeing what the preferences are in "agent-shaped" models are powerful for their size. (At least, up to some finite amount of shuffling that's like a choice of prior or universal Turing machine. Also note that the details of the definition of "agent-shaped" matter. I choose a definition where it's okay to have models that have limited domains of validity, which means you can get conflicts between preferences with partially overlapping domains of validity.)

So when we call certain behavior "bad," the usage of that word might carry with it the implication of what way of thinking about the world that judgment is situated in, like how "I like dancing" makes sense when situated in a model of chunks of my day. There's not one True Model in which the True Meaning of the word "bad" is expressed, though there can still be regularities among the different notions of badness.

II - Mergers

What were the patterns that have stood out from our previous discussions of what humans think of as bad behavior in value learning?

The most common type of failure, especially in modern day AI, is when humans are actively wrong about what's going to happen. They have something specific in mind when designing an AI, like training a boat to win the race, but then they run it and don't get what they wanted. The boat crashes and is on fire. We could make the boat racing game more of a value learning problem by training on human demonstrations rather than the score, and crashing and being on fire would still be bad behavior.

For simple systems where humans are good at understanding the state space and picturing what they want, this is all you need, but for more complicated systems (e.g. our galaxy) humans can only understand small parts or simple properties of the whole system, and we apply our preferences to those parts we can understand. From the inside, it can be hard to feel the distinction! We can want things about tic-tac-toe or about the galaxy with the same set of emotions. What makes deciding what to do with the galaxy different is that we have these scattered preferences about different parts and patterns, and the different parts don't stay neatly separate from each other. They can interact or overlap in ways that bring our preferences into conflict.

This is a key point. Inter-preference conflicts aren't an issue that ever comes up if you think of humans as having a utility function, but they're almost unavoidable if you thin...