Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: New EA Cause Area: Run Blackwell's Bookstore, published by Ben Pace on February 5, 2022 on The Effective Altruism Forum.

Founded in 1879, Blackwell's is the leading academic bookstore in the UK. And right now, it's for sale.

During my time at Oxford it was one of my favorite places to be, an immersive book store where I could sit and read all day. It was the sort of niche academic place that had a wall with 30 copies of Superintelligence, as well as a popular enough place to have an in-store Starbucks.

(They even once had an academic publishing arm. You can see their dozens of academic anthologies on Amazon such as Epistemology, Romanticism, Philosophy of Science and more.)

The famous Norrington Room (an underground cavern of books) feels very immersive and left quite an impression on anyone I brought with me.

When I shared this news with friends earlier today, I wrote "In another world, I’d love to buy Blackwell’s and run it". I think that running a large business like this profitably would be difficult and exciting... I also suspect it would offer a lot of levers into the world of academic publishing and for building and shaping the growth of the intellectual scenes throughout the UK.

It seemed to me like a potential philanthropic opportunity for growing and shaping the academic and intellectual scenes in the UK, and so below I've written up a little of the case for what that might look like.

Epistemic status: I got excited about it and wrote this post in ~4 hrs. Currently more like a dream than a plan, and I don't have much familiarity with the storefront book retail industry (though I have published, printed and sold books). But I think it’s a real opportunity to consider.

Outline of the sections of this post:

Blackwell's has 18 stores and ~350 staff. It's in decline and available to buy, and I estimate it’s on sale in the range of $5-$15MM.

Blackwell's is a place with a long-held cultural space in Oxford and a respected brand.

Blackwell's is in a relatively good position to help grow and build the academic and intellectual scene in the UK, due to it being a respected former-publisher, book seller, event organizer, and having prime real-estate in many UK university towns.

I speculate that a 90th percentile successful outcome from running Blackwell's could look some of the following three outcomes:

10x or 100x-ing our ability to broadly publish respected and widely-read books. This is due to being a respected academic publisher in the past, and potentially doing the same again in the future.

Reward scientists whose work is real and interesting leading to better scientific progress. This is due to being an academic marketplace that chooses what to buy and what to advertise.

Build up Rationalist/EA communities throughout university towns to 10x-100x the level of engagement. This would be due to holding regular events with authors and public intellectuals who have written books or are in-town, and building a local Rationalist/EA/other society around the bookstore and its events.

The capacity to run large functional organizations is rare and valuable in Rationality/EA, and I suspect people who gain this skill to be able to allow us to make moves we otherwise would not be able to (i.e. building new functional organizations on direct priorities).

This idea is most likely but-a-dream, because Waterstones/Barnes&Noble (two brands, one company) currently have a period of exclusivity in which to negotiate a deal. Also I do not plan to do this myself, and I do not have a founder-type person in mind for the job. But I thought I'd share the idea anyway because I felt excited about it. (And challenges of this magnitude have certainly been overcome in the past.)

If you think you could take on this job of managing 350+ people and have some basic taste in scientific research, and ...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: A Quick Look At 20% Time, published by Elizabeth on February 4, 2022 on LessWrong.

I was approached by a client to research the concept of 20% time for engineers, and they graciously agreed to let me share my results. Because this work was tailored to the needs of a specific client, it may have gaps or assumptions that make it a bad 101 post, but in the expectation that it is more useful than not publishing at all, I would like to share it (with client permission).

Side project time, popularized as 20% time at Google, is a policy that allows employees to spend a set percentage of their time on a project of their choice, rather than one directed by management. In practice this can mean a lot of different things, ranging from “spend 20% of your time on whatever you want” to “sure, spend all the free time you want generating more IP for us, as long as your main project is completely unaffected” (often referred to as 120% time) to “theoretically you’re free to do whatever, but we’ve imposed so many restrictions that this means nothing”. I did a 4-hour survey to get a sense of what implementations were available and how they felt for workers.

A frustration here is that almost all of what I could find via Google searches were puff-pieces, anti-puff-pieces, and employees complaining on social media (and one academic article). The single best article I found came not through a Google search, but because I played D&D with the author 15 years ago and she saw me talking about this on Facebook. She can’t be the only one writing about 20% time in a thoughtful way and I’m mad that that writing has been crowded out by work that is, at best, repetitive, and at worst actively misleading.

There are enough anecdotal reports that I believe 20% time exists and is used to good effect by some employees at some companies (including Google) some of the time. The dearth of easily findable information on specific implementations, managerial approaches, trade-offs, etc, makes me downgrade my estimate of how often that happens, vs 20% time being a legible signal of an underlying attitude towards autonomy, or a dubious recruitment tool. I see a real market gap for someone to explain how to do 20% time well at companies of different sizes and product types.

But in the meantime, here’s the summary I gave my client. Reminder: this was originally intended for a high-context conversation with someone who was paying me by the hour, and as such is choppier, less nuanced, and has different emphases than ideal for a public blog post.

My full notes are available here.

To the extent it’s measured, utilization appears to be low, so the policy doesn’t cost very much.

In 2015, a Google HR exec estimated utilization at 10% (meaning it took 2% of all employees’ time).

In 2009, 12 months after Atlassian introduced 20% time, recorded utilization was at 5% (meaning employees were measured to spend 1.1% of their time on it) and estimated actual utilization was <=15% (Notably, nobody complains that Atlassian 20% is fake, and I confirmed with a recently departed employee that it was still around as of 2020).

Interaction with management and evaluation is key. A good compromise is to let people spend up to N hours on a project, and require a check-in with management beyond that.

Googlers consistently (although not universally) complained on social media that even when 20% time was officially approved, you’d be a fool to use it if you wanted a promotion or raises.

However a manager at a less famous company indicated this hadn’t been a problem for them, and that people who approached perf the way everyone does at Google would be doomed anyway. So it looks like you can get out of this with culture.

An approval process is the kiss of death for a feeling of autonomy, but letting employees work on garbage for 6 months and the...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Antifungal Resistance - The Neglected Cousin of Antibiotic Resistance, published by Madhav Malhotra on February 4, 2022 on The Effective Altruism Forum.

For any researchers interested in healthcare and neglected problems, I recorded this podcast with an expert on antifungal resistance. Here are a few key stats (sources linked):

Fungal diseases kill 1.7M people every year. Comparatively, cancer kills 10M per year - but also has a lot more funding & >10x more developed drugs.

Fungal diseases are harder to treat than bacterial infections since fungal cells are like our cells. So drugs against them also have side effects on us.

Large-scale research on them started 80 years after research on bacterial/viral diseases.

More fungal species are infecting humans due to climate change. Since most fungi couldn't survive at high temperatures like in our body before.

We have about 4x fewer antifungals than antibiotics being developed in any given year. But both fungi and bacteria are becoming resistant.

If you have skills in biomedical or public health research and want to get involved in a more neglected field, feel free to reach out to the expert I interviewed. Or find more resources at the Global Action Fund for Fungal Infections

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Impactful Forecasting Prize for forecast writeups on curated Metaculus questions, published by elifland on February 4, 2022 on The Effective Altruism Forum.

cross-posted to LessWrong

TLDR

We’re giving out $4,000 to the best forecast writeups submitted via this form on these Metaculus questions by March 11 to encourage more people to forecast on impactful questions and write up their reasoning.

Motivation

We believe that forecasting on impactful questions is a great candidate for an activity more EA-interested people should partake in, given that it:

Provides direct value when done on decision-relevant questions.

Improves and demonstrates good judgment and research skills.

Can be fun: provides a concrete and gamified framing for activities that look similar to research.

Leads to generally learning more about the world.

Helps match up bright, like-minded collaborators.

This is informed by personal experience: we have learned a lot and found great collaborators through forecasting.

However, it’s difficult to start doing impactful forecasting right now. Metaculus has lots of questions, most of which aren’t selected for impact. It can be overwhelming to navigate and find questions that are both impactful and interesting. It may also be difficult to know where to start in an analysis. Additionally, it can be scary to share your thoughts publically without a push and if it’s good reasoning, the incentives are usually against you.[1]

How to participate

We curated 25 Metaculus questions for prediction in this Airtable. Either Eli Lifland or Misha Yagudin has forecasted on each of these to provide a starting point for further analysis. The table has cause area tags to allow filtering for interest and expertise.

To participate: do the following by March 11 2022, 11:59 PM Anywhere On Earth:

Make a forecast on one of the selected questions and write up your reasoning.

Write up the reasoning for your forecast, on Metaculus or elsewhere (e.g. a forum/blog post).

We require you to share your reasoning publically, unless there are infohazards in which case we will consider private submissions.

Fill out this form with a link to your writeup and contact info; anonymous writeups and email addresses are okay.

You may submit up to one writeup per question; if you submit multiple, your final one will be used. You may submit entries for as many questions as you please.

We will host meetups in this gather.town space to chat about the contest and facilitate forecasting on the questions together. The meetups will be on Wed Feb 16 and Wed Mar 2, 6:30 - 8:30 PM UTC. See this ICS file for calendar events.

Prize details

We will distribute a total of $4,000 to the best forecast writeups of questions in the curated Airtable. Eli and Misha will judge which writeups are most valuable. Sam Glover may help with preliminary evaluations. The first prize will be a maximum of $2,000 and we will distribute prizes for a maximum of 15 writeups.

We will rate each analysis individually, and there is no limit on the amount of prizes a participant can win. Collaborative writeups are allowed and encouraged: writeups which are a collaboration by multiple forecasters should select one person as contact, and may distribute the prize among themselves as they please.

The primary criterion we will use for judging entries is how much it changes our mind compared to our initial forecasts. In that sense, this can be viewed as a large forecasting amplification experiment.

The following caveats apply:

We may adjust based on a subjective notion of how strongly held our previous views were, such that changing a strongly held view is rewarded more than changing a weakly held view.

Writeups that are less counterfactually valuable, e.g. widespread news updates that would be shared anyway, will be less likely to receive prizes.

We ...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Alignment versus AI Alignment, published by Alex Flint on February 4, 2022 on The AI Alignment Forum.

Financial status: This work is supported by individual donors and a grant from LTFF.

Epistemic status: This post contains many inside-view stories about the difficulty of alignment.

Thanks to Adam Shimi, John Wentworth, and Rob Miles for comments on this essay.

What exactly is difficult about AI alignment that is not also difficult about alignment of governments, economies, companies, and other non-AI systems? Is it merely that the fast speed of AI makes the AI alignment problem quantitatively more acute than other alignment problems, or are there deeper qualitative differences? Is there a real connection between alignment of AI and non-AI systems at all?

In this essay we attempt to clarify which difficulties of AI alignment show up similarly in non-AI systems, and which do not. Our goal is to provide a frame for importing and exporting insights from and to other fields without losing sight of the difficulties of AI alignment that are unique. Clarifying which difficulties are shared should clarify the difficulties that are truly unusual about AI alignment.

We begin with a series of examples of aligning different kinds of systems, then we seek explanations for the relative difficulty of AI and non-AI alignment.

Alignment in general

In general, we take actions in the world in order to steer the future in a certain direction. One particular approach to steering the future is to take actions that influence the constitutions of some intelligent system in the world. A general property of intelligent systems seems to be that there are interventions one can execute on them that have robustly long-lasting effects, such as changing the genome of a bacterium, or the trade regulations of a market economy. These are the aspects of the respective intelligent systems that persist through time and dictate their equilibrium behavior. In contrast, although plucking a single hair from a human head or adding a single barrel of oil to a market does have an impact on the future, the self-correcting mechanisms of the respective intelligent systems negate rather than propagate such changes.

Furthermore, we will take alignment in general to be about utilizing such interventions on intelligent systems to realize our true terminal values. Therefore we will adopt the following working definition of alignment:

Successfully taking an action that steers the future in the direction of our true terminal values by influencing the part of an intelligent system that dictates its equilibrium behavior.

Our question is: in what ways is the difficulty of alignment of AI systems different from that of non-AI systems?

Example: Aligning an economic society by establishing property rights

Suppose the thing we are trying to align is a human society and that we view that thing as a collection of households and firms making purchasing decisions that maximize their individual utilities. Suppose that we take as a working operationalization of our terminal values the maximization of the sum of individual utilities of the people in the society. Then we might proceed by creating the conditions for the free exchange of goods and services between the households and firms, perhaps by setting up a government that enforces property rights. This is one particular approach to aligning a thing (a human society) with an operationalization of The Good (maximization of the sum of individual utilities). This particular approach works by structuring the environment in which the humans live in such a way that the equilibrium behavior of the society brings about accomplishment of the goal. We have:

An intelligent system being aligned, which in this case is a human society.

A model of that system, which in this case is a collection o...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Covid 2/3/22: Loosening Bounds, published by Zvi on February 3, 2022 on LessWrong.

In better discourse this week, I have a long reply to Scott Alexander’s Bounded Distrust. I also spun off another piece about a toolkit that’s advocating for schools to return to normal, The Urgency of Normal: An Exercise in Bounded Distrust.

As a result of all that, I ran out of time before I could get to everything this week, so this post does not include any coverage of The Convoy, which is getting pushed into either its own post or next week, depending on what I find when I dig into it more.

Executive Summary

Cases declining quickly.

My people are being let go in some places but not many yet, mostly slowly.

Looks like we’ve got us a convoy up in Canada as a protest, will cover later.

Let’s run the numbers.

The Numbers

Predictions

Prediction from last week: 2.85mm cases (-30%) and 20,000 deaths (+25%).

Results: 2.48mm cases (-39%) and 17,669 deaths (+10%)

Prediction for next week: 1.49mm cases (-40%) and 18,000 deaths (+2%).

Basically the whole nation is in rapid decline in cases so no reason not to expect the trend to continue another week. At some point the decline will slow, but it should be a while before there’s enough adjustments for too much of that. Deaths were a pleasant surprise after an unpleasant one, which usually means things are evening out from some wonkiness. The death peak should be approaching soon given the curve of cases. On reflection I do expect one final increase, and we might see two or possibly a small third one, but that should be it and I doubt we ever hit 20,000 in a week.

Deaths

Worth noting that recent rise in UK deaths is due to a backlog, actual new deaths are falling.

Also figured I’d post this here.

Cases

That’s an even bigger decline than I expected, and it is happening across 49 states and all four regions. You love to see it. Now that the trend is clear, no reason not to expect it to continue.

New York ER doctor who has given us several updates in the past reports no Covid patients his entire shift.

Vaccinations

You know. For kids. Pfizer is officially applying to the FDA to allow them to vaccinate children as young as six months.

This is vitally needed, because everyone involved is responding to the inability to vaccinate those children by acting completely crazy given that these children are at essentially zero risk.

Let’s be honest. We are not going to be vaccinating one-year-olds against Covid-19. We are going to be vaccinating one-year-olds against Covid-19 prevention. With two (or perhaps three) shots that have an acceptable level of short term side effects, we protect such children against being unable to show their face or see the faces of others, of being unable to go to playgrounds and the houses of family friends, and allow them to be children again.

Does the vaccine prevent Covid-19 in children this young given the doses we’ll be giving them?Um, well, yeah, heard to say, quite possibly no? They cut the dose quite a lot, and the data looks a lot like it mostly doesn’t work for kids over the age of two. If it does, it likely requires a booster shot, at which point being immune will matter even less.

But also why should we much care?

Once this is done, we no longer have to hear about ‘and some of us have children who aren’t eligible for vaccination.’ If you want to be paranoid parent and take care of that, sure, great, you can go do that. Let my people go.

I have skin in this game. I have a four-year-old. If I could tell no one whether he’s vaccinated, would I get him vaccinated? Probably not, but both costs and benefits are tiny so it’s not clear.

But is it worth it if it lets everyone involved relax? If it means the preschool lets them take their masks off a lot faster and we don’t have to test constantly for no reason? Then yes, of c...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: What can we learn from a short preview of a super-eruption and what are some tractable ways of mitigating it, published by Mike Cassidy on February 3, 2022 on The Effective Altruism Forum.

On the 15 January 2022, the partially submerged Hunga Tonga-Hunga Ha’apai volcano (~<200 metres underwater) erupted explosively, sending a shockwave that lapped the Earth multiple times. It had been erupting since December 2021, but had so far passed with little to no concern. However, the power of the eruption on the 15 January was something we have not witnessed in 21st Century, certainly not with our global earth observations like satellites, which witnessed the resulting shockwaves crossing the Earth, and measured a volcanic ash plume over 30 km high (55 km high max into the mesosphere) spreading over 650 km in just one hour. The explosion itself was equivalent to a 4 -18 megatons of TNT. In an eruption lasting just hours long (perhaps ~6, but sustained and most intense part was ~3 hours long, with initial blast potentially much shorter) the impacts from this eruption were significant:

Initial impacts

Accompanying the eruption a tsunami struck Tonga and the South Pacific, washing away all the houses from islands 70 km away- about 5 people died as far as we know, but 100’s houses have been destroyed damaged or are unliveable

The tsunami also affected coasts 1000’s km away in multiple nations, in two different oceans, with two people dying in Peru.

Ash fall (a few cm thick) covered a huge area of a Tonga, with the majority of the population (approx. 105,000 people) impacted.

Power and communications to the whole of Tonga was cut off when the submarine Tonga Cable was severed by submarine gravity flows from the collapse of the Hunga Tonga Hunga Ha’apai islands.

It estimated that about 60 – 70 percent of livestock-rearing households either had livestock perished, experienced damage to grazing land, or have contaminated water supplies.

These have led to wide ranging and sometimes unexpected cascading impacts:

The tsunami caused an oil tanker to crash, creating a huge oil leak in Peru, causing widespread ecological damage.

Sanitation, sewage and drinking water (e.g. groundwater) are affected in Tonga from seawater and ash infiltration, with 50,000 people affected from lack of drinking water.

Vulnerable people will suffer from respiratory ill health from ash inhalation which may strain healthcare.

Delays to aid provisions as planes could not land, or fly due to lots of ash in the atmosphere and on the runways.

Internet, phone signal and communications will not be fully restored for 4 weeks following the eruption, until a ship comes to replace the submarine cable.

Some initial signs that the stratospheric ozone has been slightly depleted

Ash fall considerably damaged crops. The agricultural sector contributed nearly 14% of Tonga’s GDP in 2015-16 and represented over 65% of exports, so the country will be relying on aid for some time.

Tonga has recorded its first cases of Covid-19 from those dealing with aid in the ports forcing Tonga to go into lockdown for the first time (despite contactless aid protocols to avoid this).

Main risk going forward is the increased risk of infectious diseases because of sewage disruption, water contamination and shortage.

Despite these impacts, the death toll has so far been low, and the worst impacts have been mainly confined to 100s km from eruptions, that being said the impacts on Tonga from this short eruption will be felt for months or longer. But its ferocity for just several hours provides a glimpse into the impacts a much larger eruption could have. On the volcano magnitude scale (which measures the mass of magma erupted – Magnitude = log10(mass in kg) - 7), this eruption ranks fairly low (estimates so far are Mag 5), together with its erupted sul...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Why you should contribute to Giving What We Can as a writer or content creator (opportunities inside!), published by julianhazell on February 3, 2022 on The Effective Altruism Forum.

TL;DR

Good content can reach a lot of people with a relatively low investment of time, effort, and money.

We have a huge potential to grow the number of people aware of effective giving principles through good-quality content.

The kind of content we want to produce can get people to donate more effectively, but also has indirect value by getting more people interested in EA as a whole.

We can build a large audience while maintaining a commitment to excellent research quality.

There are many ways to get involved with our content team. You could:

Apply to write for us on a contract basis.

Submit a blog post. To do so, email [email protected] with a brief pitch about what you’d like to write about, and a bit about yourself. If successful, you’ll be published on our website.

Work with us to summarise your academic research for our audience.

Have a member profile written about you if you’ve taken a pledge.

Use some other creative skill to help us produce something awesome (think illustrations, videos, or infographics).

Reach out to us with that great idea we haven’t even considered.

Intro

I started writing for Giving What We Can back in 2020 as a volunteer, and then joined the team a few months ago to work on our content production. We have plans to massively ramp up our content output in 2022, and we’d like you to be a part of them.

If you’re someone who is a particularly strong writer or creator, you could play a key role in making these plans a success. In this post I'll explain the ‘why’ by outlining the reasons I think content can be extremely impactful, and the ‘how’ by providing clear ways you can get involved.

Why

Content can be impactful

Good content can impact a lot of people.

Our two best-performing blog posts each received around 25,000 unique views in under a year. In addition, our How Rich Am I calculator and our charity recommendations page each receive hundreds of thousands of views each year. This is just conjecture, but I feel confident that the return on investment on pieces like these can be huge relative to the time it takes to write, edit, and promote them.

If each well-performing article causes just one person to either pledge, become highly engaged with EA, or both, the benefits can far exceed the costs. But even thinking solely about the pledge, for example, still shows the large potential for impact that content can have. Our current best guess of a counterfactual pledge's value is $73,000; even if you think the real number is a half, or even a quarter of this, the benefits still seem like they significantly exceed the costs of writing an article or some other form of content. And we think good content can affect far more than just one person in this way.

Sam Harris’s advocacy for us contributed to at least 1,000 people taking either The Pledge or the Trial Pledge, and this is likely an underestimate. There’s also evidence from the 2019 EA Survey that his podcast increased the number of people who became involved with effective altruism. Of course, Sam already has a huge audience, so maybe his successful advocacy is the exception rather than the rule. But my impression is that there are still plenty of people out there who we can reach — by collaborating with other popular figures, creating our own content, or both.

I’m also reasonably confident that we can increase our content output without sacrificing quality, which I will touch on later.

Accessible content is neglected

There is an endless amount of excellent writing on many different EA topics, so why write for Giving What We Can?

My impression is that most of the best work of late tends to focus ...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: The Urgency of Normal: An Exercise in Bounded Distrust, published by Zvi on February 3, 2022 on LessWrong.

[NOTE: This post, except for the introductory section here in brackets, was written continuously as I investigated the situation, then split off from the weekly post. Except for adding this first paragraph to provide context I did not go back and edit based on later observations. You can choose whether to look at the toolkit in question first, or whether to proceed as I did and look at it after. I hope people do a mix of those two things.]

This attack is an excellent example of the rules of bounded distrust, and what happens when the ‘experts’ say something that someone thinks the Narrative should continue to dislike, so they get turned on. What matters isn’t truth, what matters is whether you are being loyal to Narrative, or giving aid and comfort to ‘the enemy’ with wrongfacts.

I wrote that before looking at the toolkit, because I used the rules of Bounded Distrust or anything other than the Twitter thread above. This is a hit job, so one can assume this is the best they’ve got to hit them with. When ‘the first problem’ is that something is ‘being picked up by’ the wrong people, you make your True Objection very clear. It’s that this supports conclusions you don’t like, and you are in pure Soldier Mindset. When your second problem is not an error but warning that ‘when you misrepresent science to further a cause trust suffers’ you doth protest way too much. What you’re saying is ‘look what you made me do, forcing me to misrepresent the situation and forcing me to label you as providing misinformation. This will destroy trust.’

Or more bluntly, ‘Credentialed scientists going against other credentialed scientists will destroy trust in science, so you’re not allowed to do that.’ Or at a minimum, oh boy are we going to use isolated demands for rigor.

The last note, about them being ‘somewhat receptive’ to correcting errors makes it clear that they’re willing to correct such errors (‘somewhat’ being a clear insinuation word), but that this does not matter because accuracy never mattered anyway, only Narrative.

And then the kicker, that seals it all in place and shows exactly how this mindset works: “But this is a Pandora’s Box situation, and it will set us back in our collective goal of getting schools back to normal.”

Advocating for getting things back to normal, you see, is going to delay getting things back to normal, because now the wrong people are calling for a return to normal before we want them to, and we can’t reward this kind of behavior so now we have to punish both you and them by delaying the return to normal until you learn who is boss. Or alternatively, by advocating for this against Narrative while being experts, and going too soon, you’ve poisoned the well for when we actually want to reverse course, since we’ll have to wait a while now to avoid being too embarrassed, which will slow things down.

There’s also a bit of the logic that goes ‘if you go back to normal, that will cause more Covid, thus preventing us from going back to normal, which is everyone’s goal, why do you hate being normal?’ I’m not sure if I’m hoping it’s delusional or not.

And of course, the part where our ‘collective goal’ is a return to normal is a blatant lie.

Level two is to read the Mother Jones post. The headline (which, remember, is always bullshit in these situations, also clickbait) is “These Doctors Wanted to Get Schools Back to Normal. Their Botched Effort May Backfire.”

If you actually wanted to get things back to normal, of course, you wouldn’t be going around highlighting the ways in which a ‘botched effort may backfire.’ You’d be trying to improve the quality of the effort and fix the errors.

If you were worried about ‘damaging trust in science’ and that a given ...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Why CEA has stopped using Net Promoter Score, published by Ben West on February 3, 2022 on The Effective Altruism Forum.

Net Promoter Score is a widely used method for determining consumer satisfaction, asking “How likely is it that you would recommend [brand] to a friend or colleague?” and the response is (usually) a number between 0 and 10. However, instead of an average, the aggregate score is a complex nonlinear function of the results. CEA has moved away from this complex function in favor of just simply taking the arithmetic mean. Briefly, this is because the results don’t replicate, NPS is not empirically supported, it requires larger sample sizes, and it violates survey best practices.

Summary

NPS is widely used, but the research has failed to replicate, even when the replication was using the originally published data set (!).

Measures of satisfaction are more predictive than NPS of outcomes such as firm growth and whether the respondent actually recommends the product to others.

The American Customer Satisfaction Index is an alternative which has stronger empirical grounding, as well as a huge number of publicly available benchmarks. It uses 3 questions, on a 10 point scale, whose scores are averaged and normalized to a 0-100 scale:

What is your overall satisfaction with X?

To what extent has X met your expectations?

How well did X compare with the ideal (type of offering)?

CEA mostly still asks the NPS question, but switched to taking the arithmetic mean of the results. We call this the “likelihood to recommend” (LTR).

More information

NPS was introduced in 2003 with the claim that it was the best predictor of growth across a data set of companies. This data set was small and subject to p-hacking. The raw data has not been published (including, ironically, the pieces the author says should always be published when reporting NPS scores). The original research methodology was:

“We then obtained a purchase history for each person surveyed and asked those people to name specific instances in which they had referred someone else to the company in question. The data allowed us to determine which survey questions had the strongest statistical correlation with repeat purchases or referrals..One question was best for most industries. “How likely is it that you would recommend [company X] to a friend or colleague?” ranked first or second in 11 of the 14 cases studies”

Replication attempts (including ones which reverse engineered the original data set from published scatterplots) have failed to find significant predictive value from NPS. A wide variety of alternative statistical methods exist, some of which have stronger empirical grounding.

Notably, NPS is worse at predicting whether the respondent will actually recommend the product.

Replication attempts find alternate definitions of the NPS scale to be more predictive than the commonly used one, even if the question is kept the same (e.g. using a 7 point scale).

The weird way NPS is calculated means that it requires substantially larger sample sizes.

The NPS question disagrees with commonly accepted best practices in survey design (e.g. using an 11-point scale instead of a 5-point one).

There doesn’t seem to be any particular reason to think that NPS is good, apart from it being widely used.

So if it’s so terrible, why does everyone use it? This Wall Street Journal article implies that it is used precisely because it’s so easy to manipulate: “Out of all the mentions the Journal tracked on earnings calls, no executive has ever said the score declined.”

Further Reading

%20customer%20satisfaction%20and%20loyalty.pdf

http://www.tsisurveys.com/morgan-rego.pdf

%20customer%20satisfaction%20and%20loyalty.pdf

(Note: different sources seem to use slightly different wording and I’m not sure what the “official” wording is bec...