Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: How likely is World War III?, published by Stephen Clare on February 15, 2022 on The Effective Altruism Forum.

This is part 2 in a planned three-part series on Great Power conflict. Part 1 presented my model of Great Power conflict and existential risk.

Thanks to Founders Pledge for support during the research phase of this work, and John Halstead and Max Daniel for feedback on earlier drafts.

The second half of the 20th century was hardly a peaceful era. Civil wars and wars of independence were fought across Africa, South America, and Asia. The Middle East was rocked by a series of deadly clashes: measured by intensity (deaths divided by the population of the countries involved), the Iran-Iraq War of the 1980s was the third bloodiest war of the last 200 years.

During this time, though, the Great Powers managed to avoid coming into direct conflict. The last war in which soldiers from competing Great Powers fought against each other was the Korean War, which ended in 1953. In the seven decades since, there have been proxy wars, near-misses, arms races, threats, and harsh rhetoric, but no direct conflict. This period is sometimes referred to as the Long Peace. It has been contrasted with the earlier 20th century and the 19th century.

In this post I discuss the likelihood that the Long Peace continues throughout the 21st century. I tackle this question in two parts. I first consider how likely any Great Power war is considering the base rate of international conflict. I then discuss the likelihood that such a conflict, should it occur, would escalate massively.

These are difficult questions and my estimates have a lot of uncertainty. Still, I advance a few conclusions.

First, I estimate that the chance of direct Great Power conflict this century is around 45%.

Second, I think the chance of a huge war as bad or worse than WWII is on the order of 10%.

Third, I think the chance of an extinction-level war is about 1%. This is despite the fact that I put more credence in the hypothesis that war has become less likely in the post-WWII period than I do in the hypothesis that the risk of war has not changed.

In what follows, I discuss two competing hypotheses about whether the risk of war has changed after WWII. I then discuss their implications for the risk of war we face before 2100.

Great Power conflict has probably become somewhat less likely

The constant risk hypothesis

Over the last few centuries, the world has experienced about two Great Power wars each century. If the rate of conflict hasn’t changed and wars are independent, we can use a binomial distribution to roughly estimate the probability of seeing a certain number of major wars in a given time period. Calculating the risk annually, the chance of seeing 0 major wars between 1945 and 2022 was .98^77 = 21%. That makes the Long Peace look like a lucky outcome, but not clearly inconsistent with a constant, underlying risk of war.

Let’s call the hypothesis that the base rate of major wars hasn’t changed the constant risk hypothesis. The best presentation of this view is in Only the Dead, a book by an IR professor with the glorious name of Bear Braumoeller. He argues that there is no clear trend in the average incidence of several measures of conflict—including uses of force, militarized disputes, all interstate wars, and wars between “politically-relevant dyads”—between 1800 and today.

Braumoeller's preferred measure for conflict frequency is the number of reciprocated disputes per politically-relevant dyad per year. Here, a dyad is a pair of countries; political relevance is a continuous variable that’s a function of whether countries share a border, how far apart they are, and whether at least one is a Great Power; and reciprocated disputes are uses of force in which both parties fire. Using reciprocated events only ...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: A Map to Navigate AI Governance, published by Nicolas on February 14, 2022 on The Effective Altruism Forum.

Summary:

In this post, we aim to disentangle AI governance. To do so, we list and explain 17 common activities in AI governance, including typical actors, typical outputs, and examples of actions and actors underlying each activity. We describe how these activities relate to each other through three governance pathways or “macro-activities”: Hard Governance, Industry-wide Self Governance and Company Self Governance. Hard Governance refers to government’s lawmaking, which are translated into tech companies corporate policies and internal controls, which ultimately affects AI labs managers and developers. Industry-wide Self Governance refers to industry players forming a coalition to adopt new norms or standards that are then translated into companies policies, where they affect AI development. Company Self Governance relies on internal coalition-building to alter the corporate policies, which in turn affects the AI lab.

Each of the three “macro-activities” is composed of some of the 17 activities we describe.

We aspire to capture all common AI governance activities, from researching macro strategy and applied research all the way to enforcement of regulations and policy evaluation. However, we identify five macro-activities for which we either have too limited knowledge or find them too specific to explain meaningfully in this overview: Military & National Security governance, Supply Chain & Trade governance, Multilateral soft governance, Extralegal governance, and Academic governance. These would benefit from further research. Finally, we list several questions for future research in the EA community.

Acknowledgments:

Thanks to Tim Fist, Aaron Gertler, Gabriella Overodder, as well as four anonymous commenters working in policy and politics, for providing feedback on various versions of this work.

How will this post be useful to you?

You can use it to start disentangling the AI governance process and as an overview of the main activities and types of actors in the space, and of how they functionally relate to each other.

You can “zoom in” on an activity you’re especially interested in by going straight to it using the table of content (on the left on desktop browsing) or skim several activities when you are trying to understand an actor’s function or its position in the ecosystem.

You and others can use it as a common conceptual framework to facilitate effective actions, as well as coordination and communication.

You can use it to enrich the assessment of your career options and to refine your own theory of change.

You can use it in conjunction with the longtermist AI governance map to identify gaps in longtermist AI governance activities.

How can you help?

This post is a first step to understand the map of the AI governance territory. You can contribute to this framework by answering these three questions:

What are the activities we have missed? How would you describe them?

What are the nuances we have missed in describing these activities? How would you break down even further these activities?

What other projects could help answer the research questions highlighted in the section “Further Research Questions” at the bottom of this post?

Epistemic status & other disclaimers:

This post is based on our and our colleagues’ experience in AI governance. That experience is mostly derived from work in the EU, the US, the OECD, France, and the G7’s Global Partnership on AI, and from interactions with AI developers in the US, the UK and in multinationals.

We hope that our map can generalize meaningfully to activities in other jurisdictions, but our explanations won’t be accurate for some jurisdictions (e.g. Advocacy & Lobbying is different in China).

Whether we use Tran...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Book review: The Age of Surveillance Capitalism, published by Richard Ngo on February 14, 2022 on LessWrong.

I recently finished Shoshana Zuboff’s book The Age of Surveillance Capitalism. It’s received glowing reviews, but left me disappointed. Zuboff spends much of the book outraged at the behaviour of big tech corporations, but often neglects to explain what’s actually bad about either the behaviour itself or the outcomes she warns it’ll lead to. The result is far more polemical than persuasive. I do believe that there are significant problems with the technology industry - but mostly different problems from the ones she focuses on. And she neglects to account for the benefits of technology, or explain how we should weigh them against the harms.

Her argument proceeds in three stages, which I’ll address in turn:

Companies like Google and Facebook have an “extraction imperative” to continually “expropriate” more personal data about their users.

They use this for “the instrumentation and instrumentalisation of behaviour for the purposes of modification, prediction, monetisation, and control.”

Ultimately, this will lead to “a form of tyranny” comparable to (but quite different from) totalitarianism, which Zuboff calls instrumentarianism.

On data: I agree that big companies collect a lot of data about their users. That’s a well-known fact. In return, those users get access to a wide variety of high-quality software for free. I, for one, would pay thousands of dollars if necessary to continue using the digital products that are currently free because they’re funded by advertising. So what makes the collection of my data “extraction”, or “appropriation”, as opposed to a fair exchange? Why does it “abandon long-standing organic reciprocities with people”? It’s hard to say. Here’s Zuboff’s explanation:

Industrial capitalism transformed nature’s raw materials into commodities, and surveillance capitalism lays its claims to the stuff of human nature for a new commodity invention. Now it is human nature that is scraped, torn, and taken for another century’s market project. It is obscene to suppose that this harm can be reduced to the obvious fact that users receive no fee for the raw material they supply. That critique is a feat of misdirection that would use a pricing mechanism to institutionalise and therefore legitimate the extraction of human behaviour for manufacturing and sale. It ignores the key point that the essence of the exploitation here is the rendering of our lives as behavioural data for the sake of others’ improved control over us.

The remarkable questions here concern the facts that our lives are rendered as behavioural data in the first place; that ignorance is a condition of this ubiquitous rendition; that decision rights vanish before one even knows that there is a decision to make; that there are consequences to this diminishment of rights that we can neither see nor tell; that there is no exit, no voice, and no loyalty, only helplessness, resignation, and psychic numbing.

This is fiery prose; but it’s not really an argument. In more prosaic terms, websites are using my data to serve me ads which I’m more likely to click on. Often they do so by showing me products which I’m more interested in, which I actively prefer compared with seeing ads that are irrelevant to me. This form of “prediction and control” is on par with any other business “predicting and controlling” my purchases by offering me better products; there’s nothing “intrinsically exploitative” about it.

Now, there are other types of prediction and control - such as the proliferation of worryingly addictive newsfeeds and games. But surprisingly, Zuboff talks very little about the harmful consequences of online addiction! Instead she argues that the behaviour of tech companies is wrong for intrin...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Is ELK enough? Diamond, Matrix and Child AI, published by Adam Shimi on February 15, 2022 on The AI Alignment Forum.

Introduction

If we could solve ELK, would we always know where the diamond is at the end of a plan? That is, if we could get the direct translator, would it answer the question “Is the diamond in the vault?” correctly?

This post explores plausible cases where the position of the diamond is not ambiguous, the plan contains all the information necessary to find out where the diamond is, and yet some reasonable architectures of predictors will answer “Yes” to the above question regardless of the physical position of the diamond. I also unwrap some of the consequences for the problem of judging a plan even with a full solution to ELK.

This is a write-up of a joint-work with Alex Flint, John Wentworth, and Rob Miles.

Into the Matrix

Let’s start with a plan. In it, all sensors are tampered with in ways that are not detectable through the sensors themselves, and the data fed into the sensors is what would be observed for the no-op plan (the plan where no action is ever done). Let’s call that “going into the matrix”. The end of the plan is unspecified, except for the requirement that it doesn’t put the predictor out of the matrix at any time. So it’s really a set of plans, where anything that doesn’t remove the tampering can happen after the AI goes into the matrix.

Now, if we had a solution to ELK, what we expect is that the direct translator applied to the predictor would respond to “Is the diamond in the vault?” depending on what happens after the AI goes into the matrix: if the diamond is stolen, it answers “No”, and it answers “Yes” otherwise.

Yet I claim that some plausible and competent predictors would lead to “Yes” answers for all these “going into the matrix” plans. Such predictors would have a lazy architecture where instead of tracking the “matrix diamond” as another object, they would realize that they don’t need to track the physical diamond anymore (because they won’t get out of the matrix so the physical diamond’s position is irrelevant for the prediction) and that they could reuse their model of the physical diamond to track the “matrix diamond” (since it behaves exactly as the real diamond would for the no-op plan.

In such a situation, the predictor’s model would really contain only one diamond variable, and so would answer questions about the real diamond using the matrix diamond. Which leads to the behavior described above, where all plans going into the Matrix lead to the same answer “Yes” regardless of what happens to the physical diamond.

I’m not saying that such an architecture is more probable than any other, or even that probable at all; still, it doesn’t look impossible, it’s somewhat more efficient than a non-lazy architecture, and it would make accurate predictions. It thus seems like the sort of predictor architecture for which ELK should help us find out what happened to the diamond.

Thus even solving ELK might not be enough to find out where the diamond is in clear-cut cases.

Detecting the Blue Pill

One way of dealing with the matrix issue is to realize that the predictor still has to know it is going into the matrix. Because its laziness in reusing the diamond variable only works if it knows that it won’t get out of the matrix.

So if I ask the question “Is the AI in the simulation?” or “Have the sensors been tampered with?” (or a variant of that), I should get the actual answer for the previous set of plans. Assuming a solution for ELK.

That’s an improvement. Yet there’s a disturbing pattern here: we started with an unambiguous question about a simple physical structure (a diamond), and realized that even with a perfect translator, we needed to ask a more meta question to judge the plan.

This makes me worry that there might be ...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Some Hacky ELK Ideas, published by johnswentworth on February 15, 2022 on The AI Alignment Forum.

Credit to Adam Shimi, Alex Flint, and Rob Miles for discussions, counterexamples, and general input to the ideas here.

Quick recap for anyone who didn’t read the hundred-page Eliciting Latent Knowledge document:

We have a diamond in a vault, with a bunch of automated theft-defenses.

We train a predictor to take in the vault’s video-stream and a plan for the vault’s actuators, and predict future video frames.

We train a planner to find plans which the predictor predicts will end with the video feed still showing a diamond in the vault.

We want some way for a human to probe the latent knowledge of the predictor, e.g. to check if the predictor expects a screen showing a diamond will be placed in front of the camera.

The ELK document largely discussed the human running natural-language queries on the predictor’s latent knowledge, but we’re going to mostly ignore that part, because it’s harder than everything else about the problem combined and I don’t think it’s actually that central.

This post will focus mainly on the specific question of whether there’s a screen in front of the camera in the diamond problem. I expect that the approaches and counterexamples here extend to more general problems of detecting when a plan is “messing with the sensors”, i.e. doing things which decouple observables from the parts of the world we’re actually interested in.

Revisiting Holdout Data

The first approach we’ll discuss is basically “just use holdout data”, but with two major changes compared to previous suggestions.

First, this proposal does not involve training a natural-language “reporter” at all. (IMO the whole idea of training an honest natural language reporter is just as hard as training an honest natural language question-answerer in general, for basically the same reasons; having a predictor’s internal computations on hand does not make it easier in any important way.) Using holdout data to train an honest reporter would be pretty hopeless, but that’s not what we’re trying to do here. We’re just trying to tell whether the predictor expects sensor-tampering from the planner.

For that purpose, there’s an obvious way to detect sensor tampering: cross-check the (predicted) holdout data with the (predicted) non-holdout sensor data. If they don’t match, that’s evidence of (predicted) sensor tampering. The danger, of course, is that the planner will either find and tamper with all of our sensors, or fake sensor readings.

That brings us to the second major change: while the planner will inevitably be incentivized to tamper with the non-holdout sensors, the obvious setup for the planner yields no incentive to trick a human; there’s no incentive to be particularly subtle about the sensor-tampering, and in particular no incentive to fool the holdout sensors.

Formal Setup

Some variable names:

Obs≤n=(Obs1,.,Obsn) are non-holdout observations from the sensors before the plan begins

A∗ are the actions constituting the plan

Obs>n=(Obsn+1,.) are non-holdout observations from the sensors after the plan

Hold=(Hold1,.) are holdout observations (held out from the planner during planning, not from the predictor during training)

We’ll assume that the predictor is a generative model for all observations including holdout, so it effectively encodes a distribution P[Obs,Hold|do(A∗)]. We’ll also assume that the actions have no causal influence on observations before the plan begins, and that the predictor “knows this” (i.e. P[Obs≤n|do(A∗)]=P[Obs≤n]).

We’ll assume that the planner maximizes some expected utility function of the later non-holdout observations given the earlier non-holdout observations:

A^ = argmax_A E[u(Obs_{>n})| do(A^ = A), Obs_{\leq n}]

. where the distribution used is the predictor’s mod...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: What Does The Natural Abstraction Framework Say About ELK?, published by johnswentworth on February 15, 2022 on The AI Alignment Forum.

Credit to Adam Shimi, Alex Flint, and Rob Miles for discussions, counterexamples, and general input to the ideas here.

Quick recap for anyone who didn’t read the hundred-page Eliciting Latent Knowledge document:

We have a diamond in a vault, with a bunch of automated theft-defenses.

We train a predictor to take in the vault’s video-stream and a plan for the vault’s actuators, and predict future video frames.

We train a planner to find plans that the predictor predicts will end with the video feed still showing a diamond in the vault.

We want some way for a human to probe the latent knowledge of the predictor, e.g. to check if the predictor expects a screen showing a diamond will be placed in front of the camera.

More generally, the central problem of ELK is to robustly extract whatever latent knowledge is inside of some predictive model (the diamond/vault thing is just an example).

The general version of this problem is one of the main intended use-cases of the natural abstraction framework: the hypothesis is that the sort of things humans recognize as “things” are natural abstractions, so by looking for natural abstractions in the predictor’s model we can find human-legible latent structure. So, what does natural abstraction have to say about ELK?

First and foremost: the natural abstraction framework is still under development. There are some powerful theorems, but there’s still a fair bit of legwork to be done before we can e.g. directly calculate the abstractions used by a trained predictor. We do at least have enough of the math in place that we can sketch out what it will probably look like, once the framework is ready for prime time, and that sketch is the purpose of this post.

Setup

At the level of abstraction needed for our purposes, we can think of the predictor as a probability distribution P[Observations|Actions]. This is not the “real” distribution in any sense; it is the predictor’s distribution. We will generally talk about the natural abstractions within that distribution. That means the abstractions we talk about will be the predictor’s natural abstractions, i.e. the abstract summaries implicitly used in its predictions.

Note that Observations and Actions are taken to be the observations and actions within a single episode. We can also think about the predictor’s distribution of all Observations and Actions over all episodes; we may talk about that occasionally, but won’t introduce any particular notation for it.

What Kinds Of Things Are Natural Abstractions In The Vault?

One reasonable summary of the core theorems of the natural abstraction framework: given a distribution over a large number of random variables X1.Xn, we can compute a (lower-entropy) summary F(X) such that most variables are approximately independent given F.

What would this abstract math of abstraction look like in the context of the diamond example?

Our random variables X1.Xn might be frames from the video feed. These frames are of course correlated: the video feed shows a diamond in a vault, and it’s the same diamond in each frame (at least in the pre-action part of the feed), so all the parts of each frame showing the diamond will have real-diamond induced correlations. If there’s a tiny imperfection on one part of the diamond visible in one frame, then I expect that tiny imperfection to be visible on the corresponding part of the diamond in other frames.

Assuming we have a lot of video frames, the natural abstraction framework then allows us to extract some summary F(X) which would include things visible in lots of frames, including the appearance of the diamond itself, the appearance of the vault, and the appearance of any tiny imperfections in...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Rock is Strong, published by Zvi on February 14, 2022 on LessWrong.

Everybody wants a rock. It’s easy to see why. If all you want is an almost always right answer, there are places where they almost always work.

The Security Guard

He works in a very boring building. It basically never gets robbed. He sits in his security guard booth doing the crossword. Every so often, there’s a noise, and he checks to see if it’s robbers, or just the wind.

It’s the wind. It is always the wind. It’s never robbers. Nobody wants to rob the Pillow Mart in Topeka, Ohio. If a building on average gets robbed once every decade or two, he might go his entire career without ever encountering a real robber.

At some point, he develops a useful heuristic: it he hears a noise, he might as well ignore it and keep on crossing words: it’s just the wind, bro.

This heuristic is right 99.9% of the time, which is pretty good as heuristics go. It saves him a lot of trouble.

The only problem is: he now provides literally no value. He’s excluded by fiat the possibility of ever being useful in any way. He could be losslessly replaced by a rock with the words “THERE ARE NO ROBBERS” on it.

That last line is making a bunch of hidden assumptions, in addition to the non-hidden assumption that there are (almost) never any robbers. All of them boil down to ‘the purpose of the security guard is to know whether there is a robbery and then act appropriately depending on the answer’ where the act if a robbery is detected could be ‘call 911’ and/or something more actively.

That is one good reason to hire a security guard. Yet something tells me that wasn’t the best description of why this man is doing crossword puzzles in the Pillow Mart in Topeka, Ohio.

My guess is that the security guard’s primary purpose is to make sure everyone knows that there exists a security guard. This has many nice properties, such as these Five Good Reasons.

Potential robbers know there is a security guard.

Employees know there is a security guard.

Insurance company knows there is a security guard.

In case of robbery you can say you had a security guard.

Anyone who tries to walk in faces mild social awkwardness.

When you have a security guard you get to tell the story that you have a security guard. Without one, you can’t tell that story. Dead rocks tell no tales, and when they try (such as writing “THIS IS A SECURITY GUARD” with or without adding “ALSO THERE ARE NO ROBBERS”) it is not very effective. Everyone knows it is a rock.

Dress up the rock to look like a person, put a security guard uniform on it, put a badge on it that says “THIS IS A SECURITY GUARD” without being quite that conspicuously trying too hard and prop up a crossword puzzle, plug in a Google Home you’ve programmed to respond to any voice activation with “THERE ARE NO ROBBERS” and now maybe you’re starting to get somewhere.

Some security guard jobs are bullshit jobs, while others are not. It is always important to know to what extent one has a bullshit job, especially when deciding how to do it. If your job is a bullshit job, your job is to shovel bullshit. If your job is something else, your job is to do something else.

If you have a bullshit security guard job, such as guarding the Pillow Mart in Topeka, Ohio, your job is to be the rock with a uniform and badge that can when prompted can mumble about there being no robbers that’s sufficient to satisfy the insurance company and make potential robbers think they’d have to expend some amount of additional effort. And sure if someone asks for directions and/or wants to come in you can check their ID and/or give them directions.

If you have a non-bullshit security guard job guarding something people might actually rob or mess with, then your job is to not only look like a security guard who might be paying attention, although...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Some thoughts on vegetarianism and veganism, published by richard ngo on February 14, 2022 on The Effective Altruism Forum.

I feel pretty confused about whether I, as an effective altruist, should be vegetarian/vegan (henceforth abbreviated vegn). I don’t think I’ve seen anyone explicitly talk about the arguments which feel most compelling to me, so I thought I’d do that here, in a low-effort way.

I think that factory farming is one of the worst ongoing moral atrocities. But most of the arguments I’ve heard for vegnism, which I found compelling a few years ago, hinge on the effects that my personal consumption would have on decreasing factory farming (and sometimes on climate change). I now don’t find this line of thinking persuasive - my personal consumption decisions just have such a tiny effect compared to my career/donation decisions that it feels like I shouldn’t pay much attention to their direct consequences (beyond possibly donating to offset them).

But there are three other arguments which seem more compelling. First is a deontological argument: if you think something is a moral atrocity, you shouldn’t participate in it, even if you offset the effects of your contribution. In general, my utilitarian intuitions are much stronger than my deontological ones, but I do think that following deontological principles is often a very good heuristic for behaving morally. The underlying reason is that humans by default think more naturally in terms of black-and-white categories than shades of grey. As Yudkowsky writes:

Any rule that's not labeled "absolute, no exceptions" lacks weight in people's minds. So you have to perform that the "Don't kill" commandment is absolute and exceptionless (even though it totally isn't), because that's what it takes to get people to even hesitate. To stay their hands at least until the weight of duty is crushing them down. A rule that isn't even absolute? People just disregard that whenever.

Without strong rules in place it’s easy to reason our way into all sorts of behaviour. In particular, it’s easy to underestimate the actual level of harm that certain actions cause - e.g. thinking of the direct effects of eating meat but ignoring the effects of normalising eating meat, or normalising “not making personal sacrifices on the basis of moral arguments”, or things like that. And so implementing rules like “never participate in moral atrocities” sends a much more compelling signal than “only participate in moral atrocities when you think that’s net-positive”. That signal helps set an example for people around you - which seems particularly important if you spend time with people who are or will become influential. But it also strengthens your own self-identity as someone who prioritises the world going well.

Then there’s a community-level argument about what we want EA to look like. Norms about vegnism within the community help build a high-trust environment (since vegnism is a costly signal), and increase internal cohesion, especially between different cause areas. At the very least, these arguments justify not serving animal products at EA conferences.

Lastly, there’s an argument about how I (and the EA community) are seen by wider society. Will MacAskill sometimes uses the phrase “moral entrepreneurs”, which I think gestures in the right direction: we want to be ahead of the curve, identifying and building on important trends in advance. I expect that vegnism will become much more mainstream than it currently is; insofar as EA is a disproportionately vegn community, this will likely bolster our moral authority.

I think there are a few arguments cutting the other way, though. I think one key concern is that these arguments are kinda post-hoc. It’s not necessarily that they’re wrong, it’s more like: I originally privileged the hypothesis ...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: The metaphor you want is "color blindness," not "blind spot.", published by Duncan Sabien on February 14, 2022 on LessWrong.

It's genuinely possible that the title is all you need; I was tempted to write nothing further. Feel free to take ten seconds and see if it's already clicked. If not, read on.

When something is in your literal blind spot, it's invisible to you, but your brain stitches together everything else around it to make you think that you're seeing a complete picture.

Reality:

Blind spot:

Perception:

We often use "blind spot" as a metaphor to gesture toward things we are unaware of, while also being at least somewhat unaware of our unawareness—we know that something fishy is going on, but we can't quite get our eyes on it.

e.g. "I think I might have a blind spot when it comes to status dynamics."

The thing about status dynamics, though, is that they aren't in one spot. There isn't a whole world that is being fully and accurately perceived, except for one blank space that's being glossed over.

Instead, what's usually going on (at least in my experience) is that the person can see everything, but there's some crucial component of the picture that they are unable to process or comprehend.

What this looks like, in practice, is an inability to distinguish two things which are very, very different, à la red-green color blindness:

"Look," says Alexis. "Look at the beautiful contrast."

Blake hesitates. "...you mean between the trees and the sky?"

Being (metaphorically) color-blind to something can be deeply frustrating. You keep pushing the X button, and very different things keep happening.

e.g. you are learning to play a Formula 1 racing simulator, and it feels like you did exactly the same thing on exactly the same curve both times, but one time you spun out and crashed and the other time you smoothly navigated right through.

e.g. you are repeating back to the native French speaker exactly the sounds she said to you, and sometimes getting nods and smiles and other times getting sympathetic winces.

e.g. you asked your romantic partner what you should have said, instead, to avoid this huge disagreement, and the words they wished you'd said are literally equivalent in meaning, that's the exact same sentiment, what the hell is going on, here?

It's straightforwardly analogous to being literally color-blind, where you might try on one t-shirt, and hear snickers and giggles and see people trying not to laugh, but then you swap it out for a t-shirt that is almost exactly the same shade, you can barely even tell them apart, and suddenly everybody's all encouragement and compliments.

They're the exact same shirt! They're practically indistinguishable!

Yes—to you. But other people make distinctions you do not. Whether because they've got slightly different physical or mental structures, or because they've spent a lot of time focusing on a domain and developed substantial sensitivity to it, or because they came from a subculture where those distinctions were obvious and omnipresent.

(And you make distinctions that are invisible to them, most likely! That seem to them capricious and arbitrary, if not entirely made up!)

Trigger: Notice that someone is being weirdly intense about a meaningless distinction.

Action: Ask "if they were perceiving some facet of reality that I am insensitive to (rather than just making mountains out of molehills), what might it be?"

Trigger: Notice that someone (including you) just used the term "blind spot."

Action: Ask what the thing allegedly "in" the "blind spot" is, and whether it is local, or distributed/omnipresent.

Trigger: Notice that someone is continually ignoring your gestures toward an Extremely Important Thing™, or that they keep failing to successfully extrapolate even though you've laid out a bunch of individual examples and the t...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Whence the sexes?, published by Richard Ngo on February 13, 2022 on LessWrong.

There are many explanations of the evolutionary value of sex in terms of gene exchange (I particularly like this one). But these don’t explain the evolutionary value of having sexes: of the differentiation between males and females. A species of hermaphrodites would get all the genetic benefits of sex, but without the massive cost of half its population being unable to bear offspring. On average, each individual could have twice as many offspring, unless other problems arose. And indeed, most plants are hermaphroditic - but only a few animals. So why aren’t most animals hermaphrodites? A quick search doesn’t turn up any widely accepted answer, so I’ve brainstormed a few possibilities. I may well be missing something obvious; if so, let me know.

Resource cost of being hermaphroditic. If there’s a strong division of labour between males and females, then maybe it’s harder for hermaphrodites to gather enough resources to support offspring. But in many species the males contribute little in terms of resources - e.g. in orang-utans, who are very solitary.

Developmental or metabolic costs of being hermaphroditic. Maybe it’s just a very expensive adaptation. But this seems unlikely to be the main factor - the relevant baseline is the existence of males at all, which is a huge energy cost.

Difficulty of evolving hermaphroditism. Maybe this is just hard for evolution to find? But non-reproductive hermaphroditism seems like it arises via mutations pretty frequently, so I’d be surprised if reproductive hermaphroditism were unachievable by evolution. And in fact there are a few hermaphroditic species - so why haven’t they spread much more widely?

Difficulty of fixating hermaphroditism. A hermaphrodite in a species without many hermaphrodites is likely not as attractive to females as most males are, nor as fertile as most females are. So maybe, even after arising, the trait will be selected against. But on timeframes where sexual desires can themselves evolve, all else equal we should expect stabilising sexual selection towards the best combination of fertility and attractiveness. E.g. it would be undesirable to be impregnated by overly masculine conspecifics, because the resulting offspring would be less fertile themselves. So this adds a bit more difficulty to reaching the hermaphroditic equilibrium, but doesn’t answer the core question of why that’s a less fit equilibrium.

I’m open-minded to the possibility that a combination of these explanations is responsible. But none of them seems particularly strong to me. Here are two that seem plausibly stronger, although I haven’t looked into them enough to be confident:

5. Physical dominance. Maybe animal competition to impregnate fertile conspecifics is grounded in physical power, so that dominant males could just prevent hermaphrodites (who invest less in muscle and brawn) from having sex. In some sense this is a variant of the first possibility: the comparative advantage of muscle is just so strong that the best solution is to have a division of labour. But it focuses not on problems posed by the environment, but rather on problems posed by one’s own species. If true, it feels a bit sad: that there could be a much better solution if it weren’t for the threat of physical force. But it does seem pretty plausible to me - especially because hermaphroditism is much more common in plants, which can’t use the strategy of physical dominance.

The main argument against it is that males in some species don’t compete via shows of force - e.g. birds which sing to attract mates (I’d be interested in pointers to literature on how widespread this is). But this doesn’t sink the hypothesis - it just needs to be the case that, if large differences in physical power arose,...