Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: What problems do African-Americans face? An initial investigation using Standpoint Epistemology and Surveys, published by tailcalled on March 12, 2023 on LessWrong.

This post is also available at my substack.

This post started from a bit of a weird place. I was in a Discord chatroom, and someone started complaining that Standpoint Epistemology had been “taken way past its carrying weight”.

I didn’t know much about Standpoint Epistemology, so I asked for various examples and resources about it. The resources she gave me that were written by Standpoint Epistemologists seemed relatively reasonable, and the resources that criticized it seemed to me to either be misrepresenting what Standpoint Epistemologists were saying, or to be criticizing people for something other than excessive Standpoint Epistemology.

At some point I got to the conclusion that in order to evaluate these things, it would really be useful for me to apply some Standpoint Epistemology myself. Specifically, since a lot of the discussion in the Discord server was about black people’s experiences with racism, I thought I should apply Standpoint Epistemology to this. In this post, I want to detail how I went about this, and what my results were, so that others can learn from it, and maybe usefully apply Standpoint Epistemology themselves.

Disclaimer: As you will see, this is not a thorough investigation into what African-Americans want. Rather, it is a brief initial investigation, which suggests places for further investigation and further learning. This is probably more a practical tutorial into how I would apply Standpoint Epistemology than an article on race issues per se.

What is Standpoint Epistemology?

It may be good to think of Standpoint Epistemology as an erisology, i.e. a theory of disagreement. If you observe a disagreement, Standpoint Epistemology provides one possible answer for what that disagreement means and how to handle it.

According to Standpoint Epistemology, people get their opinions and beliefs about the world through their experiences (also called their standpoint). However, a single experience will only reveal part of the world, and so in order to get a more comprehensive perspective, one must combine multiple experiences. In this way the ontology of Standpoint Epistemology heavily resembles rationalist-empiricist epistemologies such as Bayesian Epistemology, which also assert that people get their opinions by accumulating experiences that contain partial information.

One important difference is that whereas rationalists often focus on individual epistemology, such as overcoming biased heuristics or learning to build evidence into theories, Standpoint Epistemology instead focuses on what one can learn from other people’s experiences. There is only one underlying reality, but different people observe different aspects of it. As such, Standpoint Epistemology emphasizes that if someone tells you about something that you haven’t had experience with, you should take this as a learning opportunity, rather than concluding that they must be irrational, biased, or crazy.

This notion that one should listen to and believe what others say does not contradict the mathematical underpinnings of traditional rationalist epistemology such as Bayesian Epistemology. Instead, it can be mathematically proven from the assumptions of Bayesian Epistemology, in a theorem known as Aumann’s Agreement Theorem. However, while Standpoint Epistemology follows from Bayesian Epistemology, I feel like we don’t necessarily see rationalists being as positive towards it as they could be.

In the specific case of racism, one article that the person in the Discord server shared with me as an example of Standpoint Epistemology was The Part about Black Lives Mattering Where White People Shut Up and Listen. This article, take...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: An AI risk argument that resonates with NYTimes readers, published by Julian Bradshaw on March 12, 2023 on LessWrong.

Ezra Klein of NYT put out a surprisingly sympathetic post on AI risk in the Sunday edition. It even quotes Paul Christiano and links back to LessWrong!But what I'm actually here to talk about is the top reader-recommended comment on the article as of Sunday 11pm UTC:

Dwarf Planet

I wonder how many of these AI researchers have children. What Ezra describes here is what I see every day with my teenager. Of course, no one understands teenagers, but that's not what I mean. I taught my daughter to play chess when she was very young. I consider myself a reasonably good player, and for many years (as I was teaching her), I had to hold myself back to let her win enough to gain confidence. But now that she is thirteen, I suddenly discovered that within a span of weeks, I no longer needed to handicap myself. The playing field was level. And then, gradually and then very suddenly, she leapt past my abilities. As with AI, could understand the broad outlines of what she was doing--moving this knight or that rook to gain an advantage--but I had no clue how to defend against these attacks. And worse (for my game, at least), I would fall into traps where I thought I was pursuing a winning hand but was lead into ambush after ambush.

It was very humbling: I had had the upper hand for so long that it became second nature, and then suddenly, I went to losing every game. As parents, we all want our children to surpass us. But with AI, these "summoners" are creating entities whose motives are not human. We seem to be at the cusp of where I was before my daughter overtook me: confident and complacent, not knowing what lay ahead. But, what we don't realize is that very soon we'll begin to lose every game against these AIs. Then, our turn in the sun will be over.

Generally NYT comments on AI risk are either dismissive, or just laden with general anxiety about tech. (Indeed, the second-most recommended comment is deeply dismissive, and the third is generic anxiety/frustration.) There's hopefully something to learn from commentor "Dwarf Planet" in terms of messaging.

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Are there cognitive realms?, published by Tsvi Benson-Tilsen on March 12, 2023 on The AI Alignment Forum.

[Metadata: crossposted from. First completed November 16, 2022. This essay is more like research notes than exposition, so context may be missing, the use of terms may change across essays, and the text may be revised later; only the versions at tsvibt.blogspot.com are definitely up to date.]

Are there unbounded modes of thinking that are systemically, radically distinct from each other in relevant ways?

Note: since I don't know whether "cognitive realms" exist, this essay isn't based on clear examples and is especially speculative.

Realms

Systemically, radically distinct unbounded modes of thinking

The question is, are there different kinds--writ large--of thinking?

To the extent that there are, interpreting the mental content of another mind, especially one with different origins than one's own, may be more fraught than one would assume based on experience with minds that have similar origins to one's own mind.

Are there unbounded modes of thinking that are systemically, radically distinct from each other?

"Unbounded" means that there aren't bounds on how far the thinking can go, how much it can understand, what domains it can become effective in, what goals it can achieve if they are possible.

"Systemically" ("system" = "together-standing-things") means that the question is about all the elements that participate in the thinking, as they covary / coadapt / combine / interoperate / provide context for each other.

"Radical" (Wiktionary) does not mean "extreme". It comes from the same etymon as "radish" and "radix" and means "of the root" or "to the root"; compare "eradicate" = "out-root" = "pull out all the way to the root", and more distantly through PIE wréh₂ds the Germanic "wort" and "root". Here it means that the question isn't about some mental content in the foreground against a fixed background; the question asks about the background too, the whole system of thinking to its root, to its ongoing source and to what will shape it as it expands into new domains.

Terms

Such a mode of thinking could be called a "realm". A cognitive realm is an overarching, underlying, systemic, total, architectural thoughtform that's worth discussing separately from other thoughtforms. A realm is supposed to be objective, a single metaphorical place where multiple different minds or agents could find themselves.

Other words:

systemic thoughtform

system of thought, system of thinking

cognitive style

state of mind

cluster / region in mindspace

mode of being

species of thinking

Realm vs. domain

A domain is a type of task, or a type of environment. A realm, on the other hand, is a systemic type of thinking; it's about the mind, not the task.

For the idea of a domain see Yudkowsky's definition of intelligence as efficient cross-domain optimization power. Compare also domain-specific programming languages, and the domain of discourse of a logical system.

It might be more suitable for a mind to dwell in different realms depending on what domain it's operating in, and this may be a many-to-many mapping. Compare:

The mapping from computational subsystems to cognitive talents is many-to-many, and the mapping from cognitive talents plus acquired expertise to domain competencies is also many-to-many, [...].

From "Levels of Organization in General Intelligence", Yudkowsky (2007).

Domains are about the things being dealt with; it's a Cartesian concept (though it allows for abstraction and reflection, e.g. Pearlian causality is a domain and reprogramming oneself is a domain). Realms are about the thing doing the dealing-with.

Realm vs. micro-realm

A micro-realm is a realm except that it's not unbounded. It's similar to a cognitive faculty, and similar to a very abstract domain, but includes t...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Bill prohibiting the use of QALYs in US healthcare decisions?, published by gordoni on March 12, 2023 on The Effective Altruism Forum.

Is anyone familiar with H.R. 485? It has been introduced in the House, but it is not yet law.

According to the CRS "This bill prohibits all federal health care programs, including the Federal Employees Health Benefits Program, and federally funded state health care programs (e.g., Medicaid) from using prices that are based on quality-adjusted life years (i.e., measures that discount the value of a life based on disability) to determine relevant thresholds for coverage, reimbursements, or incentive programs".

I think the motivation might be to prevent discrimination against people with disabilities, but it seems to me like it goes too far.

It seems to me it would prevent the use of QALYs for making decisions such as is a particular cure for blindness worthwhile, and how might it compare to treatments for other diseases and conditions.

Is anyone familiar with this bill and able to shed more light on it?

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: "Liquidity" vs "solvency" in bank runs (and some notes on Silicon Valley Bank), published by rossry on March 12, 2023 on LessWrong.

Epistemic status: Reference post, then some evidenced speculation about emerging current events (as of 2023-03-12).

A "liquidity" crisis

There's one kind of "bank run" where the story, in stylized terms, starts like this:

A bank opens up and offers 4%/ann interest on customer deposits.

100 people each deposit $75 to the bank.

The bank uses $7,500 to buy government debt that will pay back $10,000 in five years (let's call this "$10,000-par of Treasury notes", and call that a 5%/ann interest rate for simplicity). We're going to assume for this entire post that government debt never defaults and everyone knows that and assumes it never defaults.

The thing you hope will happen is for every depositor to leave their money for five years, at which point you'll repay them $95 each and keep $500—which is needed to run the bank.

Instead, the next week, one customer withdraws their deposit; the bank sells $100-par of T-notes for $75, and gives them $75 back. No problem.

A second customer withdraws their deposit; oops, the best price the bank can get for $100-par of T-notes, right now after it just sold a bit, is $74. Problem.

But next week, let's say, it would be possible to sell another $100-par for $75 again.

At this point, the simplified bank is stuck. If it sells ~$101-par of T-notes to return the $75 deposit, it won't have enough to pay everyone else back, even if the withdrawals stop here! But if it doesn't give the depositor back $75 right now, then bad things will start to happen.

Equity capital: A liquidity solution

So, we fix this problem by going back in time and starting with an extra step that's now required by law:

Before taking $7,500 of deposits, the bank has to raise 10% of that—so, $750—of what we'll call "equity capital". Equity capital will get used to fill the gap between asset sales and returned deposits

Now, the final step of the original story goes differently:

$1 of equity capital, plus the $74 from the T-notes sale, go to repaying the withdrawn deposit.

Now the bank has 98$75 of deposits, and $749 of equity capital. If nothing happens until next week (when the T-note price will go back to $75), everything will be fine. (In fact, the bank now has 10.19% of deposits in equity capital, making it safer then before.)

A third customer withdrawal forces the bank to sell another $100-par of T-notes at $73, and use $2 of equity capital to repay the deposit. Now the bank has $747 of equity capital, 97$75 of deposits, and a equity-to-deposits ratio of 10.27%.

A fourth customer withdrawal forces the bank to sell another $100-par of T-notes at $72, and use $3 of equity capital to repay the deposit. Now the bank has $744 of equity capital, 96$75 of deposits, and a equity ratio of 10.33%. Even as the withdrawals force the bank to sell T-notes for greater and greater losses (relative to the $75 that the price will go back to next week), the equity ratio stays above 10%.

Until...

The fourteenth customer withdrawal forces the bank to sell another $100-par of T-notes at $62, and use $13 of equity capital to repay the deposit. Now the bank has $659 of equity capital, 86$75 of deposits, and a equity ratio of 10.22%.

The fifteenth customer withdrawal forces the bank to sell another $100-par of T-notes at $61, and use $14 of equity capital to repay the deposit. Now the bank has $645 of equity capital, 85$75 of deposits, and a equity ratio of 10.12%.

The sixteenth customer withdrawal forces the bank to sell another $100-par of T-notes at $60, and use $15 of equity capital to repay the deposit. Now the bank has $630 of equity capital, 84$75 of deposits, and a equity ratio of 10.0%.

...and here is where the oops happens. Still, we're much better th...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Two directions for research on forecasting and decision making, published by Paal Fredrik Skjørten Kvarberg on March 11, 2023 on The Effective Altruism Forum.

An assessment of methods to improve individual and institutional decision-making and some ideas for further research

Forecasting tournaments have shown that a set of methods for good judgement can be used by organisations to reliably improve the accuracy of individual and group forecasts on a range of questions in several domains. However, such methods are not widely used by individuals, teams or institutions in practical decision making.

In what follows, I review findings from forecasting tournaments and some other relevant studies. In light of this research, I identify a set of methods that can be used to improve the accuracy of individuals, teams, or organisations. I then note some limitations of our knowledge of methods for good judgement and identify two obstacles to the wide adoption of these methods to practical decision-making. The two obstacles are

Costs. Methods for good judgement can be time-consuming and complicated to use in practical decision-making, and it is unclear how much so. Decision-makers don't know if the gains in accuracy of adopting particular methods outweigh the costs because they don't know the costs.

Relevance. Rigorous forecasting questions are not always relevant to the decisions at hand, and it is not always clear to decision-makers if and when they can connect rigorous forecasting questions to important decisions.

I look at projects and initiatives to overcome the obstacles, and note two directions for research on forecasting and decision-making that seem particularly promising to me. They are

Expected value assessments. Research into the costs of applying specific epistemic methods in decision-making, and assessments of the expected value of applying those practices in various decision-making contexts on various domains (including other values than accuracy). Also development of practices and tools to reduce costs.

Quantitative models of relevance and reasoning. Research into ways of modelling the relevance of rigorous forecasting questions to the truth of decision-relevant propositions quantitatively through formal Bayesian networks.

After I have introduced these areas of research, I describe how I think that new knowledge on these topics can lead to improvements in the decision-making of individuals and groups.

This line of reasoning is inherent in a lot of research that is going on right now, but I still think that research on these topics is neglected. I hope that this text can help to clarify some important research questions and to make it easier for others to orient themselves on forecasting and decision-making. I have added detailed footnotes with references to further literature on most ideas I touch on below.

In the future I intend to use the framework developed here to make a series of precise claims about the costs and effects of specific epistemic methods. Most of the claims below are not rigorous enough to be true or false, although some of them might be. Please let me know if any of these claims are incorrect or misleading, or if there is some research that I have missed.

Forecasting tournaments

In a range of domains, such as law, finance, philanthropy, and geopolitical forecasting, the judgments of experts vary a lot, i.e. they are noisy, even in similar and identical cases.In a study on geopolitical forecasting by the renowned decision psychologist Philip Tetlock, seasoned political experts had trouble outperforming “dart-tossing chimpanzees”—random guesses—when it came to predicting global events. Non-experts, eg. “attentive readers of the New York Times” who were curious and open-minded, outperformed the experts, who tended to be overconfident.

In a series of...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Paper Replication Walkthrough: Reverse-Engineering Modular Addition, published by Neel Nanda on March 12, 2023 on The AI Alignment Forum.

I'm excited about trying different formats for mechanistic interpretability education! I've made a video walkthrough where we replicate my paper, Progress Measures for Grokking via Mechanistic Interpretability. With Jess Smith, one of my co-authors, we record ourselves coding a replication and discussed what we did at each step. This is a three part walkthrough and you can see the accompanying code for the walkthrough here:

In part 1, we train a model to perform modular addition, and see that it does grok!

In part 2, we take this model and reverse-engineer the trig-based circuit it's learned to do modular addition. We show that you can both read out intermediate steps of the circuit from the activations, and that you can just read off some of the algorithm's steps from the model weights.

In part 3, we define some progress measures that let us distinguish progress towards the generalising and the memorising algorithm. We then look at the model during training and watch how the circuits develop, and use this to understand why it groks.

This is an experiment with a new format, and I'd love to hear about how useful you find it!

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: "You'll Never Persuade People Like That", published by Zack M Davis on March 12, 2023 on LessWrong.

Sometimes, when someone is arguing for some proposition, their interlocutor will reply that the speaker's choice of arguments or tone wouldn't be effective at persuading some third party.

This would seem to be an odd change of topic. If I was arguing for this-and-such proposition, and my interlocutor isn't, themselves, convinced by my arguments, it makes sense for them to reply about why they, personally, aren't convinced. Why is it relevant whether I would convince some third party that isn't here?

What's going on in this kind of situation? Why would someone think "You'll never persuade people like that" was a relevant reply?

"Because people aren't truthseeking and treat arguments as soldiers" doesn't seem like an adequate explanation by itself. It's true, but it's not specific enough: what particularly makes appeal-to-persuading-third-parties an effective "soldier"?

The bargaining model of war attempts to explain why wars are fought—and not fought; even the bitterest enemies often prefer to grudgingly make peace with each other rather than continue to fight.

That's because war is costly. If I estimate that by continuing to wage war, there's a 60% chance my armies will hold a desirable piece of territory, I can achieve my war objectives equally well in expectation—while saving a lot of money and human lives—by instead signing a peace treaty that divides the territory with the enemy 60/40.

If the enemy will agree to that, of course. The enemy has their own forecast probabilities and their own war objectives. There's usually a range of possible treaties that both combatants will prefer to fighting, but the parties need to negotiate to select a particular treaty, because there's typically no uniquely obvious "fair" treaty—similar to how a buyer and seller need to negotiate a price for a rare and expensive item for which there is no uniquely obvious "fair" price.

If war is bargaining, and arguments are soldiers, then debate is negotiation: the same game-theoretic structure shines through armies fighting over the borders on the world's political map, buyer and seller haggling over contract items, and debaters arguing over the beliefs on Society's shared map. Strong arguments, like a strong battalion, make it less tenable for the adversary to maintain their current position.

Unfortunately, the theory of interdependent decision is ... subtle. Although recent work points toward the outlines of a more elegant theory with fewer pathologies, the classical understanding of negotiation often recommends "rationally irrational" tactics in which an agent handicaps its own capabilities in order to extract concessions from a counterparty: for example, in the deadly game of chicken, if I visibly throw away my steering wheel, oncoming cars are forced to swerve for me in order to avoid a crash, but if the oncoming drivers have already blindfolded themselves, they wouldn't be able to see me throw away my steering wheel, and I am forced to swerve for them.

Thomas Schelling teaches us that one such tactic is to move the locus of the negotiation elsewhere, onto some third party who has less of an incentive to concede or is less able to be communicated with. For example, if business purchases over $500 have to be approved by my hard-to-reach boss, an impatient seller of an item that ordinarily goes for $600 might be persuaded to give me a discount.

And that's what explains the attractiveness of the appeal-to-persuading-third-parties. What "You'll never persuade people like that" really means is, "You are starting to persuade me against my will, and I'm laundering my cognitive dissonance by asserting that you actually need to persuade someone else who isn't here." When someone is desperate enou...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Parasitic Language Games: maintaining ambiguity to hide conflict while burning the commons, published by Hazard on March 12, 2023 on LessWrong.

“They are playing a game. They are playing at not playing a game. If I show them I see they are, I shall break the rules and they will punish me. I must play their game, of not seeing I see the game”- R. D. Laing

"It's not lying if everyone knows it's lying."

I see this sentiment in a lot of places. It pops up in corporate managerial contexts. It's been used as a legal defense and worked. It's a claim that communication that looks adversarial isn't, it's just high-context communication between people "in the know", there's no deception happening, no conflict, you just don't get how we do things here.

I don't buy it. My claim in a nutshell:

It situations where people insist "it's not lying because everyone knows it's lying" the people in the know aren't deceiving each other, but the reason this game is being played is to fool people not in the know, and insisting that it's just "high context communication" is part of an effort to obscure the fact that a conflict is going on.

If that makes perfect sense to you, dope, you already get my main point. The rest of this post is adding nuance, actually arguing the case, and providing more language for talking about these sorts of dynamics.

Case Study: "Are Founders Allowed to Lie?"

This essay by Alex Danco talks about how "it's not lying because everybody knows it's lying" works in the Silicon Valley startup scene. It's short enough that it's worth reading now so you can decide for yourself if I'm misrepresenting him. If you don't feel like reading it I still quote enough of it for my post to make sense.

Some snippets.

It's really hard to start a business without lying:

If you are only allowed to tell the literal, complete truth, and you’re compelled to tell that truth at all times, it is very difficult to create something out of nothing. You probably don’t call it “lying”, but founders have to will an unlikely future into existence. To build confidence in everyone around you – investors, customers, employees, partners – sometimes you have to paint a picture of how unstoppable you are, or how your duct tape and Mechanical Turk tech stack is scaling beautifully, or tell a few “pre-truths” about your progress. Hey, it will be true, we’re almost there, let’s just say it’s done, it will be soon enough.

It's not lying because everyone's in on it.

You’re not misleading investors; your investors get it: they’re optimizing for authenticity over ‘fact-fulness’. It’s not fraud. It’s just jump starting a battery, that’s all.

Some abstracted examples of what this "pre-truth" looks like:

You’ve all seen this. It doesn’t look like much; the overly optimistic promises, the “our tech is scaling nicely” head fakes, the logo pages of enterprise customers (whose actual contract status might be somewhat questionable), maybe some slightly fudged licenses to sell insurance in the state of California. It’s not so different from Gates and Allen starting Microsoft with a bit of misdirection. It comes true in time; by the next round, for sure.

Why it's important and also why you can't talk about it:

Founders will present you with something pre-true, under the total insistence that it’s really true; and in exchange, everyone around them will experience the genuine emotion necessary to make the project real. Neither party acknowledges the bargain, or else the magic is ruined.

Before investigating if Danco's story checks out I'm going to introduce some frames for talking about communication to make it easier for me to clarify what's going on here.

Context & Language Games

All communication relies on context and context has a nested structure which operates on multiple levels of communication. Some context operate...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: [Linkpost] Scott Alexander reacts to OpenAI's latest post, published by Akash on March 11, 2023 on The Effective Altruism Forum.

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.