Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Gradient Hacking via Schelling Goals, published by Adam Scherlis on December 28, 2021 on The AI Alignment Forum.

Thanks to several people for useful discussion, including Evan Hubinger, Leo Gao, Beth Barnes, Richard Ngo, William Saunders, Daniel Ziegler, and probably others -- let me know if I'm leaving you out.

See also: Obstacles to Gradient Hacking by Leo Gao.

I want to argue that gradient hacking (strongly enough to create a local minimum) is possible even under somewhat pessimistic assumptions. The specific strategy that achieves this is kind of interesting: it requires the agent to coordinate on a complicated task with versions of itself that have slightly different objectives, and it requires the agent to do this without any direct communication.

A simple gradient-hacking strategy for a mesa-optimizer is to deliberately perform worse on the base objective if your mesa-objective is changed. However, this isn't as straightforward as it sounds.

One difficulty is that if the gradient hacking is hard-coded, gradient descent can just remove the hacking behavior without touching anything else. A nice way around that is to assume that gradient hacking is discovered from first principles during the model's computation. (Thanks to whoever pointed this out to me.) If it's good enough at reasoning, then it will be pretty hard for the base optimizer to modify it in ways that stop it from noticing that gradient hacking would be a good idea right about now.

(For simplicity, I have in mind something like a really deep feed-forward network, possibly with some weights tied between layers. Equivalently, a recurrent net unrolled and trained via SGD. This may not be realistic but it's easier to think about as a test case.)

A bigger problem is that it's not really correct to think of a model (with particular parameter values) as having control over its gradients. It's better to think of it as having control (in an FDT kind of sense) over only the single value of the loss for its actual parameters. The value of the loss at nearby points in parameter space is decided by slightly different versions of itself, and the gradient is sort of decided collectively. This is important if we're talking about SGD steps that change its mesa-objective, because these nearby versions of the model will have directly conflicting incentives when it comes to the gradient. To say "I am such that gradients on nearby mesa-objectives point towards my mesa-objective" is to say "if I had a different mesa-objective, I would deliberately do slightly worse in order to end up with this one instead." But you wouldn't! That would be bad for your mesa-objective!

Another way to look at this is that it's really hard to implement "do worse if my objective changes" if you don't have an external way to tell that your objective used to be different, let alone a way to commit other versions of yourself to things they don't want to do. You can implement rules like "if your mesa-objective isn't X, throw a fit", but the base optimizer will just change X at the same time it changes your actual mesa-objective. (Actually, it doesn't have to: we're assuming that gradient hacking has to be derived from first principles on every forward pass, so X also has to be derived from first principles. But even without that assumption this strategy wouldn't work.)

You can leave messages for yourself, if you have somewhere to jot down durable notes (like the external world), but let's assume you don't.

Maybe your mesa-objective is something nice and clear and crystalline, an island in mesa-objective parameter space, and you can implement the rule "do badly in proportion to how un-crystalline your mesa-objective is". But it's not clear why your less-crystalline neighbors in parameter space would cooperate with this plan.

Well... maybe if they'...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Omicron: My Current Model, published by Zvi on December 28, 2021 on LessWrong.

A year and a half ago, I wrote a post called Covid-19: My Current Model. Since then things have often changed, and we have learned a lot. It seems like high time for a new post of this type.

Note that this post mostly does not justify and explain its statements. I document my thinking, sources and analysis extensively elsewhere, little of this should be new.

This post combines the basic principles from my original post, which mostly still stand, with my core model for Omicron. I’ll summarize and update the first post, then share my current principles for Omicron and how to deal with and think about it.

There’s a lot of different things going on, so this will likely be incomplete, but hopefully it will prove useful. The personally useful executive summary version first.

Omicron has already taken over, most cases are being missed, crunch time is now. Crunch time will likely last 1-2 months.

First two shots don’t protect against infection, boosters do somewhat (60%?).

Vaccination and natural infection protect against severe disease, hospitalization and death (best guess ~80% reduction in death for double vaccination, 95%+ reduction in death for boosters but too soon to know).

Tests work, but when delayed are mostly useless for preventing infection especially when delayed, as Omicron can spread within 1-2 days after exposure. Rapid tests mostly test for infectiousness, not being positive.

Omicron probably milder than Delta (~50%) so baseline IFR likely ~0.3% unless hospitals overload, lower for vaccinated or reinfected.

Being young and healthy is robust protection against severe disease and death, being not that means a lot more risk. Long Covid risk small but real for all age groups, vaccination likely helps a lot.

Medical system is under strain, could be overwhelmed soon, should be better again in a few months at most if it gets bad. Delaying infection has value but stopping it fully is likely not worth the cost. If you care about real prevention, the tools that matter are vaccination, good masks (N95 or even better P100+), social distancing and air ventilation.

Vitamin D and Zinc, and if possible Fluvoxamine, are worth it if you get infected, also Vitamin D is worth taking now anyway (I take 5k IUs/day). Paxlovid is great (~88%) if available right after you test positive, but in very limited supply for now.

Default action on positive test is 5 days isolation at home as per new CDC guidelines, if possible is good to get a negative rapid test before ending isolation. If things get bad, especially if you have trouble breathing, call your doctor, seek treatment and so on.

Here are the old principles that still apply, with adjustments as appropriate:

Risks follow Power Laws. Focus on reducing your biggest risks.

Sacrifices to the Gods are demanded everywhere. Most intervention effort treats Covid-19 as a morality in which the wicked must be punished, rather than aiming physical interventions to achieve physical results.

Governments Most Places Are Lying Liars With No Ability To Plan or Physically Reason. They Can’t Even Stop Interfering and Killing People. There is a War, and the WHO, FDA and CDC, and most similar agencies abroad, and most elected officials, are not on our side of it. Instead they focus mostly on getting in the way, protecting their power and seeking to avoid blame on a two week time horizon.

Silence is Golden. Talking or singing greatly increases infection risk, and the directions people face matter too. You’re still not safe or anything, but it helps.

Surfaces are Mostly Harmless. Mostly don’t worry about them.

Food is Mostly Harmless. Mostly don’t worry about it.

Outdoor Activity Is Relatively Harmless. It’s a huge relative risk reduction.

Masks Are Effective. I’m less excite...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Ten Minutes with Sam Altman, published by lsusr on December 28, 2021 on LessWrong.

"Do you believe in the American Dream?" my brother asked.

I looked out the airplane window. Y-Combinator (YC) had paid for our tickets. Our application had been good enough to qualify us for a ten-minute interview in Silicon Valley. If we passed the interview we'd receive $120k in funding and entry into an exclusive club of visionary tech pioneers.

Like most Americans, we are descended from immigrants. Our ancestors arrived between 50 and 100 years ago. They built a better life for themselves in the New World than was possible in their lands of origin. That's not saying much. Some of them grew up on Asian sugar cane farms. Others were European Jews.

Success is earning more money at a higher status job than your parents. That was always an easy bar to clear. My father was a private in the US Army. I could have made him proud by becoming an officer in the US Army.

But the American Dream is about more than material wealth and legible status. My great-great grandfather designed tanks for Lenin. In Soviet Russia, there used to be something called a Coke party. You saved up money for days to buy a single can of contraband Coca-Cola. You got all of your friends together and poured each of them a single shot. It tasted like freedom.

"Yes. Yes I do," I answered.

Silicon Valley is a land of rainbows and unicorns where anything is possible. Stepping out of the airport, we were blasted with sunshine, palm trees and warm air. We took a train to YC's headquarters.

Everything I knew about Silicon Valley came secondhand. I had read Steve Jobs' book, Steve Wozniak's book and Elon Musk's book. I had read each of Paul Graham's essays countless times. I had read Zero to One and The Soul of a New Machine. I expected to see robots walking around or at least a Google Street View car but San Francisco appeared to be a perfectly ordinary American city except for the good weather.

YC seemed to be located in the boringest part. We walked past suburban house after suburban house until we got to YC's headquarters. We checked in at the front desk and waited for our startup's name to be called.

YC's headquarters is tiny for a company with such a great impact because YC encourages its startups to work in their own spaces. YC's headquarters is mostly just a big room. They fill it with chairs when they're doing presentations and tables when they're doing interviews. The big room was sparsely populated. A handful of teams waited around to be interviewed.

One of the teams had a cool cellphone app. Another team carried around a heavy metal box that looked like a microwave. My brother and I sat down next to a pair of Nigerian bankers.

The Nigerians explained to us how banking in Africa works. You try a credit card. It gets declined. You try another credit card. It gets declined too. You carry around a wallet full of credit cards. If you're lucky one of the credit cards gets accepted and a fee is sent out of Africa to a corporation in the United States. The Nigerians had created a payment system that actually worked. They were making money hand over fist. So many people were joining their system they had to throttle new customer sign-ups.

The Nigerians were self-conscious about their accents; it was the first time they'd been to America. We assured them it didn't matter. These guys were going to own the financial system of an entire continent.

My brother and I looked similar because we're brothers, because we were of similar age and because we were the only half-Asians in the room. The Nigerians looked alike because they wore identical purple sweatshirts, they were of similar age and because they were the only black people in the room. An employee from YC joined our conversation. I don't remember her exact words (I'm sure t...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Democratising Risk - or how EA deals with critics, published by CarlaZoeC on December 28, 2021 on The Effective Altruism Forum.

Luke Kemp and I just published a paper which criticises existential risk for lacking a rigorous and safe methodology:

It could be a promising sign for epistemic health that the critiques of leading voices come from early career researchers within the community. Unfortunately, the creation of this paper has not signalled epistemic health. It has been the most emotionally draining paper we have ever written.

We lost sleep, time, friends, collaborators, and mentors because we disagreed on: whether this work should be published, whether potential EA funders would decide against funding us and the institutions we're affiliated with, and whether the authors whose work we critique would be upset.

We believe that critique is vital to academic progress. Academics should never have to worry about future career prospects just because they might disagree with funders. We take the prominent authors whose work we discuss here to be adults interested in truth and positive impact. Those who believe that this paper is meant as an attack against those scholars have fundamentally misunderstood what this paper is about and what is at stake. The responsibility of finding the right approach to existential risk is overwhelming. This is not a game. Fucking it up could end really badly.

What you see here is version 28. We have had approximately 20 + reviewers, around half of which we sought out as scholars who would be sceptical of our arguments. We believe it is time to accept that many people will disagree with several points we make, regardless of how these are phrased or nuanced. We hope you will voice your disagreement based on the arguments, not the perceived tone of this paper.

We always saw this paper as a reference point and platform to encourage greater diversity, debate, and innovation. However, the burden of proof placed on our claims was unbelievably high in comparison to papers which were considered less “political” or simply closer to orthodox views. Making the case for democracy was heavily contested, despite reams of supporting empirical and theoretical evidence. In contrast, the idea of differential technological development, or the NTI framework, have been wholesale adopted despite almost no underpinning peer-review research. I wonder how much of the ideas we critique here would have seen the light of day, if the same suspicious scrutiny was applied to more orthodox views and their authors.

We wrote this critique to help progress the field. We do not hate longtermism, utilitarianism or transhumanism,. In fact, we personally agree with some facets of each. But our personal views should barely matter. We ask of you what we have assumed to be true for all the authors that we cite in this paper: that the author is not equivalent to the arguments they present, that arguments will change, and that it doesn’t matter who said it, but instead that it was said.

The EA community prides itself on being able to invite and process criticism. However, warm welcome of criticism was certainly not our experience in writing this paper.

Many EAs we showed this paper to exemplified the ideal. They assessed the paper’s merits on the basis of its arguments rather than group membership, engaged in dialogue, disagreed respectfully, and improved our arguments with care and attention. We thank them for their support and meeting the challenge of reasoning in the midst of emotional discomfort. By others we were accused of lacking academic rigour and harbouring bad intentions.

We were told by some that our critique is invalid because the community is already very cognitively diverse and in fact welcomes criticism. They also told us that there is no TUA, and if the approach doe...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: 13 Very Different Stances on AGI, published by Ozzie Gooen on December 27, 2021 on The Effective Altruism Forum.

Epistemic Status: Quickly written (~4 hours), uncertain. AGI policy isn't my field, don't take me to be an expert in it.

This was originally posted to Facebook here, where it had some discussion. Also, see this earlier post too.

Over the last 10 years or so, I've talked to a bunch of different people about AGI, and have seen several more unique positions online. Theories that I've heard include:

AGI is a severe risk, but we have decent odds of it going well. Alignment work is probably good, AGI capabilities are probably bad. (Many longtermist AI people)

AGI seems like a huge deal, but there's really nothing we could do about it at this point. (Many non-longtermist EAs)

AGI will very likely kill us all. We should try to stop it, though very few interventions are viable, and it’s a long shot. (Eliezer. See Facebook comments for more discussion)

AGI is amazing for mankind and we should basically build it as quickly as possible. There are basically no big risks. (lots of AI developers)

It's important that Western countries build AI/AGI before China does, so we should rush it (select longtermists, I think Eric Schmidt)

It's important that we have lots of small and narrow AI companies because these will help draw attention and talent from the big and general-purpose companies.

It's important that AI development be particularly open and transparent, in part so that there's less fear around it.

We need to develop AI quickly to promote global growth, without which the world might experience severe economic decline, the consequences will be massively violent (Peter Thiel, maybe some Progress Studies people to a lesser extent)

We should mostly focus on making sure that AI does not increase discrimination or unfairness in society. (Lots of "Safe AI" researchers, often liberal)

AGI will definitely be fine because that's what God would want. We might as well make it very quickly.

AGI will kill us, but we shouldn't be worried, because whatever it is, it will be more morally important than us anyway. (fairly fringe)

AGI is a meaningless concept, in part because intelligence is not a single unit. The entire concept doesn't make sense, so talking about it is useless.

There's basically no chance of transformative AI happening in the next 30-100 years. (Most of the world and governments, from what I can tell)

Naturally, smart people are actively working on advancing the majority of these. There's a ton of unilateralist and expensive actions being taken.

One weird thing, to me, is just how intense some of these people are. Like, they choose one or two of these 13 theories and really go all-in on them. It feels a lot like a religious divide.Some key reflections:

A bunch of intelligent/powerful people care a lot about AGI. I expect that over time, many more will.

There are several camps that strongly disagree with each other. I think these disagreements often aren't made explicit, so the situation is confusing. The related terminology and conceptual space are really messy. Some positions are strongly believed but barely articulated (see single tweets dismissing AGI concerns, for example).

The disagreements are along several different dimensions, not one or two. It's not simply short vs. long timelines, or "Is AGI dangerous or not?".

A lot of people seem really confident[1] in their views on this topic. If you were to eventually calculate the average Brier score of all of these opinions, it would be pretty bad.

Addendum

Possible next steps

The above 13 stances were written quickly and don't follow a neat structure. I don't mean for them to be definitive, I just want to use this post to highlight the issue.

Some obvious projects include:

Come up with better lists.

Try to isolate t...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Ben Pace's Controversial Picks for the 2020 Review, published by Ben Pace on December 27, 2021 on LessWrong.

This year, the LessWrong books had about 187,000 words in them. This was the top 59 posts last year in the review.

If we count up the posts in the early vote this year, then we get the top 43 posts. Basically, it’s everything up-to-and-including “Forecasting Thread: AI Timelines”. The spreadsheet where I did the math is here.

Now, we may do something fairly different with the results of the review this year. But for now I'm going to run with this as a "passed review" and "didn't pass review" watermark. Then in this post I'm going to make my case for 15 underrated posts in the review. (I encourage others to try this frame out for prioritizing which posts to review.)

Note that I'm about to defend my picks that were controversial within the LW crowd, which is a fun and weird optimization criteria. I'm not going to talk about the super defensible posts or the posts everyone here loved, but the posts many people don't share my impressions of, in the hope that people change their votes. Here goes.

Covid

First are my three Covid picks.

Mazes

Okay, this is time to review the Mazes sequence.

We have 17 posts, summing to 46,000 words. That's nearly a quarter of last year's book.

The sequence is an extended meditation on a theme, exploring it from lots of perspective, about how large projects and large coordination efforts end up being eaten by Moloch. The specific perspective reminds me a bit of The Screwtape Letters. In The Screwtape Letters, the two devils are focused on causing people to be immoral. The explicit optimization for vices and personal flaws helps highlight (to me) what it looks like when I'm doing something really stupid or harmful within myself.

Similarly, this sequence explores the perspective of large groups of people who live to game a large company, not to actually achieve the goals of the company. What that culture looks like, what is rewarded, what it feels like to be in it.

I've executed some of these strategies in my life. I don't think I've ever lived the life of the soulless middle-manager stereotyped by the sequence, but I see elements of it in myself, and I'm grateful to the sequence for helping me identify those cognitive patterns.

Something the sequence really conveys, is not just that individuals can try to game a company, but that a whole company's culture can change such that gaming-behavior is expected and rewarded. It contains a lot of detail about what that culture looks and feels like.

The sequence (including the essay "Motive Ambiguity") has led me see how in such an environment groups of people can end up optimizing for the opposite of their stated purpose.

The sequence doesn't hold together as a whole to me. I don't get the perfect or superperfect competition idea at the top. Some of the claims seem like a stretch or not really argued for, just completing the pattern when riffing on a theme. But I'm not going to review the weaknesses here, my goal is mostly to advocate for the best parts of it that I'd like to see score more highly in the book.

My three picks are:

(Also Moloch Hasn't Won but that was in last year's review and books, so skipping it here.)

(Also Motive Ambiguity, but everyone already agrees with me on that, and also it's not technically part of the sequence.)

Overall, I don't know if this all works out, but it's my current bet on which posts should go into a hypothetical book. Also they're all short, only summing to 1200 + 2000 + 1200 + 1800 = 6200 words (including Motive Ambiguity), which is about 15% of the sequence length, but I claim gets like 50% of the value.

Agent Foundations

There were a couple of truly excellent posts in the quest to understand foundational properties of agents, an area of research that I ...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: On Stateless Societies, published by Martin Sustrik on December 27, 2021 on LessWrong.

Acemoglu & Robinson in their book The Narrow Corridor use Tiv people as an example of a stateless society.

The statelessness among Tiv was achieved by penalizing or eliminating those who grew successful or powerful enough to become an eventual condensation point for creation of a state. The traditional mechanism for doing so was accusing such people of witchcraft.

This, on one hand, left Tiv without the state and the associated oppression, but, on the other hand, kept them dirt-poor.

That sounds all right on the theoretical level, but what made me really visualize the process was a story about how Shaka Zulu, the builder of the Zulu Kingdom, broke the power of stateless elements in the traditional Zulu society.

When a hammerhead crane flew over his kraal and a porcupine wandered into it, when a crow perched on a fence and began to utter human words he summoned a team of witch doctors led by a woman called Nobela:

The Zulus were lined up and Nobela and here associates began "smelling out" the witches who had brought on the evil omens. They picked on prosperous people. One had grown rich through frugality. Another had put cattle manure on his lands as fertilizer, producing a bountiful harvest much greater than his neighbors'. Yet another was a fine stock breeder who had picked the best bulls and taken great care of his stock and as a result had seen a prodigious expansion of his herds.

Shaka took offense, accused the witch doctors of false accusations and demanded that two of them must die in compensation. The witch doctors panicked and asked Shaka for protection. He agreed, conditional on that they "won't cheat any more". That way he broke the power of the institution of witch doctors.

Another interesting data point about stateless societies is this paper by Lowes, Nunn, Robinson and Weigel (yes, the same Robinson as in The Narrow Corridor book).

In turns out that in Congo there is a set of tribes which are ethnically and culturally pretty homogeneous, except that while all of them are historically stateless, one tribe managed to establish a state. The researchers made the people from this area play different economic games and find out that people with a state are less cooperative than stateless people.

This is a single study and may eventually prove to be non-reproducible, but do have a look at the paper: The methodology is pretty rigorous and the experiments are cleverly devised. Moreover, I am inclined to believe the results just because how genuinely confused the researchers seem to be. (They have done the experiment with the expectation that people with the state will be more cooperative.)

The above has challenged my implicit assumption that stateless societies are in the permanent Hobbesian "warre". That they are, basically, failed states.

But from the above it looks like that stateless societies aren't kept around by the failure to coordinate. In fact, they survive by actively cooperating to prevent the hierarchical structures to emerge. From that point of view, it's rather the state that is the result of coordination failure. For whatever reason the traditional mechanisms of coordination suddenly break and the resulting societal collapse is what we refer to as a "state".

In should be said that given how oppressive many early states have been it's hard to blame the Tiv for wanting to avoid it. The approach, interestingly, proved useful also in the colonial era. From Wikipedia:

These socio-political arrangements caused great frustration to British attempts to incorporate the population into Colonial Nigeria and establish an administration on the lower Benue. The strategy of indirect rule, which the British felt to be highly successful in regards to ruling over the Hausa and F...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Movie review: Don't Look Up, published by quinn on December 25, 2021 on The Effective Altruism Forum.

Meryl Streep:

Dr. Strangelove for 2020

Eliezer Yudkowsky:

I'm not going to see this because it looks too on the nose

Before I begin, I'd like you to take a moment to consider what properties you'd like xrisk fiction to satisfy.

I'll go first. I would like my xrisk fiction to

Give a relatable account of "hero license", make trying to stop literally everyone from dying relatable

Dodge the "yeah this disaster has a ginormous impact, but what really matters is that family sticks together / my girlfriend and I stop fighting" tropes

Avoid trivial villainy; coordination failure is more parsimonious than malice, and teaches more valuable lessons

Don't Look Up was poignant and inspiring. It was also imperfect and annoying. In writing a review for the forum, my goals are to 1. unpack what it offers us as an xrisk movement, if anything; 2. make some claims about what Don't Look Up did wrong and how xrisk fiction can improve upon it. Ultimately, I will conclude with a recommendation to suggest it to your non-community friends. There is room for another EA reviewer to 1. take a "hard scifi" approach and cover the numbers and science, as well as 2. optimize the document to be a non-EA's intro to the real-life xrisk community.

Adam McKay was an SNL writer who by 2015 found his way to serious movies with Michael Lewis' the Big Short (film adaptation of the same name) about the 2008 mortgage crisis, then he did a Dick Cheney biopic called Vice. This period of his films is defined by extending his comedy heritage into drama through a rather unique blend of darkish comedy and information-dense historical rehash, frantic editing including found footage, as well as his political opinions. I think of Don't Look Up as being of this period, although it stands out as the only actual fiction.

What happened in Don't Look Up?

If you're the type of person who likes to know as little as possible about a movie before watching, now's the time to stop. All you need to know is that it's about astronomers getting laughed out of the room when they try to raise the alarm. I will say that I greatly enjoyed my inability to predict how it would end for the vast majority of it's length, but most EA Philly members who I was with said they predicted it trivially easily.

A PhD student (Kate Dibiaski played by Jennifer Lawrence) at the University of Michigan astronomy department is looking into evidence of supernovae or something when she sees an asteroid. She reports it to her supervisor (Randall Mindy played by Leonardo DiCaprio) and the lab opens some champaigne and does a bunch of math on the data. When they determine that it's an extinction-level event (direct impact of a 5-9 kilometer object), they contact the person at NASA in charge of such things (Teddy Ogglethorpe played by Rob Morgan). The first thing that happens is the merry trio gets more or less laughed out of the room by the president (Meryl Streep). "You don't know how many apocalypse meetings we have in here" she says, while her mind is really on her friend/supreme court nominee's politically problematic antics.

The trio decide to go to the press, a paper heavily implied to be the New York Times supports them, mentions that they'll need to lawyer up and go on talk shows, and they break the story by appearing on a Regis & Kelly -like show (Cate Blanchett and Tyler Perry) the same morning it's printed. At the talk show, the once-lighthearted hosts are shocked, take it very seriously, which inspires members of key institutions across engineering and policy to do their utmost. That would additionally be a great movie to watch, but I'm kidding: the astronomers get laughed off stage. The overall segment of the show had the least engagement of the wh...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Native languages in the EA community (and issues with assessing promisingness), published by Lizka on December 27, 2021 on The Effective Altruism Forum.

Epistemic status: not certain— a hand-wavy ramble.

Disclaimer: all opinions here are my own, not my employer’s.

Summary

Language barriers can cause people to be dismissive of writing/speech due to aesthetics or communicative clumsiness rather than the content of the speech.

This manifests in the EA community in a number of ways, one of which is that the language we speak is informed by our STEM-leaning community and our corresponding tendency to think quantitatively, which creates a context (discursive context?) that is foreign to people who are less STEM-y.[1]

Harm 1: Good ideas and useful knowledge from these groups get discounted as a result.

Harm 2: Talented non-STEM people who are into EA get misunderstood when they try to communicate, and don’t get noticed as “promising” (and we do in fact really need the these folks).

Harm 3: non-STEM folks have a worse experience in the EA community.

Some suggestions

Pre-amble

My university had a lot of international students. In class discussions and group interactions, I would often notice that the contributions of international students (those whose first language wasn’t English) tended to be dismissed faster and interrupted more frequently than others’, even when the others’ thoughts seemed less insightful. My guess was that this was due to the smoother and more familiar presentation of the native speakers’ thoughts.

This situation was not new to me; I had spent a total of around three years in school in France, experiencing something similar first-hand (as a nerdy American kid). I would frequently have to suppress a mix of anger, annoyance, and shame at being patronized or ignored because my French was halting and awkward.

The point of these stories is that not being fluent (or being less-than-native) in the language of the community with which you are conversing makes everything harder. And it can be unpleasant and discouraging.

I think a version of this happens in the EA community with people who are less STEM-y than the average. (It also happens with other underrepresented groups in EA, but I’m focusing on this one for this post.)

Harm 1: We lose out on ideas and knowledge

I have most frequently seen this phenomenon in live conversations. These folks’ natural speech or writing follows different norms, and they contort their thoughts to make the EA community hear them. They misuse jargon like “updating” and “outside view” in an attempt to get their point across, and their interlocutors decide that talking with them is not worth their time.[2]

More generally, I think experienced (and assimilated) members of the community tend to interpret a lack of fluency in the “language of EA” as a general lack of knowledge or skill. This, together with our tendency to miss writing that comes from outside the community,[3] leads to the community systematically deafening itself to communication by non-STEM folks.

Harm 2: EA is less actively welcoming towards non-STEM people, so we lose out on some of those people

This also factors into the ways promising people (e.g. students) who should be mentored and helped on their way to impact are identified by the EA community. My impression is that currently, this often happens through connections or random conversations where someone’s potential gets identified by someone well-placed in the community. I’m worried that we’re ending up with self-sustaining mechanisms by which interested and talented people who don’t speak numerically (or misuse “outside view”) are considered less promising and are not supported in earlier stages of their career.

(Notice that this can happen both ways: if EA always speaks the language of the STEM-y, less STEM-y ...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: [Book Review] "The Most Powerful Idea in the World" by William Rosen, published by lsusr on December 23, 2021 on LessWrong.

The first nuclear bomb had a immediate impact on world affairs. The first steam engine did not. It took decades for the steam engine to transform civilization.

Why? Because of math.

The engineers at the Manhattan Project built a working nuclear device on their first try because they ran careful calculations. For example, they knew the atomic weights of different isotopes. By measuring the energy and velocity of different decay products and plugging them into E=√p2c2+m20c4 you can calculate the theoretical maximum yield of a nuclear weapon. The limiting factor in going nuclear is (and always has been) uranium[1] and industrial capacity, not technical know-how.

The first commercial steam engine was invented in 1698 by Thomas Savery but the Carnot cycle wasn't proposed by Sadi Carnot until 1824 and the ideal gas law wasn't discovered until 1856.

Imagine building a steam engine 126 years before the discovery of the Carnot cycle. That's like building a nuclear reactor 126 years before E=mc2. It's like building a nuclear reactor with Benjamin Franklin's knowledge of physics.

The early steampunks did not have statistical mechanics in their toolbox. They built their machines first. The science came afterward. The earliest steam engines were extremely inefficient compared to the earliest atomic bombs because they were developed mostly through trial-and-error instead of computing the optimal design mathematically from first principles.

Coal Mines

Mining was a big industry in eighteenth-century England. "Three-quarters of the patents for invention granted prior to the Savery engine [a steam pump] were, one way or another, mining innovations; 15 percent of the total were for drainage alone, as the shortage of surface coal became more and more acute and prices rose."

Water flows downward. Mines are underground. The deeper you dig your mine, the more likely you are to hit the water table. A mine full of water is unusable. A mine must be dry before you can send a miner down with his pickaxe.

Pumping water via animal power is expensive compared to coal pumps. "In 1742, a study was made of a 240-foot-deep coal mine in northeast England in which a horse-driven pump lifted just over 67,000 gallons every twenty-four shillings, while Newcomen's engine pumped more than 250,000 gallons using twenty shillings' of coal—a demonstration not only of the value of the engine, but a newfound enthusiasm for cost accounting."

I am skeptical of author William Rosen's implication that there was anything "newfound" about cost accounting in the eighteenth century. Double-book accounting was invented in Italy in the thirteenth century. Entrepreneurship is thousands of years old.

But putting steam engines next to coal mines does make economic sense. Steam engines (especially the early inefficient ones) require lots of coal. Coal is expensive to transport because locomotives haven't been invented yet. Even if 75% of the coal you mine goes into feeding your coal-powered water pump, the remaining 25% of coal is coal you wouldn't have at all if it weren't for the steam engine. "The coal-fired atmospheric engines designed by Newcomen and Calley burned so much coal for the amount of water pumped that the only cost-effective place for their use was at the coal mine itself.because the cost of transporting the coal to a steam engine more than a few hundred yards from the mine itself ate up any savings the engine offered."

Iron Smelters

To build steam engines, you need iron. Iron ore is common. It's more than five percent of the Earth's crust. But the iron in the Earth is iron oxide i.e. rust. To make iron oxide useful, you need to remove the oxygen. You can remove the oxygen from iron oxide b...