Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: A new database of nanotechnology strategy resources, published by Ben Snodin on November 5, 2022 on The Effective Altruism Forum.

Target audience: people who are considering trying out nanotechnology strategy research, or who want to learn more about nanotechnology (strategy)-related things.

Marie Davidsen Buhl and I have made a database of resources relevant for nanotechnology strategy research. It contains over 40 articles, sorted by relevance for people new to nanotechnology strategy research.

Additional context:

Why nanotechnology strategy research might be important from a longtermist EA perspective: Advanced nanotechnology might arrive in the next couple of decades (my wild guess: there’s a 1-2% chance in the absence of transformative AI) and could have very positive or very negative implications for existential risk. There has been relatively little high-quality thinking on how to make the arrival of advanced nanotechnology go well, and I think there should be more work in this area (very tentatively, I suggest we want 2-3 people spending at least 50% of their time on this by 3 years from now).

See also my EA Forum post on this area from earlier this year, and the 80000 Hours cause profile.

Marie and I created this resources database as part of our roles at Rethink Priorities.

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: [Links post] Economists Chris Blattman and Noah Smith on China, Taiwan, and the likelihood of war, published by Stephen Clare on November 7, 2022 on The Effective Altruism Forum.

Two prominent economists and writers, Chris Blattman and Noah Smith, both recently and independently published blog posts on the likelihood of war between China and the US over Taiwan. They are not optimistic. Both seem to think that war is relatively likely. Unfortunately, neither offers a quantitative forecast, but I think it's interesting and useful for serious researchers to write about their views on such important questions.

Chris's post is called "The prospects for war with China: Why I see a serious chance of World War III in the next decade". Noah's is "Why I think an invasion of Taiwan probably means WW3". They complement each other well, with Chris's post arguing that war seems likely and Noah's post arguing that the US would probably get involved in the event of a war.

Chris's post on why bargaining may break down

Chris applies the bargaining framework he used throughout his book Why We Fight to show why war is becoming more likely as China continues to grow its economy and strengthen its military. Eventually, the status quo of de facto Taiwanese independence will slip outside of its "compromise range". When this happens, Chinese leaders may decide going to war to try to bring about a different outcome may be preferable, despite the costs.

Still, war is risky and negative-sum. A negotiated settlement should, in principle, be preferable for all parties. Chris suggests that negotiations may fail because ideological principles (e.g. democracy vs autocracy) can be non-negotiable, China's crackdown on Hong Kong harmed its reputation for sticking to negotiated settlements, and Xi Jinping is increasingly unchecked and isolated in his leadership.

Noah's post on why the US would probably get drawn into the war

Noah also applies economic thinking to inform his geopolitical analysis, using game theory to predict which war scenarios seem more likely based on the interests of the actors involved.

Noah considers about how important various factors, like national pride, reputation, and military costs, seem to Chinese and American leaders. He then tries to weigh up how important these costs and benefits seem relative to each other to work out the payoffs to each actor is in each of the four possible outcomes of the game. Then he assigns probabilities to work out expected payoffs, and figures out the equilibrium through backwards induction. Long story short, the equilibrium solution under these assumptions is for China to invade and attack the US to maximize its chances of victory, nearly assuring the outbreak of a major great power war.

Of course, there are many ways this analysis could be wrong, and Noah touches on them at the end of the post. For example, given escalation risk, perhaps the payoffs in any "US fights" outcome are hugely negative for both parties, making it very unlikely that they would both choose to fight. Or, perhaps losing a fight over Taiwan would be so negative for China's leadership that it's just too risky (the expected payoff is negative) to undertake. Or, perhaps the leadership of each country is misinformed about the likelihood of each outcome, leading to miscalculation, bad decision-making, and a sub-optimal (i.e. very costly) outcome.

Why they're worth reading

Both posts analyze the US-China-Taiwan situation from a specific, economic framework. How insightful you find them will depend somewhat on how much resemblance you think this kind of rational analysis of weighing expected costs and benefits bears to actual foreign policy decision-making. But I think it's great to see serious thinkers trying to make progress on important questions like "Will World War III happen...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Opportunities that surprised us during our Clearer Thinking Regrants program, published by spencerg on November 7, 2022 on The Effective Altruism Forum.

This post was written by a subset of Clearer Thinking team members, and not all of the team members involved with the regranting necessarily agree with everything said here.

As part of the Clearer Thinking Regrants program, we evaluated over 630 project proposals and ended up with 37 finalists. In the many hours that we spent evaluating these finalists, we learned some things that surprised us and saw many potential opportunities to help the world that we hadn’t considered before. Of course, if you’re an expert in any of these areas, what surprised us non-experts may not surprise you.

Our aim in this post is to share object-level information that we did not know before the regranting program and that we think readers may find interesting or useful. In several cases, we are sharing the fact that specific organizations or projects have significantly more room for funding than we would have guessed, even after accounting for the outcomes of our regrants program. We hope that readers find this information useful. By highlighting some organizations that have room for more funding, we hope that this will lead to impactful giving opportunities.

Summary:

We were surprised that there hasn't been more work to quantify the risks from large-magnitude volcanic eruptions (considering the impact that such eruptions could have).

We were surprised that the Rethink Priorities Surveys Team has significant room for funding for their own project ideas (since most of their work is research/consulting for existing orgs).

We hadn’t previously considered the extent to which the use of boiling as a water treatment method in low- and middle-income countries contributes to indoor air pollution (and, therefore, to potentially negative health outcomes).

We were surprised to learn that the Happier Lives Institute (HLI) has substantial room for additional funding.

We hadn’t considered that there may be significant new ideas about how to reduce the chance of nuclear war (given the age of the field).

Some of our team hadn’t realized how difficult it can be to tell whether there’s a vitamin deficiency in a population, and none of us had realized that point-of-care biosensors might be feasible to roll out in the foreseeable future.

We hadn’t realized that 1Day Sooner conducts activities outside of their human challenge trial work, and that these activities have significant room for funding.

1. We were surprised that there hasn't been more work to quantify the risks from large-magnitude volcanic eruptions (considering the impact that such eruptions could have).

Toby Ord’s best guess (discussed in The Precipice) is that there is a ~1 in 10,000 chance of an existential catastrophe via supervolcano eruption within the next 100 years. He also thinks that most of the risks associated with volcanic eruptions would relate to them causing the collapse of civilization and/or reducing the chance of recovering from that collapse, and that further research quantifying these risks would be helpful.

Volcanologists Mike Cassidy and Lara Mani have argued that there has been insufficient attention paid to quantifying these risks, as well as to identifying where large-magnitude volcanic eruptions are most likely to occur and which measures should be taken to mitigate these risks. It was surprising to us that these questions have not already been investigated in greater detail, considering the impact that such eruptions could have (and that improved quantification may be quite low-cost if carried out by the right team).

2. We were surprised that the Rethink Priorities Surveys Team has significant room for funding for their own project ideas (since most of their work is re...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: The Welfare Range Table, published by Bob Fischer on November 7, 2022 on The Effective Altruism Forum.

Key Takeaways

Our objective: estimate the welfare ranges of 11 farmed species.

Given hedonism, an individual’s welfare range is the difference between the welfare level associated with the most intense positively valenced state that the individual can realize and the welfare level associated with the most intense negatively valenced state that the individual can realize.

Given some prominent theories about the functions of valenced states, we identified over 90 empirical proxies that might provide evidence of variation in the potential intensities of those states.

There are many unknowns across many species.

It’s rare to have evidence that animals lack a given trait.

We know less about the presence or absence of traits as we move from terrestrial vertebrates to most invertebrates.

Many of the traits about which we know the least are affective traits.

We do have information about some significant traits for many animals.

Introduction

This is the second post in the Moral Weight Project Sequence. The aim of the sequence is to provide an overview of the research that Rethink Priorities conducted between May 2021 and October 2022 on interspecific cause prioritization—i.e., making resource allocation decisions across species. The aim of this post is to provide an overview of the Welfare Range Table, which records the results of a literature review covering over 90 empirical traits across 11 farmed species.

Motivations

If we want to do as much good as possible, we have to compare all the ways of doing good—including ways that involve helping members of different species. The Moral Weight Project’s assumptions entail that everyone’s welfare counts the same and that all welfare improvements count equally. Still, some may be able to realize more welfare than others. We’re particularly interested in how much welfare different individuals can realize at a time—that is, their respective welfare ranges. An individual’s welfare range is the difference between the best and worst welfare states the individual can realize at a time. We assume hedonism, according to which all and only positively valenced states increase welfare and all and only negatively valenced states decrease welfare.

Given as much, an individual’s welfare range is the difference between the welfare level associated with the most intense positively valenced state that the individual can realize and the welfare level associated with the most intense negatively valenced state that the individual can realize. In the case of pigs, for instance, that might be the difference between the welfare level we associate with being fully healthy on a farm sanctuary, on the one hand, and a botched slaughter, on the other.

If there’s variation in welfare ranges across taxa, then there’s variation in the capacities that generate the determinants of welfare. So, if there’s such variation and hedonism is true, then there’s variation in the capacities that generate positively and negatively valenced experiences.

As Jason Schukraft argues, we don’t have any good direct measures of the intensity of valenced states that let us make interspecific comparisons. Indeed, we rely on indirect measures even in humans: behavior, physiological changes, and verbal reports. We can observe behavior and physiological changes in nonhumans, but most of them aren’t verbal. So, we have to rely on other indirect proxies, piecing together an understanding from animals’ cognitive and affective traits or capabilities. The Welfare Range Table includes over 90 such traits: some behavioral, others physiological; some more cognitive, others more affective. Then, it indicates whether the empirical literature provides reason to think that members of 11 farmed spec...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: 2022 LessWrong Census?, published by SurfingOrca on November 7, 2022 on LessWrong.

From 2011-2017, there was an annual LessWrong census/survey. Much like a national census, this provided a valuable lens into the demographics and beliefs of LessWrongers. Unfortunately, this tradition appears to have stopped in recent years, with the exception of a mini-revival in 2020. (Scott Alexander appears to have moved the census to SlateStarCodex.)

From what I've read, this is mainly due to of a lack of will/time among those in the community to run this project, and not a general judgement against the census.

If this is the case, I'd like to start a new version of the census this year, with a greater emphasis on alignment research/beliefs about AI and timelines.

Is this a good idea?

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Exams-Only Universities, published by Mati Roy on November 6, 2022 on LessWrong.

Quality: fast write-up (~45 minutes); little researched

I want a university that only does exams, which would include a description of what you need to know for the exams. Bonus would be suggestions for textbooks and online classes to learn the material, but that's optional.

Cost

The costs are:

Exam creation

Supervision

Scoring

Exam creation is a fixed cost per topic per round (you need to change the exam each year).

Supervision, if in person, is partly a fixed cost per location, and partly a variable cost. (Someone can supervise students doing exams on different topics.) Online supervision would also be an option ideally.

Scoring is partially a fixed cost (the part that can be automated) and partially a variable cost (the part that need to be reviewed manually).

Frequency

Frequency of exams could change with demand, but, to start with, you could have one cheap exam per topic per year, timed with the end of normal school years. This would be the exam most people would take, so it would spread the fixed cost among more people. This might also be a more valuable test because it could position you on a normal curve more precisely given the greater amount of people taking it. There could also be more expensive tests throughout the year—more expensive given the fixed cost would be spread among fewer people.

Problem it solves

What this solves:

Decouples learning and exams a) You can learn at your own pace (whether that's more slowly of faster) b) You can learn from wherever you want (maybe you want to learn from different places for different topics)

Creates standardized tests making it easier to compare the competency of students between different schools

Why wanting exams in the first place? Because many governments want them for immigration and many organizations want them for employees.

Questions I have

Has this been done?

Could this be done? Could you have a university that only does exams and emits degrees that are recognized by the US government?

Current state

There are universities that don't require you to attend classes, but I still find those non-ideal because it's:

Bias: In my experience, students that attend have an unfair advantage—Not because they learn more, but because teachers literally (and even intentionally) give unfair hints about their exams to reward students for showing up and justifying their salary.

Inconvenient:

a) You need to be in a specific location for a few years.

b) You can still do some exams only once per year, which prevents doing a degree faster.

Future

While all I'm asking in this post is for non-zero universities to offer this so that the demand for it gets fulfilled, I also have the impression there would be significant benefits at a societal level from more fully decoupling exams and learning in general. It seems to me like test scores would become more meaningful, and it would become cheaper to get scored.

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Effective altruism as a lifestyle movement (A Master’s Thesis), published by Ada-Maaria Hyvärinen on November 6, 2022 on The Effective Altruism Forum.

Earlier this year, Jemina Mikkonen published her Master’s Thesis in Sociology on the topic of “Effective altruism as a lifestyle movement”. For her research, she interviewed 8 persons who were or had been involved in effective altruism through the university/city group Effective Altruism Helsinki. The purpose of the study was to find out if EA is a lifestyle movement as defined by Haenfler, Johnson, Jones (2012). Mikkonen concluded it is.

If you understand Finnish, you can read the whole thesis here. Since the contents of the thesis might be of interest to non-Finnish-speakers as well, I will summarize her key points in this post. To my understanding, the thesis received the grade “good”, so the research is not exceptionally well done, but should fill the expected function of a thesis. Reading these findings you should keep in mind that this work was done by a student, not by an experienced researcher.

To my knowledge this is the only time anyone has conducted sociology research on an EA group. Mikkonen could not find previous research on EA groups specifically, making it her main motivation to choose EA as a thesis topic. Mikkonen is not involved in EA, but followed the EA Helsinki Telegram chat for a while and participated in one online meetup for the purposes of her study.

According to Mikkonen, EA is a lifestyle movement

The concept of a lifestyle movement was developed by Haenfler, Johnson, Jones (2012). Lifestyle movements are loosely organized or non-organized movements that aim for social change primarily by the means of individual lifestyle choices. They are different from “traditional” social movements that have an external focus, aim for collective (often political) action and have some degree of organization. They define lifestyle movements by the following characteristics:

individual (as opposed to collective) action: participation occurs primarily at the individual level with the subjective understanding that others are taking similar action, collectively adding up to social change

private and ongoing action: participation occurs in daily life (so its not public and not episodic)

action is understood as an effort towards social change (so it is not for example exclusively done as self-help or religious exploration)

personal identity is a site of social change: adherents engage in identity work, focusing particularly on cultivating a morally coherent, personally meaningful identity in the context of a collective identity

Mikkonen found that EA satisfied this definition in the following way:

Action is individual: the interviewees felt their impact comes from personal choices, such as donating and studying to later build an EA aligned career

EA is present in daily life: study and work are methods of doing good, donating influences finances and participants use their free time to learn more about EA

Action is done for social change: Mikkonen noted that EA stands out from other lifestyle movements in the sense that in EA, actions are never taken just for the sake of participating in the movement, but they are always tied to the end goal of having an impact

Identity work: Usually, members of a social movement can see their actions as a sign of moral virtue, and taking action helps them keep up with the idea of themselves as a good person. However, in Mikkonen’s interviews nobody seemed to get this feeling out of EA. But many people described having experienced pressure to follow EA principles, and emphasized that they were “just ordinary people”, which Mikkonen interpreted as a way of coping with the high standards of EA. According to her, EA can act as a way of avoiding the identity of an immoral person, rathe...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: A philosopher's critique of RLHF, published by ThomasW on November 7, 2022 on LessWrong.

In the spring, I went to a talk with Brian Christian at Yale. He talked about his book, The Alignment Problem, and then there was an audience Q&A. There was a really remarkable question in that Q&A, which I have transcribed here. It came from the Yale philosophy professor L.A. Paul. I have since spoken to Professor Paul, and she has done some work on AI (and coauthored the paper “Effective Altruism and Transformative Experience”) but my general impression was that she hasn't yet spent a huge amount of time thinking about AI safety. Partly because of this question, I invited her to speak at the CAIS Philosophy Fellowship, which she will be doing in the spring.

The transcript below really doesn't do her question justice, so I'd recommend watching the recording, starting at 55 minutes.

During the talk, Brian Christian described reinforcement learning from human feedback (RLHF), specifically the original paper, where a model was trained with a reward signal generated by having humans rate which of two videos of a simulated robot was closer to a backflip. Paul's question is about this (punctuation added, obviously):

L.A. Paul: So, I found it very interesting, but I'm just not fully understanding the optimistic note you ended on...so, in that example, what was key was that the humans that did the "better" thing, knew what a backflip was. It was something they recognized. It was something they recognized so they could make a judgment. But the real issue for us is recognizing, or for machines is recognizing, entirely new kinds of events, like a pandemic, or a president that doesn't follow the rule of law, or something interesting called the internet, you know there's radically new technological advances. And when something like that happens, those rough judgments of "this is better than that"... In other words, those new things: first, we're terrible at describing them before they come and predicting them. (Although, humans are very good at a kind of one shot learning, so they can make judgments quite quickly. Machines are not like that).

L.A. Paul: Moreover, these better-than judgments that the machine might be relying on could I think quite straightforwardly be invalidated, because everything changes, or deep things change, in all kinds of unexpected ways. That just seems to be...that's the real problem. It's not... using machines for things that we already have control over. No, it's about trust with entirely new categories of events. So, I was just sort of deeply unclear on... I mean that seems like a nice thing...but that's not, for me, the real alignment problem.

Brian Christian: [Agrees, and then talks about calibrated uncertainty in models.]

L.A. Paul: Sorry, there's a difference between uncertainty, where you're not sure if it's A, B, or C, and unknown, OK, which is a different kind of uncertainty in probabilistic literature. And then you haven't got, oh is it A, is it B, is it C? It's some other kind of thing that you can't classify and that's the problem I'm trying to target.

I'm not claiming that these are original ideas or that they represent all possible critiques of RLHF. Rather:

I think that the phrasing is especially crisp (for speaking, I'm sure she could write it even more crisply).

I think it's interesting that somebody who is very intelligent and accomplished in philosophy but is not (I think) steeped in the alignment literature could seemingly easily carve this problem at its joints.

Also, the rest of the talk is pretty good too! Especially the Q&A, there were some other pretty good questions (including one from myself). But this one stood out for me.

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: What's Happening in Australia, published by Bradley Tjandra on November 7, 2022 on The Effective Altruism Forum.

Introduction

Crikey! There's a lot going on in 'Straya right now!

There are a lot of new and exciting projects happening in Australia— we want to share some of that with the wider community and talk about how you can be involved.

Projects in Australia

Much of the effective altruist aligned work being done in Australia involves working remotely with the international community, at organisations already known by the international community. Given that, we decided to focus on projects which are being at least partially led by Australians.

If you’re leading an exciting project in Australia, please send me a message with a completed copy of this template and I can add it into the post.

AI Safety Australia & New Zealand

AI Safety Support

EA Pathfinder

Foundations for Tomorrow

Giving What We Can

Good Ancestors Project (Sydney Office)

Good Ancestors Project (Operations)

Good Ancestors Project (Policy)

High Impact Engineers

High Impact Recruitment

Insights for Impact

Lead Exposure Elimination Project

Quantifying Uncertainty in GiveWell CEAs

Ready Research

Sentience Institute

AI Safety Australia & New Zealand

What do we do?

We’re organising a number of community building programs:

AI Sydney Fellowship - A fellowship for aspiring AI researchers in Australian/New Zealand to meet and collaborate on their projects.

Sydney Retreat - A weekend getaway near Maroubra Beach and trying out a range of activities to facilitate connection, ideation, sharing, and, of course, alignment.

Informal chats - Discussion group for people who have already completed the AGI Safety Fundamentals technical course or otherwise have an equivalent level of knowledge.

About the team

Chris Leong - Main Organiser. Previously interned at Non-Linear.

Yanni Kyriacos - Marketing & Communications Lead. Recently employed at Spark Wave as Head of Marketing. 10 years of experience as a marketing strategist.

Mark Carauleanu - Intern. Previously Admin Support at CEA, participated in the SERI Summer Fellowship and MLSS.

Jenna - Operations Contractor. Also Design & Communications Consultant at Fortify Health and Cofounder of EA Canberra.

Call to action

Reach out to chris [at] aisafetysupport.org.

More links

Join the Facebook group

Nudge competition

AI Safety Support

What do we do?

Providing support services to early career, independent and transitioning AI alignment researchers. This is done through career coaching, events, training programs and fiscal sponsorship.

About the team

JJ Hepburn - Cofounder and CEO based in Sydney

Rachel Williams - COO based in Colorado Springs

Frances Lorenz - Operations Manager based in Boston

Shay Gestal - Health Coach based in Berlington

Call to action

Signup for our mailing list

Join the AI Alignment slack

Apply for coaching

Checkout our free Health Coaching

Contact us about fiscal sponsorships

More links

aisafetysupport.org

EA Pathfinder

What do we do?

EA Pathfinder is an organisation that wants to create a world in which professionals motivated by Effective Altruism can do high impact work. We are addressing the bottlenecks specific to mid-career professionals by offering career advising, peer support, research mentorship and project management support.

About the team

Neil Ferro has recently come on board as a career advisor.

Call to action

If you have 5+ years of career experience and have a successful career, but feel your job doesn't have the positive impact you’d like then you can apply for free career advice here.

More links

You can visit the website here

Foundations for Tomorrow

What do we do?

Foundations for Tomorrow is a proudly independent, leader-focused non-profit with the mission of protecting Australia’s future interests. Our aspiration over the next 3-5 years i...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Has anyone increased their AGI timelines?, published by Darren McKee on November 6, 2022 on LessWrong.

The question: Within the last 5-10 years, is there is any person or group that has openly increased their AGI timelines?Ideally, they would have at least two different estimates (years apart?), with the most recent estimate showing that they think AGI is further into the future than the prior estimate(s). Background: Whenever I see posts about AGI timelines, they all seem to be decreasing (or staying the same, with methodological differences making some comparisons difficult). I wondered if I'm missing some subset of people or forecasters that have looked at recent developments and thought that AGI will come later not sooner. Another framing, am I wrong if I say, "Almost everyone is decreasing their timelines and no one is increasing them" ?

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.