Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Satisficers Tend To Seek Power: Instrumental Convergence Via Retargetability, published by Alex Turner on November 18, 2021 on The AI Alignment Forum.

Summary: Why exactly should smart agents tend to usurp their creators? Previous results only apply to optimal agents tending to stay alive and preserve their future options. I extend the power-seeking theorems to apply to many kinds of policy-selection procedures, ranging from planning agents which choose plans with expected utility closest to a randomly generated number, to satisficers, to policies trained by some reinforcement learning algorithms. The key property is not agent optimality—as previously supposed—but is instead the retargetability of the policy-selection procedure. These results hint at which kinds of agent cognition and of agent-producing processes are dangerous by default.

I mean "retargetability" in a sense similar to Alex Flint's definition:

Retargetability. Is it possible, using only a microscopic perturbation to the system, to change the system such that it is still an optimizing system but with a different target configuration set?

A system containing a robot with the goal of moving a vase to a certain location can be modified by making just a small number of microscopic perturbations to key memory registers such that the robot holds the goal of moving the vase to a different location and the whole vase/robot system now exhibits a tendency to evolve towards a different target configuration.

In contrast, a system containing a ball rolling towards the bottom of a valley cannot generally be modified by any microscopic perturbation such that the ball will roll to a different target location.

(I don't think that "microscopic" is important for my purposes; the constraint is not physical size, but changes in a single parameter to the policy-selection procedure.)

I'm going to start from the naive view on power-seeking arguments requiring optimality (i.e. what I thought early this summer) and explain the importance of retargetable policy-selection functions. I'll illustrate this notion via satisficers, which randomly select a plan that exceeds some goodness threshold. Satisficers are retargetable, and so they have orbit-level instrumental convergence: for most variations of every utility function, satisficers incentivize power-seeking in the situations covered by my theorems.

Many procedures are retargetable, including every procedure which only depends on the expected utility of different plans. I think that alignment is hard in the expected utility framework not because agents will maximize too hard, but because all expected utility procedures are extremely retargetable—and thus easy to "get wrong."

Lastly: the unholy grail of "instrumental convergence for policies trained via reinforcement learning." I'll state a formal criterion and some preliminary thoughts on where it applies.

The linked Overleaf paper draft contains complete proofs and incomplete explanations of the formal results.

Retargetable policy-selection processes tend to select policies which seek power

To understand a range of retargetable procedures, let's first orient towards the picture I've painted of power-seeking thus far. In short:

Since power-seeking tends to lead to larger sets of possible outcomes—staying alive lets you do more than dying—the agent must seek power to reach most outcomes. The power-seeking theorems say that for the vast, vast, vast majority of variants of every utility function over outcomes, the max of a largerFootnote: similarity set of possible outcomes is greater than the max of a smaller set of possible outcomes. Thus, optimal agents will tend to seek power.

But I want to step back. What I call "the power-seeking theorems", they aren't really about optimal choice. They're about two facts.

Being powerful means...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: EA Should Spend Its “Funding Overhang” on Curing Infectious Diseases, published by joshcmorrison on November 17, 2021 on The Effective Altruism Forum.

Intro

Tl;dr

Epistemic Status

Big Problem

Good Solutions

Advanced Market Commitments

Challenge Studies

Other

Good Timing

Good for the EA Movement

Diversification

Reputation

Catastrophic Risk

Good Arguments Against

No Rigorous Methodology

Long-Term Disease Burden Could Be Much Lower

1Day Sooner May Have been Wrong About COVID Challenge

Vaccines May Be Very Difficult to Develop

Vaccines Aren’t a Silver Bullet

Gates and Wellcome Already Fund This

Next Steps

Intro

There has been much discussion in the EA community lately of a “funding overhang” where there is more effective altruist money to spend than EA uses to spend it on.

As a nonprofit employee of 1Day Sooner whose salary is largely paid by various members of the EA community, the prospect of a funding overhang sounds too good to be true, but insofar as it does exist, I’d like to argue that a fair amount of this decade’s portion should be spent developing and deploying vaccines against infectious diseases.

Millions of deaths each year are caused by tuberculosis (1.4M), Group A Strep (500K), malaria (400K), influenza (300K), hepatitis C (300K), shigella (200K), and respiratory syncytial virus (160K). On certain reasonable assumptions, I’d argue they can absorb billions of dollars in GiveWell-charity equivalent funding over the next ten years. Beyond the object-level of saving millions of lives, developing effective vaccines to major diseases would benefit pandemic preparedness and EA’s political reputation in a way that could reduce existential risks long-term.

My expertise on vaccine development is at best incomplete, and I am biased because 1Day Sooner is raising money to work in this area. But the scale of the problem is large, multiple reasonable strategies exist to help address it, the timing is opportune, and the approach usefully diversifies EA funding strategies while complementing other long-term EA goals.

Tl;dr: If EA’s investing $10 billion in vaccination over the next ten years could save the equivalent of 3-5 years of disease burden of a disease like tuberculosis, it would represent a cost per disability-adjusted-life-year (DALY) saved of roughly $50-$85 (on par with GiveWell top charities). There are plausible mechanisms available to accomplish this, such as advanced market commitments, challenge studies, systems immunology research, and subsidizing mRNA vaccine manufacturing capacity. The effective altruism movement should quickly begin investigating this area more thoroughly and consider developing funding mechanisms commensurate with the scale of the opportunity.

Epistemic Status:I feel this makes a decent but simplistic default case that merits mild confidence in the absence of refutation (which I encourage people to do so as to sharpen the analysis).

Big Problem: Infectious disease is a big enough problem that a number of GiveWell’s top charities (e.g. Against Malaria Foundation and the END Fund) focus on mitigating its impact. Developing and deploying vaccines can dramatically reduce disease incidence in the long-run.

To give a sense of scale, here are the DALY’s lost annually to some vaccine-preventable diseases.

Per this workshop on Group A Strep, an accelerated vaccine development schedule for a disease might typically take about eight years with a further eight years for global deployment. It estimates a more traditional development process at 13 years and deployment at 15-20 years thereafter.

If GiveWell charities buy a DALY at $80, it would be worth it to invest (for example) $3.2 billion to eliminate a single year of tuberculosis disease burden.

Good Solutions: While I have not done comprehensive research into potential strategies to deploy fu...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: EA Should Spend Its “Funding Overhang” on Curing Infectious Diseases, published by joshcmorrison on November 17, 2021 on The Effective Altruism Forum.

Intro

Tl;dr

Epistemic Status

Big Problem

Good Solutions

Advanced Market Commitments

Challenge Studies

Other

Good Timing

Good for the EA Movement

Diversification

Reputation

Catastrophic Risk

Good Arguments Against

No Rigorous Methodology

Long-Term Disease Burden Could Be Much Lower

1Day Sooner May Have been Wrong About COVID Challenge

Vaccines May Be Very Difficult to Develop

Vaccines Aren’t a Silver Bullet

Gates and Wellcome Already Fund This

Next Steps

Intro

There has been much discussion in the EA community lately of a “funding overhang” where there is more effective altruist money to spend than EA uses to spend it on.

As a nonprofit employee of 1Day Sooner whose salary is largely paid by various members of the EA community, the prospect of a funding overhang sounds too good to be true, but insofar as it does exist, I’d like to argue that a fair amount of this decade’s portion should be spent developing and deploying vaccines against infectious diseases.

Millions of deaths each year are caused by tuberculosis (1.4M), Group A Strep (500K), malaria (400K), influenza (300K), hepatitis C (300K), shigella (200K), and respiratory syncytial virus (160K). On certain reasonable assumptions, I’d argue they can absorb billions of dollars in GiveWell-charity equivalent funding over the next ten years. Beyond the object-level of saving millions of lives, developing effective vaccines to major diseases would benefit pandemic preparedness and EA’s political reputation in a way that could reduce existential risks long-term.

My expertise on vaccine development is at best incomplete, and I am biased because 1Day Sooner is raising money to work in this area. But the scale of the problem is large, multiple reasonable strategies exist to help address it, the timing is opportune, and the approach usefully diversifies EA funding strategies while complementing other long-term EA goals.

Tl;dr: If EA’s investing $10 billion in vaccination over the next ten years could save the equivalent of 3-5 years of disease burden of a disease like tuberculosis, it would represent a cost per disability-adjusted-life-year (DALY) saved of roughly $50-$85 (on par with GiveWell top charities). There are plausible mechanisms available to accomplish this, such as advanced market commitments, challenge studies, systems immunology research, and subsidizing mRNA vaccine manufacturing capacity. The effective altruism movement should quickly begin investigating this area more thoroughly and consider developing funding mechanisms commensurate with the scale of the opportunity.

Epistemic Status:I feel this makes a decent but simplistic default case that merits mild confidence in the absence of refutation (which I encourage people to do so as to sharpen the analysis).

Big Problem: Infectious disease is a big enough problem that a number of GiveWell’s top charities (e.g. Against Malaria Foundation and the END Fund) focus on mitigating its impact. Developing and deploying vaccines can dramatically reduce disease incidence in the long-run.

To give a sense of scale, here are the DALY’s lost annually to some vaccine-preventable diseases.

Per this workshop on Group A Strep, an accelerated vaccine development schedule for a disease might typically take about eight years with a further eight years for global deployment. It estimates a more traditional development process at 13 years and deployment at 15-20 years thereafter.

If GiveWell charities buy a DALY at $80, it would be worth it to invest (for example) $3.2 billion to eliminate a single year of tuberculosis disease burden.

Good Solutions: While I have not done comprehensive research into potential strategies to deploy fu...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Our Criminal Justice Reform Program Is Now an Independent Organization: Just Impact, published by evelynciara on November 17, 2021 on The Effective Altruism Forum.

Open Phil just announced that they're spinning off their criminal justice reform program, headed by Chloe Cockburn, into an independent organization! I've reproduced the entire announcement below, subject to Open Phil's Creative Commons license.

Today, we’re making three announcements:

After hundreds of grants totaling more than $130 million over six years, one of our first programs – criminal justice reform (CJR) – is becoming an independent organization.

The team that had been leading our CJR program, Chloe Cockburn and Jesse Rothman, is transitioning to Just Impact, which describes itself as “a criminal justice reform advisory group and fund that is focused on building the power and influence of highly strategic, directly-impacted leaders and their allies to create transformative change from the ground up”.

We are helping to launch Just Impact with approximately $50 million in seed funding spread over 3.5 years.

We’ve had internal discussions around the possibility of a different structure for more than a year, and have spent the past few years continuing to search for new potential causes that might yield cost-effective giving opportunities. That has led to some important updates:

As we wrote in 2019, we think the top global aid charities recommended by GiveWell (which we used to be part of and remain closely affiliated with) present an opportunity to give away large amounts of money at higher cost-effectiveness than we can achieve in many programs, including CJR, that seek to benefit citizens of wealthy countries. Accordingly we’re shifting the focus of future grantmaking from our Global Health and Wellbeing portfolio (which CJR has been part of) further towards the types of opportunities outlined in that post – specifically, efforts to improve and save the lives of people internationally (including things like distributing insecticide-treated bednets to prevent the spread of malaria in Sub-Saharan Africa, and fighting air pollution in South Asia).

At the same time, we have been impressed with the CJR team’s work, which we believe has significantly influenced the field’s priorities, attracted other major donors, and contributed to some notable wins. Rather than shutting down the portfolio entirely, we are instead helping to launch Just Impact so Chloe and her team can continue to make grants that seek to safely reduce incarceration. We hope that other donors interested in criminal justice reform in the United States will join.

The $50 million seed grant to Just Impact is intended to support work for 3.5 years. We will continue to follow progress and continually revisit the right level of support in light of both Just Impact’s impact and our understanding of our alternative giving opportunities, and may continue our support beyond this initial seed grant. It is important to us to make this transition in a way that positions the CJR work to maintain its successes, navigate the transitional period smoothly, and hopefully raise enough from other funders to have even more impact in the future.

Additionally, we think there are other advantages to a spinout:

This is a natural progression for the CJR program and Chloe, who has partnered with other donors for several years, providing advice and supporting donors to make effective investments in criminal justice reform. We believe that an independent organization will allow Chloe to build on that work, and we hope Just Impact will be attractive to donors who want to support these important efforts.

Independence will also better position Just Impact to implement its vision and strategy.

More generally, we see this as a valuable experiment for Open Philanthropy. ...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Our Criminal Justice Reform Program Is Now an Independent Organization: Just Impact, published by evelynciara on November 17, 2021 on The Effective Altruism Forum.

Open Phil just announced that they're spinning off their criminal justice reform program, headed by Chloe Cockburn, into an independent organization! I've reproduced the entire announcement below, subject to Open Phil's Creative Commons license.

Today, we’re making three announcements:

After hundreds of grants totaling more than $130 million over six years, one of our first programs – criminal justice reform (CJR) – is becoming an independent organization.

The team that had been leading our CJR program, Chloe Cockburn and Jesse Rothman, is transitioning to Just Impact, which describes itself as “a criminal justice reform advisory group and fund that is focused on building the power and influence of highly strategic, directly-impacted leaders and their allies to create transformative change from the ground up”.

We are helping to launch Just Impact with approximately $50 million in seed funding spread over 3.5 years.

We’ve had internal discussions around the possibility of a different structure for more than a year, and have spent the past few years continuing to search for new potential causes that might yield cost-effective giving opportunities. That has led to some important updates:

As we wrote in 2019, we think the top global aid charities recommended by GiveWell (which we used to be part of and remain closely affiliated with) present an opportunity to give away large amounts of money at higher cost-effectiveness than we can achieve in many programs, including CJR, that seek to benefit citizens of wealthy countries. Accordingly we’re shifting the focus of future grantmaking from our Global Health and Wellbeing portfolio (which CJR has been part of) further towards the types of opportunities outlined in that post – specifically, efforts to improve and save the lives of people internationally (including things like distributing insecticide-treated bednets to prevent the spread of malaria in Sub-Saharan Africa, and fighting air pollution in South Asia).

At the same time, we have been impressed with the CJR team’s work, which we believe has significantly influenced the field’s priorities, attracted other major donors, and contributed to some notable wins. Rather than shutting down the portfolio entirely, we are instead helping to launch Just Impact so Chloe and her team can continue to make grants that seek to safely reduce incarceration. We hope that other donors interested in criminal justice reform in the United States will join.

The $50 million seed grant to Just Impact is intended to support work for 3.5 years. We will continue to follow progress and continually revisit the right level of support in light of both Just Impact’s impact and our understanding of our alternative giving opportunities, and may continue our support beyond this initial seed grant. It is important to us to make this transition in a way that positions the CJR work to maintain its successes, navigate the transitional period smoothly, and hopefully raise enough from other funders to have even more impact in the future.

Additionally, we think there are other advantages to a spinout:

This is a natural progression for the CJR program and Chloe, who has partnered with other donors for several years, providing advice and supporting donors to make effective investments in criminal justice reform. We believe that an independent organization will allow Chloe to build on that work, and we hope Just Impact will be attractive to donors who want to support these important efforts.

Independence will also better position Just Impact to implement its vision and strategy.

More generally, we see this as a valuable experiment for Open Philanthropy. ...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Common Points of Advice for Students and Early-Career Professionals Interested in Global Catastrophic Risk, published by SethBaum on November 16, 2021 on The Effective Altruism Forum.

This is cross-posted from the GCRI website.

GCRI runs a recurring advising and collaboration program in which we connect with people at all career points who are interested in getting more involved in global catastrophic risk. Through that program, I have had the privilege of speaking with many people to share my experience in the field and help them find opportunities to advance their careers in global catastrophic risk. It has been an enriching experience, and I thank all of our program participants.

Many of the people in our program are students and early-career professionals. In my conversations with them, several common themes tend to emerge, which is to say that I find myself offering certain recurring points of advice to a wide range of people. This post summarizes these themes for anyone who may benefit from it.

The post covers the following themes:

Seek a mix of advice

There are many paths to success in global catastrophic risk

Get the right balance between specialization and generalization

Work across the divide between (A) humanities-social science-policy and (B) engineering-natural science

Know your allies

Some graduate school options

Some fellowship programs

Advice for graduate students

Professional networking and etiquette

Seek a mix of advice

Perhaps the most important advice I can give is to seek advice from a range of different people. What follows below is my own advice from my own particular experience. Other people may reasonably disagree, or may have ideas that I’m less familiar with. My advice is probably worth something, but I would caution against treating it as definitive.

Similarly, the words of advice written here may differ from the personalized advice I would give someone in a one-on-one conversation.

There are many paths to success in global catastrophic risk

One of the most common questions people ask me is what they need to be doing to prepare for a career in global catastrophic risk. They want to know things like which subjects they should study at university, which professional experiences they should pursue through jobs and internships, and what cities or countries they should live in. They may likewise worry that they’re on the wrong path.

This is certainly an understandable question. Global catastrophic risk is not a typical career option. There is no such thing as “pre-GCR” in the way that there is “pre-law” or “pre-med”. And so it is less obvious how to position oneself for a GCR career.

The answer I give to this question is that there is no one single answer. There are many paths that one can take toward a successful career. Global catastrophic risk is a highly multifaceted topic. One can start from virtually any background and find a way to fit in. This is good! One need not stress out about being on the wrong path. Whatever path one is already on, there is almost always a good way forward. Likewise, instead of recommending a generic course of study and work, I recommend people find a path that is customized for their own background interests and abilities.

My own career is a case in point. I started out in engineering, did a PhD in geography with a dissertation on climate change policy, and a post-doctoral fellowship in environmental psychology. These topics provided a good basis for work on global catastrophic risk, though they are also not the primary focus of my ongoing work at GCRI. Instead, I continuously learn new topics to keep up with the mix of activities needed to advance efforts to address global catastrophic risk. My background has worked well for me, but I wouldn’t recommend it for everyone, and I could have also done well with othe...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Common Points of Advice for Students and Early-Career Professionals Interested in Global Catastrophic Risk, published by SethBaum on November 16, 2021 on The Effective Altruism Forum.

This is cross-posted from the GCRI website.

GCRI runs a recurring advising and collaboration program in which we connect with people at all career points who are interested in getting more involved in global catastrophic risk. Through that program, I have had the privilege of speaking with many people to share my experience in the field and help them find opportunities to advance their careers in global catastrophic risk. It has been an enriching experience, and I thank all of our program participants.

Many of the people in our program are students and early-career professionals. In my conversations with them, several common themes tend to emerge, which is to say that I find myself offering certain recurring points of advice to a wide range of people. This post summarizes these themes for anyone who may benefit from it.

The post covers the following themes:

Seek a mix of advice

There are many paths to success in global catastrophic risk

Get the right balance between specialization and generalization

Work across the divide between (A) humanities-social science-policy and (B) engineering-natural science

Know your allies

Some graduate school options

Some fellowship programs

Advice for graduate students

Professional networking and etiquette

Seek a mix of advice

Perhaps the most important advice I can give is to seek advice from a range of different people. What follows below is my own advice from my own particular experience. Other people may reasonably disagree, or may have ideas that I’m less familiar with. My advice is probably worth something, but I would caution against treating it as definitive.

Similarly, the words of advice written here may differ from the personalized advice I would give someone in a one-on-one conversation.

There are many paths to success in global catastrophic risk

One of the most common questions people ask me is what they need to be doing to prepare for a career in global catastrophic risk. They want to know things like which subjects they should study at university, which professional experiences they should pursue through jobs and internships, and what cities or countries they should live in. They may likewise worry that they’re on the wrong path.

This is certainly an understandable question. Global catastrophic risk is not a typical career option. There is no such thing as “pre-GCR” in the way that there is “pre-law” or “pre-med”. And so it is less obvious how to position oneself for a GCR career.

The answer I give to this question is that there is no one single answer. There are many paths that one can take toward a successful career. Global catastrophic risk is a highly multifaceted topic. One can start from virtually any background and find a way to fit in. This is good! One need not stress out about being on the wrong path. Whatever path one is already on, there is almost always a good way forward. Likewise, instead of recommending a generic course of study and work, I recommend people find a path that is customized for their own background interests and abilities.

My own career is a case in point. I started out in engineering, did a PhD in geography with a dissertation on climate change policy, and a post-doctoral fellowship in environmental psychology. These topics provided a good basis for work on global catastrophic risk, though they are also not the primary focus of my ongoing work at GCRI. Instead, I continuously learn new topics to keep up with the mix of activities needed to advance efforts to address global catastrophic risk. My background has worked well for me, but I wouldn’t recommend it for everyone, and I could have also done well with othe...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Can we influence the values of our descendants?, published by Jsevillamol on November 16, 2021 on The Effective Altruism Forum.

I have recently made public a peer-reviewed preprint. There I talk about how events that happened to our ancestors a long time ago affect us today (intergenerational persistence).

The preprint is blowing off my charts with thousands of visits. So I thought I would share more thoughts while the iron is still hot.

In this post I write more informal comments - my personal takeaways. Take these as more informed by my intuition than by hard evidence.

You have been warned.

In short

Intergenerational persistence is an important topic for Effective Altruism. It can help us understand how our actions today can affect people many generations later. For example, I want to know if promoting truth and cooperation today can persistently affect the values of future people.

The Forethought Foundation hired me to review seven papers on these sorts of persistent effects. These papers study things that may seem a bit... outlandish. Examples include a) the relation between the agricultural tools used by your ancestors and modern gender attitudes; b) whether descendants of Africans in places raided by slavers are less trusting; and c) whether descendants of early Catholics are more individualistic.

I'll admit it - I had a strong bias against intergenerational persistence when I started. With all due respect to my grand-grand-grand-grand-grand parents, I found it hard to believe that they have influenced me very much. So on I went, hunting for how the researchers had managed to mess it up.

After engaging with the literature, I have reluctantly changed my mind. Many papers I reviewed had some methodological issues, yes. But I also found careful, robust and reasonable analysis on some key papers. Short of manufacturing the data, I don’t see how they could have got such strongly significant results otherwise.

I now believe that there is a small degree of intergenerational persistence of values. Emphasis on small - if the effects were bigger I would have expected to see more dramatic examples in the papers.

So sorry grand-grand-grand-grand-grand mother - seems like I owe you a bit more of my heritage than I thought I did.

It is still not clear to me how the transmission actually happens. Is it because of parents teaching their children who teach their children and so on? Parents founding institutions that locked-in their values? Parents etching sternly worded reminders in family heirlooms?

Most papers I reviewed argued the transmission was primarily about parents-teaching-children. However, they rely on historical and psychological evidence I don't feel comfortable evaluating.

It is also not clear whether the persistent effects stay constant or diminish with time. I'd expect the latter - if only because I have 4 grandparents, 8 great-grandparents and 16 grand-grand-grand parents. So naively I would expect any effect to dilute with each successive generation.

My impression of the papers in a nutshell

Schulz et al paper on religion, kinship and psychological variation puts up a strong case for cultural transmission. They look at how places where the Catholic Church was established earlier are WEIRDer (particularly, more individualistic). But as the authors point out we are not sure of the causal mechanism. It likely has something to do with religion and how families are structured. But the data studied in the paper is not enough to tell whether the correlation is causal.

Nunn’s previous work on slavery and economic variation is suggestive. He looks at African countries that were specially affected by slavery, and finds that those countries have relatively lower GDP today. But the analysis is underpowered, the causal analysis stops being significant after controlling fo...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Can we influence the values of our descendants?, published by Jsevillamol on November 16, 2021 on The Effective Altruism Forum.

I have recently made public a peer-reviewed preprint. There I talk about how events that happened to our ancestors a long time ago affect us today (intergenerational persistence).

The preprint is blowing off my charts with thousands of visits. So I thought I would share more thoughts while the iron is still hot.

In this post I write more informal comments - my personal takeaways. Take these as more informed by my intuition than by hard evidence.

You have been warned.

In short

Intergenerational persistence is an important topic for Effective Altruism. It can help us understand how our actions today can affect people many generations later. For example, I want to know if promoting truth and cooperation today can persistently affect the values of future people.

The Forethought Foundation hired me to review seven papers on these sorts of persistent effects. These papers study things that may seem a bit... outlandish. Examples include a) the relation between the agricultural tools used by your ancestors and modern gender attitudes; b) whether descendants of Africans in places raided by slavers are less trusting; and c) whether descendants of early Catholics are more individualistic.

I'll admit it - I had a strong bias against intergenerational persistence when I started. With all due respect to my grand-grand-grand-grand-grand parents, I found it hard to believe that they have influenced me very much. So on I went, hunting for how the researchers had managed to mess it up.

After engaging with the literature, I have reluctantly changed my mind. Many papers I reviewed had some methodological issues, yes. But I also found careful, robust and reasonable analysis on some key papers. Short of manufacturing the data, I don’t see how they could have got such strongly significant results otherwise.

I now believe that there is a small degree of intergenerational persistence of values. Emphasis on small - if the effects were bigger I would have expected to see more dramatic examples in the papers.

So sorry grand-grand-grand-grand-grand mother - seems like I owe you a bit more of my heritage than I thought I did.

It is still not clear to me how the transmission actually happens. Is it because of parents teaching their children who teach their children and so on? Parents founding institutions that locked-in their values? Parents etching sternly worded reminders in family heirlooms?

Most papers I reviewed argued the transmission was primarily about parents-teaching-children. However, they rely on historical and psychological evidence I don't feel comfortable evaluating.

It is also not clear whether the persistent effects stay constant or diminish with time. I'd expect the latter - if only because I have 4 grandparents, 8 great-grandparents and 16 grand-grand-grand parents. So naively I would expect any effect to dilute with each successive generation.

My impression of the papers in a nutshell

Schulz et al paper on religion, kinship and psychological variation puts up a strong case for cultural transmission. They look at how places where the Catholic Church was established earlier are WEIRDer (particularly, more individualistic). But as the authors point out we are not sure of the causal mechanism. It likely has something to do with religion and how families are structured. But the data studied in the paper is not enough to tell whether the correlation is causal.

Nunn’s previous work on slavery and economic variation is suggestive. He looks at African countries that were specially affected by slavery, and finds that those countries have relatively lower GDP today. But the analysis is underpowered, the causal analysis stops being significant after controlling fo...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Applications for AI Safety Camp 2022 Now Open!, published by Adam Shimi on November 17, 2021 on The AI Alignment Forum.

If you’ve read about alignment research and you want to start contributing, the new iteration of the AI Safety Camp is a great opportunity!

It’s a virtual camp from January to May 2022, where you collaborate with other applicants to work (1h / normal workday, 7h / weekend sprint day) on open problems proposed and supervised by mentors like John Wentworth, Beth Barnes, Stuart Armstrong, Daniel Kokotajlo. Around this core of research, the camp also includes talks and discussions about fundamental ideas in the field, how alignment research works, and how and where to get a job/funding.

All in all, the AI Safety Camp is a great opportunity if:

You have read enough about alignment that you’re convinced of the importance of the problem

You want to do alignment research (whether conceptual or applied), or to collaborate with alignment researchers (doing policy for example)

You don’t feel yet like you have enough research taste and grasp of the field to choose your research problems yourself yet

Note that you don’t need advanced maths skills to participate in the camp, as some of the projects don’t require any specific skillset or very unusual ones (evolutionary genetics, history...). If you care about alignment and are in this situation, I encourage you to apply for a project without required skillsets and learn what you need as you go along.

All the details on how to apply are available on the website (including the list of open problems).

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.