Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Takeaways on US Policy Careers (Part 2): Career Advice, published by Mauricio on November 8, 2021 on The Effective Altruism Forum.

Summary / Key Takeaways

Over the summer, about a dozen very cool policy professionals shared their takes on impactful US policy careers. The sessions were not recorded, so—in hopes that others find the content as useful as I did—here’s Part 2 of what I learned (companion post here). My main personal takeaways are in this section: suggestions for people at various stages in their careers, corrections of a few misconceptions, and a list of resources/compilations in this post.

Moves to seriously consider at various career stages: Here are some options that are often among the best things to do at various stages of your career, if you’re optimizing for US policy jobs and don’t need to worry about US work authorization (although there are plenty of exceptions, and you can very much be fine if you skip some steps or do them differently):

1) If you’re an undergrad, DC-based internships are very useful, and Semester in Washington programs are often very useful for getting these internships.

Your undergraduate degree matters little for most policy jobs; internships, connections, learning, etc. matter more.

2) If you just finished your undergrad, spend about 1-2 years interning/working in Congress or DC think tanks.

This can both set you up well for a policy-relevant graduate degree and inform you about your fit for it.

On breaking into policy work:

See these two other posts for advice on applications.

The hardest part of getting into policy work is getting a foot in the door; then, you can use your network from your first job to get the next job. So if you want to go into policy, just get started somehow—even if your first job isn’t very high-impact.

If you’re sympathetic to this community and looking to break into policy work (or connect with likeminded policy professionals), get in touch with the DC EA community.

Get your application materials thoroughly reviewed by people who know what application reviewers are looking for. Call in favors / draw on connections to get this very useful help.

3) If you’ve worked in US policy for ~1-2 years—or without doing that, if you have trouble getting those jobs)—a great next move is often going to graduate school to get a policy master’s degree (in a DC-based program) or a law degree (in DC or a top program). If you can, intern / work (part-time / in summers) in Congress or think tanks as you do your degree.

Broadly, high-impact jobs in government disproportionately require some sort of graduate study.

Top options for policy-oriented master’s degrees include:

Georgetown’s School of Foreign Service / McCourt School of Public Policy

Johns Hopkins’ School of Advanced International Studies[1]

Law degrees are often very useful, although you typically shouldn’t start a law program if you’re set on doing think tank research (they’re not as useful for that).

(Some career advice seems to have given some people the impression that they need to go to e.g. Harvard Law School to do the most impactful US policy work. Anecdotally, this seems to be incorrect.[2])

While relevant PhDs can be useful, the time/opportunity cost is huge, so starting a (US-based) PhD is often not as good for a career in US policy as doing a shorter graduate degree followed by several years of policy work.

Getting a PhD might be a great move if you’ve already started a PhD program, if you do it in the UK (much lower time costs), if you’re targeting jobs in US executive agencies for which PhDs are very helpful, or if you’re very interested in certain jobs outside of US policy.[2:1]

It’s very valuable if the graduate program you do is located in DC, for work and networking opportunities.

You can often get funding to use a top degree to tackle existen...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: What are red flags for Neural Network suffering?, published by Marius Hobbhahn on November 8, 2021 on The AI Alignment Forum.

Epistemic status: High uncertainty; This is exploratory work; Our goal is to provide possible research directions rather than offering solutions.

This is shared work. Most of the part on neural correlates is by Jan. Most of the parts on behavior and high-level considerations are by Marius.

Background

I (Marius) was listening to the 80k podcast with Chris Olah on his work on interpretability. When he talked about neural circuits, he said they found many circuits similar to structures in the human brain (some are also quite different). The question that I can’t unthink since then is “What evidence would we need to see in Neural Networks that would convince us of suffering in a morally relevant way?”. So I teamed up with Jan to explore the question.

To clarify, we don’t care about whether they suffer in a similar way that humans do or to the same extent. The question that interests us is “Do neural networks suffer?” and, more importantly, “How would we tell?”.

Broadly, we think there are three different approaches to this problem: a) Neural correlates, b) Behavioural data, and c) high-level considerations.

In this post, we want to find out if there are any avenues that give us reasonable answers or if you just run into a wall, like with most other questions about suffering, patienthood, etc.

It’s not super clear what we are looking for exactly. In general, we are looking for all kinds of evidence that can potentially update our probability of NN suffering.

Summary of the results:

Neural Correlates: Neuroscience helps to de-confuse some terminology, but has not (yet) unambiguously identified a unique neural correlate for suffering. Pain is much better characterized as a “homeostatic behavioral drive” with associated cognitive routines. Since pain usually leads to suffering, finding the homeostatic behavioral drive might also be indicative of suffering in NNs. Finally, the PANAS measure from psychology might provide comparability to humans.

Behavior: Looking at the response of NNs to different stimuli might yield some evidence about their ability to suffer. In language models, it is unclear how to test for the existence of suffering. But testing if GPT-3, for example, has a consistent concept of suffering or observing its answers to the PANAS questions might be a start. For RL agents we can create tests comparable to animal studies. For example, one could look at the reaction to negative stimuli, lesion studies, theory of mind or self-awareness as potential evidence for the ability to suffer.

High-level considerations: We think that this is the most promising avenue. Firstly, the ability to suffer probably develops because it gives an evolutionary edge. Thus, it is plausible that artificial agents such as reinforcement learners would develop it given the right capabilities and conditions. Secondly, current NN architectures are much smaller than human brains or other biological neural networks if you compare the number of parameters with the number of neurons or synapses. Especially the architectures that would plausibly develop the ability to suffer, e.g., reinforcement learners, are still much smaller than tiny animals.

Combining all of the above, we think it is more plausible that current NN architectures don’t suffer than that they do. However, we are pretty uncertain since we mix the fuzzy concept of suffering with NNs, which are also not well understood.

Neural correlates

First, to avoid specific philosophical tripwires[1], we engineer our concepts using insights from neuroscience. The hope is that we might be able to derive necessary or sufficient conditions (or at least bundles of features that strongly correlate with suffering) that general...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: What are red flags for Neural Network suffering?, published by Marius Hobbhahn on November 8, 2021 on The AI Alignment Forum.

Epistemic status: High uncertainty; This is exploratory work; Our goal is to provide possible research directions rather than offering solutions.

This is shared work. Most of the part on neural correlates is by Jan. Most of the parts on behavior and high-level considerations are by Marius.

Background

I (Marius) was listening to the 80k podcast with Chris Olah on his work on interpretability. When he talked about neural circuits, he said they found many circuits similar to structures in the human brain (some are also quite different). The question that I can’t unthink since then is “What evidence would we need to see in Neural Networks that would convince us of suffering in a morally relevant way?”. So I teamed up with Jan to explore the question.

To clarify, we don’t care about whether they suffer in a similar way that humans do or to the same extent. The question that interests us is “Do neural networks suffer?” and, more importantly, “How would we tell?”.

Broadly, we think there are three different approaches to this problem: a) Neural correlates, b) Behavioural data, and c) high-level considerations.

In this post, we want to find out if there are any avenues that give us reasonable answers or if you just run into a wall, like with most other questions about suffering, patienthood, etc.

It’s not super clear what we are looking for exactly. In general, we are looking for all kinds of evidence that can potentially update our probability of NN suffering.

Summary of the results:

Neural Correlates: Neuroscience helps to de-confuse some terminology, but has not (yet) unambiguously identified a unique neural correlate for suffering. Pain is much better characterized as a “homeostatic behavioral drive” with associated cognitive routines. Since pain usually leads to suffering, finding the homeostatic behavioral drive might also be indicative of suffering in NNs. Finally, the PANAS measure from psychology might provide comparability to humans.

Behavior: Looking at the response of NNs to different stimuli might yield some evidence about their ability to suffer. In language models, it is unclear how to test for the existence of suffering. But testing if GPT-3, for example, has a consistent concept of suffering or observing its answers to the PANAS questions might be a start. For RL agents we can create tests comparable to animal studies. For example, one could look at the reaction to negative stimuli, lesion studies, theory of mind or self-awareness as potential evidence for the ability to suffer.

High-level considerations: We think that this is the most promising avenue. Firstly, the ability to suffer probably develops because it gives an evolutionary edge. Thus, it is plausible that artificial agents such as reinforcement learners would develop it given the right capabilities and conditions. Secondly, current NN architectures are much smaller than human brains or other biological neural networks if you compare the number of parameters with the number of neurons or synapses. Especially the architectures that would plausibly develop the ability to suffer, e.g., reinforcement learners, are still much smaller than tiny animals.

Combining all of the above, we think it is more plausible that current NN architectures don’t suffer than that they do. However, we are pretty uncertain since we mix the fuzzy concept of suffering with NNs, which are also not well understood.

Neural correlates

First, to avoid specific philosophical tripwires[1], we engineer our concepts using insights from neuroscience. The hope is that we might be able to derive necessary or sufficient conditions (or at least bundles of features that strongly correlate with suffering) that general...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Transcript for Geoff Anders and Anna Salamon's Oct. 23 conversation, published by Rob Bensinger on November 8, 2021 on LessWrong.

Geoff Anders of Leverage and Anna Salamon of CFAR had a Twitch conversation on October 23, triggered by Zoe Curzi's post about having horrible experiences at Leverage.

Technical issues meant that the Twitch video was mostly lost, and the audio only resurfaced a few days ago, thanks to Lulie.

I thought the conversation was pretty important, so I'm (a) using this post the signal-boost the audio, for people who missed it; and (b) posting a transcript here, along with the (previously unavailable, and important for making sense of the audio) last two hours of chat log.

The chat messages I have aren't timestamped, and sometimes consist of side-conversations that don't directly connect with what Geoff and Anna are discussing. So I've inserted blocks of chat log at what seemed like roughly the right parts of the conversation; but let me know if any are in the wrong place for comprehension, and I'll move them elsewhere.

1. (1:57:57) Early EA history and EA/Leverage interactions

[...]Anna Salamon:All right, let's talk.Geoff Anders:All right, all right. Yeah, let's get to it.Anna Salamon:I hear that the rationality and Leverage communities have some sort of history!Geoff Anders:That's right. Okay, so basically, Anna, as I mentioned to you when we chatted about this, I wanted to talk about in general Leverage history, and then there's a long history with the rationality community, and right now, it seems like there's... I'd like to understand what happened with the relations, basically.

Like, earlier on, things were substantially more congenial. More communication. There were various types of joint events. Ben, who suggested he would be called by first name, organized the first solstice for the rationality community. He did that while working at Leverage. That was totally fine. I volunteered at the CFAR workshops. Ben also did that. Then, EA Summit 2013, I thought was sort of a big deal. We had you guys sort of do a half-workshop during that. I thought that was great. Ben reminded me that we actually held a book launch for MIRI somehow. I remember the event very vaguely. Yeah, but it was at Luke Nosek's house, and I don't know. Maybe it was a joint... I don't remember, but there was a bunch of stuff.Let's see. All right, yeah, so I just switched my audio. Ali, let me know if that's better or worse.

But anyway, so there was that, and then I feel like we just had lots of interests in common and so forth, but then somehow, things didn't end up working out as excellently as possible. So I was interested in just chatting about what happened, getting some common context on the history, maybe trying to...

sowgone: 'Our Final Invention', maybe

Oh, yeah. Maybe Our Final... Yes! Yes! I think that was the book. At least, that sounds pretty likely.

Yeah, so basically looking a bit at the past, and also trying to understand where things are presently, and maybe figuring out whether there's anything constructive to be done. Like, in some circumstances it seems like trust is so low that I'd love to know what could make things better, basically.

So anyway, I just talked for a bit. Yeah, so let's talk about it.Anna Salamon:Yeah. Are we still trying to end at 1:00, in 20 minutes?

Geoff Anders:I mean, I'm willing to go on longer. I know you had a thing, but you know-

benitopace: I'd be down for a 60 minute conversation, starting now.

Anna Salamon:Oh, I don't have a thing.Geoff Anders:Oh, okay. Yeah, so-Anna Salamon:Yeah.Geoff Anders:Apologies to all the viewers and so forth, but yeah, let's go to 1:30.

benitopace: Sg!

Anna Salamon:Great. So yeah, I don't know. I guess one thing I want to know...

Look, I'm going to try to tell history. I'm going to get it wrong. There's ...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Upcoming Effective Environmentalism talks, published by James Ozden on November 6, 2021 on The Effective Altruism Forum.

Hi all! We, Effective Environmentalism, are organising upcoming talks from those tackling climate change using an EA (or adjacent) approach. We've got three quite exciting talks lined up over the next two months so if anyone is interested in learning more, do sign up below.

We're always looking for new speakers so if you might be interested or have any suggestions for potentially interesting speakers, please comment below and let me know!

An exploration of the merits of universal ownership vs ESG for decarbonising our financial system, with Dr Ellen Quigley from University of Cambridge Centre for the Study of Existential Risk (CSER). In this talk, Dr Quigley will describe universal ownership as an approach to real-economy decarbonisation, and go over the ways in which ESG is ill-equipped to decarbonise the financial sector (and therefore the real economy).

Johannes Ackva, a Senior Researcher from the Founders Pledge Climate Fund, will discuss recent research, updates to their fund strategy and charity recommendations going forward. Come along to hear some exciting updates on how the focus of the Founders Pledge Climate Fund is shifting and evolving.

Dr John Halstead from the Forethought Foundation will be discussing some recent positive climate science as well as some potential worst-case scenarios. In this talk, John will firstly discuss some good news on climate change: on current policy, emissions look set to be lower than once feared, as is the risk of very high climate sensitivity. Secondly, John will discuss a worst-case scenario in which we burn all of the fossil fuels: how many fossil fuels are there, how likely we are to burn them, how we might do so if we did, the warming that would produce, and what that might mean for life on Earth.

You can also stay in touch with our events via our newsletter (highest recommendation!), Google Calendar, Facebook page (some good discussions here) and Slack Channel.

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Upcoming Effective Environmentalism talks, published by James Ozden on November 6, 2021 on The Effective Altruism Forum.

Hi all! We, Effective Environmentalism, are organising upcoming talks from those tackling climate change using an EA (or adjacent) approach. We've got three quite exciting talks lined up over the next two months so if anyone is interested in learning more, do sign up below.

We're always looking for new speakers so if you might be interested or have any suggestions for potentially interesting speakers, please comment below and let me know!

An exploration of the merits of universal ownership vs ESG for decarbonising our financial system, with Dr Ellen Quigley from University of Cambridge Centre for the Study of Existential Risk (CSER). In this talk, Dr Quigley will describe universal ownership as an approach to real-economy decarbonisation, and go over the ways in which ESG is ill-equipped to decarbonise the financial sector (and therefore the real economy).

Johannes Ackva, a Senior Researcher from the Founders Pledge Climate Fund, will discuss recent research, updates to their fund strategy and charity recommendations going forward. Come along to hear some exciting updates on how the focus of the Founders Pledge Climate Fund is shifting and evolving.

Dr John Halstead from the Forethought Foundation will be discussing some recent positive climate science as well as some potential worst-case scenarios. In this talk, John will firstly discuss some good news on climate change: on current policy, emissions look set to be lower than once feared, as is the risk of very high climate sensitivity. Secondly, John will discuss a worst-case scenario in which we burn all of the fossil fuels: how many fossil fuels are there, how likely we are to burn them, how we might do so if we did, the warming that would produce, and what that might mean for life on Earth.

You can also stay in touch with our events via our newsletter (highest recommendation!), Google Calendar, Facebook page (some good discussions here) and Slack Channel.

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: EA Organization Updates: October 2021, published by Aaron Gertler on November 4, 2021 on The Effective Altruism Forum.

These monthly posts originated as the "Updates" section of the EA Newsletter. Organizations submit their own updates, which we edit for clarity.

You can see previous updates in our repository of past newsletters.

80,000 Hours

This month, Rob Wiblin spoke to Carl Shulman on the common-sense case for existential risk work and its practical implications. He also spoke with Varsha Venugopal on using gossip to help vaccinate every child in India.

Anima International

Anima International published the results of a new survey on Russians’ attitudes towards plant-based alternatives that, among other results, showed a 3-fold increase in consumption in the last 18 months and found that the label “100% plant-based product” is 11 times more attractive compared to “100% vegan product”. You can read more about the study here or see the original in Russian.Otwarte Klatki in Poland released the results of an investigation on mink farms, including interviews with breeders and farm workers, who discuss the presence of disease on farms. They also worked with the Auchan supermarket chain, which signed an agreement to improve the welfare of chickens raised to supply meat for its private-label products; this is the most important broiler chicken commitment they’ve achieved so far.

Animal Advocacy Careers

Animal Advocacy Careers has launched applications for their free, introductory online course.

AAC ran a study of this online course last year, and the results suggested that participants were more likely than a control group to make career plan changes and find roles that seemed promising for helping animals.

The course walks you through some of the key considerations and research involved in working out which are the most cost-effective strategies for you to help animals. It will also help you identify specific opportunities and plan your next steps. The course takes 1-2 hours per week, spread out over a couple of months.

If this sounds helpful to you, see details on their website and apply via the link there. Applications are due by 10 November.

Animal Ethics

Animal Ethics has been supporting the postdoc research of Jara Gutiérrez and others over the last few years. Jara and her professor Javier de Miguel recently published an article in the European Journal of Ecology focusing on the effects of wildfires on animals and ways to reduce such harms. The full paper can be openly accessed at Fires in nature: a review of the challenges for wild animals. The abstract of the paper is available on the Animal Ethics website.

Animal Ethics also published The development of sentience in juvenile animals, which includes two short case studies of the type necessary to eventually settle the question about which animals are sentient and at what stage in development an animal develops sentience.

Centre for Effective Altruism

CEA released a series of EA Forum posts summarizing their progress over the last quarter. These include updates from the Events team, Groups team, Online team, and Executive Director.

Centre for the Governance of AI (GovAI)

GovAI has now officially relaunched as a nonprofit. Their mission remains the same: to build a global research community, dedicated to helping humanity navigate the transition to a world with advanced AI. However, owing to their new structure, they will also be exploring new ways to expand our impact.

They are hiring Research Fellows and a Chief of Staff, and taking applications for their 3-month Summer Fellowship Program (apply before January 1st). More info at governance.ai and in the announcement post.

Centre for the Study of Existential Risk (CSER)

CSER announced a call for lightning talks as part of their 2022 conference (apply by 15 November...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: EA Organization Updates: October 2021, published by Aaron Gertler on November 4, 2021 on The Effective Altruism Forum.

These monthly posts originated as the "Updates" section of the EA Newsletter. Organizations submit their own updates, which we edit for clarity.

You can see previous updates in our repository of past newsletters.

80,000 Hours

This month, Rob Wiblin spoke to Carl Shulman on the common-sense case for existential risk work and its practical implications. He also spoke with Varsha Venugopal on using gossip to help vaccinate every child in India.

Anima International

Anima International published the results of a new survey on Russians’ attitudes towards plant-based alternatives that, among other results, showed a 3-fold increase in consumption in the last 18 months and found that the label “100% plant-based product” is 11 times more attractive compared to “100% vegan product”. You can read more about the study here or see the original in Russian.Otwarte Klatki in Poland released the results of an investigation on mink farms, including interviews with breeders and farm workers, who discuss the presence of disease on farms. They also worked with the Auchan supermarket chain, which signed an agreement to improve the welfare of chickens raised to supply meat for its private-label products; this is the most important broiler chicken commitment they’ve achieved so far.

Animal Advocacy Careers

Animal Advocacy Careers has launched applications for their free, introductory online course.

AAC ran a study of this online course last year, and the results suggested that participants were more likely than a control group to make career plan changes and find roles that seemed promising for helping animals.

The course walks you through some of the key considerations and research involved in working out which are the most cost-effective strategies for you to help animals. It will also help you identify specific opportunities and plan your next steps. The course takes 1-2 hours per week, spread out over a couple of months.

If this sounds helpful to you, see details on their website and apply via the link there. Applications are due by 10 November.

Animal Ethics

Animal Ethics has been supporting the postdoc research of Jara Gutiérrez and others over the last few years. Jara and her professor Javier de Miguel recently published an article in the European Journal of Ecology focusing on the effects of wildfires on animals and ways to reduce such harms. The full paper can be openly accessed at Fires in nature: a review of the challenges for wild animals. The abstract of the paper is available on the Animal Ethics website.

Animal Ethics also published The development of sentience in juvenile animals, which includes two short case studies of the type necessary to eventually settle the question about which animals are sentient and at what stage in development an animal develops sentience.

Centre for Effective Altruism

CEA released a series of EA Forum posts summarizing their progress over the last quarter. These include updates from the Events team, Groups team, Online team, and Executive Director.

Centre for the Governance of AI (GovAI)

GovAI has now officially relaunched as a nonprofit. Their mission remains the same: to build a global research community, dedicated to helping humanity navigate the transition to a world with advanced AI. However, owing to their new structure, they will also be exploring new ways to expand our impact.

They are hiring Research Fellows and a Chief of Staff, and taking applications for their 3-month Summer Fellowship Program (apply before January 1st). More info at governance.ai and in the announcement post.

Centre for the Study of Existential Risk (CSER)

CSER announced a call for lightning talks as part of their 2022 conference (apply by 15 November...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: What high-level change would you make to EA strategy?, published by Nathan Young on November 4, 2021 on The Effective Altruism Forum.

Imagine you are a powerful person within EA. Maybe a large donor or someone with great connections. What high level change would you make in EA strategy?

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: What high-level change would you make to EA strategy?, published by Nathan Young on November 4, 2021 on The Effective Altruism Forum.

Imagine you are a powerful person within EA. Maybe a large donor or someone with great connections. What high level change would you make in EA strategy?

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.