Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Announcing the Inverse Scaling Prize ($250k Prize Pool), published by Ethan Perez on June 27, 2022 on The AI Alignment Forum.

TL;DR: We’re launching the Inverse Scaling Prize: a contest with $250k in prizes for finding zero/few-shot text tasks where larger language models show increasingly undesirable behavior (“inverse scaling”). We hypothesize that inverse scaling is often a sign of an alignment failure and that more examples of alignment failures would benefit empirical alignment research. We believe that this contest is an unusually concrete, tractable, and safety-relevant problem for engaging alignment newcomers and the broader ML community. This post will focus on the relevance of the contest and the inverse scaling framework to longer-term AGI alignment concerns. See our GitHub repo for contest details, prizes we’ll award, and task evaluation criteria.

What is Inverse Scaling?

Recent work has found that Language Models (LMs) predictably improve as we scale LMs in various ways (“scaling laws”). For example, the test loss on the LM objective (next word prediction) decreases as a power law with compute, dataset size, and model size:

Scaling laws appear in a variety of domains, ranging from transfer learning to generative modeling (on images, video, multimodal, and math) and reinforcement learning. We hypothesize that alignment failures often show up as scaling laws but in the opposite direction: behavior gets predictably worse as models scale, what we call “inverse scaling.” We may expect inverse scaling, e.g., if the training objective or data are flawed in some way. In this case, the training procedure would actively train the model to behave in flawed ways, in a way that grows worse as we scale. The literature contains a few potential examples of inverse scaling. For example, increasing LM size appears to increase social biases on BBQ and falsehoods on TruthfulQA, at least under certain conditions. As a result, we believe that the prize may help to uncover new alignment-relevant tasks and insights by systematically exploring the space of tasks where LMs exhibit inverse scaling.

In particular, submissions must demonstrate new or surprising examples of inverse scaling, e.g., excluding most misuse-related behaviors where you specifically prompt the LM to generate harmful or deceptive text; we don't consider scaling on these behaviors to be surprising in most cases, and we're hoping to uncover more unexpected, undesirable behaviors. Below, we outline two questions in AI alignment that we believe the Inverse Scaling Prize may help to answer.

Q1: In what ways is the language modeling objective outer misaligned?

The above question is important to answer to avoid running into outer-alignment-related catastrophes [1, 2]. Language Models (LMs) are “outer aligned” to the extent that doing well on the LM objective (next word prediction) results in desirable model behavior. Inverse scaling on a task we care about is evidence that the LM objective is misaligned with human preferences; better and better performance on the training objective (language modeling) leads to worse and worse performance on a task we care about. Finding inverse scaling tasks is thus helpful for us in understanding the extent to which the LM objective is outer misaligned, which may be important in two ways:

If the LM objective is fairly well-aligned with human preferences, then that should update us in two ways:

Scaling up LMs would be less likely to lead to outer-alignment-related catastrophes.

We should be more wary of alternative objectives like RL from Human Feedback (RLHF), which draw us away from the pretrained model; RLHF may improve outer alignment on the target task (e.g., summarization) but impair outer alignment in general (e.g., increasing toxicity, bias, or something else the RLHF reward...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Air Conditioner Repair, published by Zvi on June 27, 2022 on LessWrong.

A thing happened to me, seemed worth sharing.

What Happened

A few weeks ago, the in-wall air conditioner in our bedroom ceased to function.

Troy Barnes was unavailable, so instead we talked to the super, he recommended a company called Amhac, we ignored normative determinism (e.g. “Am a hack”) and called them. They charged a rather large amount simply to come out and take a look, with a diagnostic fee and also a noticeably generous per-hour fee.

Then, a few days before they were supposed to show up, our other in-wall air conditioner sprung a leak – or rather, the leak got big enough that we noticed it – and the super advised us we had to shut it down until it was fixed, but he thought that one would be a simple fix plus the addition of a failsafe that should have been there to stop the leak; he wasn’t sure about the bedroom one.

The repairmen showed up, but they hadn’t completed the necessary certificate of insurance to be let into the building – this policy is even more annoying and friction generating than it sounds – and we had to reschedule while they sorted this out. A number of rather hot nights later, they managed to come in.

Their report was that both air conditioners were unfixable. The one in the bedroom was completely shot. The one in the living area was fixable in theory, they said, but due to some EPA regulation it wasn’t possible to fix it.

They could send over a contract for the new units, it would be. $28,000.

I Googled for various new air conditioners, and couldn’t find ones that would be that expensive. The guy explained I would need two of each unit, for four units total.

I asked some trusted friends about all this, as well as the super, and all agreed a second quote would be a very good idea. I asked the super who else had done work in the building, and he remembered M.LaPenna Refrigeration, Inc. They charged by the hour as well, but without a ‘diagnostic fee’ up front.

Which on reflection should have been a hint about the first company. If you’re charging for the labor, as you do, starting with minute one, why is there an additional diagnostic fee? It doesn’t actually make any sense.

This time, they took care of the certificate of insurance as fast as the insurance company could handle it, then got a guy out to look the same day that got approved. Almost as if things were urgent. Different attitude.

Instead of two workers, this time there was one, who was keen to explain to me what was going on as he worked.

The super was busy, so I explained the situation as best I could and he got to poking around in various places.

Two hours later, both air conditioners were working again, and they sent out for the part to install the failsafe.

I asked the man if there was any way the other repairmen could have made an honest mistake saying the units needed to be replaced. His answer: “No.”

If I hadn’t checked, I wonder how much they finally would have tried to get me for, but I’m sure it was a lot.

What To Do About That First Company?

I’m not sure. I emailed my contact to say what happened and suggested a full refund would be appropriate. The response was that the person was on vacation for weeks (with no warning). Which did not endear me.

If it was one unit, I could imagine an honest mistake. But it wasn’t. It was two units, with distinct setups, experiencing distinct problems. This was not a mistake.

What am I supposed to do now? Chargeback? Report to better business bureau? Report to someone else? Do something else? What’s the responsible thing to do here?

I don’t know.

Thoughts and Takeaways

What about takeaways in general?

First of all, get a second opinion. Do not trust contractors of any kind, who you don’t have damn good reason to trust, who tell you that you need something massiv...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Announcing Epoch: A research organization investigating the road to Transformative AI, published by Jaime Sevilla on June 27, 2022 on The Effective Altruism Forum.

Summary

We are a new research organization working on investigating trends in Machine Learning and forecasting the development of Transformative Artificial Intelligence

This work is done in close collaboration with other organizations, like Rethink Priorities, Open Philanthropy, and MIT CSAIL

We will be hiring for 2-4 full-time roles this summer – more information here

You can find up-to-date information about Epoch on our website

What is Epoch?

Epoch is a new research organization that works to support AI strategy and improve forecasts around the development of Transformative Artificial Intelligence (TAI) – AI systems that have the potential to have an effect on society as large as that of the industrial revolution.

Our founding team consists of seven members – Jaime Sevilla, Tamay Besiroglu, Lennart Heim, Pablo Villalobos, Eduardo Infante-Roldán, Marius Hobbhahn, and Anson Ho. Collectively, we have backgrounds in Machine Learning, Statistics, Economics, Forecasting, Physics, Computer Engineering, and Software Engineering.

Our work involves close collaboration with other organizations, such as MIT CSAIL, Open Philanthropy, and Rethink Priorities’ AI Governance and Strategy team. We are advised by Tom Davidson from Open Philanthropy and Neil Thompson from MIT CSAIL. Rethink Priorities is also our fiscal sponsor.

Our mission

Epoch seeks to clarify when and how TAI capabilities will be developed.

We see these two problems as core questions for informing AI strategy decisions by grantmakers, policy-makers, and technical researchers.

We believe that to make good progress on these questions we need to advance towards a field of AI forecasting. We are committed to developing tools, gathering data and creating a scientific ecosystem to make collective progress towards this goal.

Our research agenda

Our work at Epoch encompasses two interconnected lines of research:

The analysis of trends in Machine Learning. We aim to gather data on what has been happening in the field during the last two decades, explain it, and extrapolate the results to inform our views on the future of AI.

The development of quantitative forecasting models related to advanced AI capabilities. We seek to use techniques from economics and statistics to predict when and how fast AI will be developed.

These two research strands feed into each other: the analysis of trends informs the choice of parameters in quantitative models, and the development of these models brings clarity on the most important trends to analyze.

Besides this, we also plan to opportunistically research topics important for AI governance where we are well positioned to do so. These investigations might relate to compute governance, near-term advances in AI and other topics.

Our work so far

Earlier this year we published Compute Trends Across Three Eras of Machine Learning. We collected and analyzed data about the training compute budget of >100 Machine Learning models across history. Consistent with our commitment to field building, we have released the associated dataset and an interactive visualization tool to help other researchers understand these trends better. This work has been featured in Our World in Data, in The Economist and at the OECD.

More recently we have published Grokking “Forecasting TAI with biological anchors” and Grokking “Semi-informative priors over AI timelines”. In these pieces, Anson Ho dissects two popular AI forecasting models. These are the two first installments of a series of articles covering work on quantitative forecasting of when we will develop TAI.

You can see more of our work on our blog. Here is a selection of further work by Epoch membe...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Announcing Epoch: A research organization investigating the road to Transformative AI, published by Jaime Sevilla on June 27, 2022 on The AI Alignment Forum.

Summary

We are a new research organization working on investigating trends in Machine Learning and forecasting the development of Transformative Artificial Intelligence

This work is done in close collaboration with other organizations, like Rethink Priorities, Open Philanthropy, and MIT CSAIL

We will be hiring for 2-4 full-time roles this summer – more information here

You can find up-to-date information about Epoch on our website

What is Epoch?

Epoch is a new research organization that works to support AI strategy and improve forecasts around the development of Transformative Artificial Intelligence (TAI) – AI systems that have the potential to have an effect on society as large as that of the industrial revolution.

Our founding team consists of seven members – Jaime Sevilla, Tamay Besiroglu, Lennart Heim, Pablo Villalobos, Eduardo Infante-Roldán, Marius Hobbhahn, and Anson Ho. Collectively, we have backgrounds in Machine Learning, Statistics, Economics, Forecasting, Physics, Computer Engineering, and Software Engineering.

Our work involves close collaboration with other organizations, such as MIT CSAIL, Open Philanthropy, and Rethink Priorities’ AI Governance and Strategy team. We are advised by Tom Davidson from Open Philanthropy and Neil Thompson from MIT CSAIL. Rethink Priorities is also our fiscal sponsor.

Our mission

Epoch seeks to clarify when and how TAI capabilities will be developed.

We see these two problems as core questions for informing AI strategy decisions by grantmakers, policy-makers, and technical researchers.

We believe that to make good progress on these questions we need to advance towards a field of AI forecasting. We are committed to developing tools, gathering data and creating a scientific ecosystem to make collective progress towards this goal.

Our research agenda

Our work at Epoch encompasses two interconnected lines of research:

The analysis of trends in Machine Learning. We aim to gather data on what has been happening in the field during the last two decades, explain it, and extrapolate the results to inform our views on the future of AI.

The development of quantitative forecasting models related to advanced AI capabilities. We seek to use techniques from economics and statistics to predict when and how fast AI will be developed.

These two research strands feed into each other: the analysis of trends informs the choice of parameters in quantitative models, and the development of these models brings clarity on the most important trends to analyze.

Besides this, we also plan to opportunistically research topics important for AI governance where we are well positioned to do so. These investigations might relate to compute governance, near-term advances in AI and other topics.

Our work so far

Earlier this year we published Compute Trends Across Three Eras of Machine Learning. We collected and analyzed data about the training compute budget of >100 Machine Learning models across history. Consistent with our commitment to field building, we have released the associated dataset and an interactive visualization tool to help other researchers understand these trends better. This work has been featured in Our World in Data, in The Economist and at the OECD.

More recently we have published Grokking “Forecasting TAI with biological anchors” and Grokking “Semi-informative priors over AI timelines”. In these pieces, Anson Ho dissects two popular AI forecasting models. These are the two first installments of a series of articles covering work on quantitative forecasting of when we will develop TAI.

You can see more of our work on our blog. Here is a selection of further work by Epoch members:

Pr...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Contest: An Alien Message, published by DaemonicSigil on June 27, 2022 on LessWrong.

Scenario

You're not certain that aliens are real. Even after receiving that email from your friend who works at NASA, with the subject line "What do you make of this?" and a single binary file attached, you're still not certain. It's rather unlikely that aliens would be near enough to Earth to make contact with humanity, and unlikelier still that of all the humans on Earth, you would end up being the one tasked with deciphering their message. You are a professional cryptanalyst, but there are many of those in the world. On the other hand, it's already been months since April Fools, and your friend isn't really the prankster type.

You sigh and download the file.

Rules

No need to use spoiler text in the comments: This is a collaborative contest, where working together is encouraged. (The contest is between all of you and the difficulty of the problem.)

Success criteria: The people of LessWrong will be victorious if they can fully describe the process that generated the message, ideally by presenting a short program that generates the message, but a sufficiently precise verbal description is fine too.

The timeframe of the contest is about 2 weeks. The code that generated the message will be revealed on Tuesday July 12.

Why is this interesting?

This contest is, of course, based on That Alien Message. Recently there was some discussion under I No Longer Believe Intelligence to be "Magical" about how realistic that scenario actually was. Not the part where the entire universe was a simulation by aliens, but the part where humanity was able to figure out the physics of the universe 1 layer up by just looking at a few frames of video. Sure it may be child's play for Solomonoff Induction, but do bounded agents really stand a chance? This contest should provide some experimental evidence.

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Before There Was Effective Altruism, There Was Effective Philanthropy, published by ColdButtonIssues on June 26, 2022 on The Effective Altruism Forum.

Cross-posted from Cold Button Issues.

Some charities and donors spend a lot of money and mean well but fail to truly help anyone. What are the best things for philanthropically minded people to do and how can they constantly improve? Wouldn’t it be great if somebody tried to solve the problem and maybe even find a way to rate the most influential nonprofits?

That was (and is) the goal of the effective philanthropy movement.

This movement predated effective altruism by over a decade and anticipated some early effective altruism claims, but by now has been eclipsed in the public consciousness by the latter movement.

Google Trends for effective philanthropy (blue) and effective altruism (red)

Yet in terms of donations driven or “money moved,” effective philanthropy has arguably outperformed effective altruism in the last several years, despite its low profile. How?

What Is Effective Philanthropy?

The book on effective philanthropy says that effective philanthropy is “philanthropy that has impact. It is philanthropy that succeeds at amassing, managing, then allocating financial and human resources in ways that have the greatest positive impact in the sectors that foundations choose to fund.”

In a more practical sense, effective philanthropy consists of a few institutions, some writers and thinkers, and a lot of sympathetic foundations. The Center for Effective Philanthropy, run by Phil Buchanan and founded in 2000, is the big player here and receives funding from long established foundations such as the Hewlett Foundation, the Duke Endowment, and the Wallace Foundation. Buchanan is probably the central figure in effective philanthropy, serving as CEP’s first executive director, writing a book on the topic, and hosting a related podcast.

There’s also the twenty-year old Grantmakers for Effective Organizations, which is a coalition of philanthropic funders interested in sharing best practices and has members from every type of philanthropy you can think of. The Hewlett Foundation also funds an effective philanthropy program.

The Center for Effective Philanthropy, like GiveWell would later do, wanted to know which nonprofits were doing a better job than others. And it didn’t want to obsess over administrative overhead or executive pay or simple but misleading measures. It wanted to know which nonprofits were actually fulfilling their function.

Focusing on grantmakers, their approach was to evaluate how well grantmakers were doing by surveying their grantees. And because there is actually a lot of demand for CEP’s surveys, foundations and other grantmakers are able to compare their performance to their peers. Grantees are asked a variety of questions about the quality of their funders along various dimensions- general performance, do they advance the state of the knowledge in their field, did they treat their grantees fairly and so on. (Example report) CEP also offers similar surveys for foundation and grantmaker staff.

This methodology can be used across a field and so hundreds of grantmakers across different causes have chosen to use CEP’s services.

Their approach had a couple of big differences from GiveWell and the other EA charity evaluators. First, it was less rigorous. The information was useful and effective philanthropy enthusiasts are happy to praise RCTs, but the approach wasn’t about building detailed spreadsheets and comparing moral weights of different goods. That said I’m sympathetic to CEP’s approach, especially in fields like advocacy or more speculative causes where even effective altruists who focus very strongly about the question seem unsure how to rate different causes and programs.

Second, effective phil...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: A Critique of The Precipice: Chapter 6 - The Risk Landscape [Red Team Challenge], published by Sarah Weiler on June 26, 2022 on The Effective Altruism Forum.

This post was written as part of the Red Team Challenge by Training For Good. We spent seven weeks writing a critique of Chapter 6 of The Precipice because it is a key text in the Effective Altruism community and we believe that it is important to ensure that our ways of comparing and prioritizing existential risks (henceforth referred to as x-risks) rest on a solid foundation.

Key takeaways

The author provides several useful frameworks for how to think about, compare, and prioritize existential risks - (1) Importance x Tractability x Neglectedness (ITN) (2) Anatomy of extinction risks, (3) Correlation between different x-risks, (4) Existential risk and security factors, and (5) Soon, sudden, sharp.

While the Importance x Tractability x Neglectedness (ITN) framework is provided as a key way to prioritize, the author mainly focuses on the Importance aspect, in particular disregarding the Tractability parameter, which we believe could make a significant impact on the final prioritization. There are some plausible reasons for not trying to estimate Tractability, but we would prefer the author to explicitly mention his reasoning.

The author does not provide a way to combine all of the different frameworks into a single prioritization, leaving this difficult to apply consistently.

Priorities research - the question of how to allocate effort between mitigating x-risks directly vs. investigating the risk landscape to improve future resource allocation - seems important to the chapter, yet is missing from the book entirely.

The methods of x-risk probability calculation are highly uncertain, and the author’s use of these uncertain estimates as a basis for prioritizing between different x-risks and for allocating large amounts of resources to specific targeted interventions is not fully justified.

In light of the large uncertainty of existential risk prioritization, we explore the idea of how broad resilience interventions might side-step some of the issues with estimating the impact of targeted risk reduction, and suggest that these warrant more attention. With the goal of preventing existential catastrophe, focusing on maximizing core resiliency of our species could be a promising alternative.

The scope of our critique

The Precipice focuses on minimizing existential risks (x-risks) to humanity, defined as “permanent destruction of humanity’s potential” (p. 6). These risks include extinction, but also other ways humanity’s long-term potential could be permanently destroyed, e.g. permanent collapse of civilization. While there are arguments that a pure focus on x-risks at the cost of everything else may be misguided, we accept the premise of the book for the purposes of our critique. Instead, we focus on critiquing whether Chapter 6, The Risk Landscape, has achieved its stated goal to “contemplate the entire landscape of existential risk, seeing how the risks compare, how they combine, what they have in common, and which risks should be our highest priorities” (p. 165).

While we focus this critique on points we believe could be even stronger, we also investigated several claims that hold up very well. In particular, we believe that the author (1) provides useful models for thinking about existential risk, (2) presents a very solid mathematical treatment of how multiple risks combine into an overall risk, clearly highlighting some counterintuitive results, (3) highlights not only direct risks, but also existential risk and security factors, which either increase or lower the chance of existential catastrophe without being a direct existential threat themselves, and (4) differentiates well between targeted vs. broad risk in...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: My current take on Internal Family Systems “parts”, published by Kaj Sotala on June 26, 2022 on LessWrong.

I was recently asked how literal/metaphorical I consider the Internal Family Systems model of your mind being divided into “parts” that are kinda like subpersonalities.

The long answer would be my whole sequence on the topic, but that’s pretty long and also my exact conception of parts keeps shifting and getting more refined through the sequence. So one would be excused for still not being entirely clear on this question, even after reading the whole thing.

The short answer would be “it’s more than just metaphorical, but also not quite as literal as you might think from taking IFS books at face value”.

I do think that there are literally neurological subroutines doing their own thing that one has to manage, but I don’t think they’re literally full-blown subminds, they’re more like. clusters of beliefs and emotions and values that get activated at different times, and that can be interfaced with by treating them as if they were actual subminds.

My medium-length answer would be. let’s see.

There’s an influential model in neuroscience called global workspace theory. It says that the brain has a thing called the “global workspace”, which links together a variety of otherwise separate areas, and its contents corresponds to that what you’re currently consciously aware of. It has a limited capacity so you’re only consciously aware of a few things at any given moment.

At the same time, various subregions in your brain are doing their own things, some of them processing information that’s in the global workspace, some of them observing stuff from your senses that you’re currently not consciously aware of. Like you’re focused on a thing, then there’s a sudden sound, and some auditory processing region that has been monitoring the sounds in your environment picks it up and decides that this is important and pushes that sound into your global workspace, displacing whatever else happened to be there and making you consciously aware of that sound.

I tend to interpret IFS “parts” as processes that are connected with the workspace and manipulate it in different ways. But it’s not necessarily that they’re really “independent agents”, it’s more like there’s a combination of innate and learned rules for when to activate them.

So like, take it when an IFS book has a case study about a person with a “confuser” part that tries to distract them when they are thinking about something unpleasant. I wouldn’t interpret that to literally mean that there’s a sentient agent seeking to confuse the person in that person’s brain. I think it’s more something like. there are parts of the brain that are wired to interpret some states as uncomfortable, and other parts of the brain that are wired to avoid states that are interpreted as uncomfortable.

At some point when the person was feeling uncomfortable, something happened in their brain that made them confused instead, and then some learning subsystem in their brain noticed that “this particular pattern of internal behavior relieved the feeling of discomfort”. And then it learned how to repeat whatever internal process caused the feeling of confusion to push the feeling of discomfort out of the global workspace, and to systematically trigger that process when faced with a similar sense of discomfort.

Then when the IFS therapist guided the client to “talk to the confuser part”, they were doing something like. interfacing with that learned pattern and bringing up the learned prediction that causing confusion will lessen the feeling of discomfort.

There’s a thing where, once information that has been previously only stored in a local neural pattern is retrieved and brought to the global workspace, it can then be accessed and potentially modified by ...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Interested in EA/longtermist research careers? Here are my top recommended resources, published by MichaelA on June 26, 2022 on The Effective Altruism Forum.

In the past ~2.5 years, I've very often given career advice to people in the EA community. This post is my up-to-date, prioritized list of recommended resources for people who are interested in research careers, longtermism-aligned careers, and/or working at EA orgs (rather than e.g. high-impact roles at non-EA orgs that work on relevant topics). The more you're interested in each of those three things, the more likely it is that this post will be useful to you - but I expect it would also be useful to many people interested in just one or two of those things.

This post is also likely to be more useful to people who are earlier in their relevant career journeys, since many of the resources will be well-known or unnecessary for people with more relevant experience.

Will these resources be useful to you? Hard to say, as I don't know you or what you already know! But this post itself should only take a few minutes to read, and then you can make your own decision about which links to open and how closely to engage with them.

These resources should hopefully help you have an impact, build your career capital (e.g., knowledge, skills, connections, and credentials), & test your personal fit for various roles, and find other good advice or valuable opportunities for doing those things. This post won't itself give you many specific tips.

I've bolded the resources that I expect will typically be most useful.

Behold! The herald resources:

Things you could apply to

Note: I suggest you don’t spend too long thinking about the pros and cons of applying to an opportunity (e.g., a job, grant, degree program, or internship). Assuming the initial application wouldn’t take you long, if it seems worth thinking hard about, you should probably just apply instead.

I and Akash elaborate on this in Don’t think, just apply! (usually).

You could use the 80,000 Hours Job Board to find listings of jobs relevant to effective altruism, including jobs that could help in building skills and testing fit.

You could apply to EA-aligned research training programs or internships. You can find lists of such programs here and here.

I previously put together a list of EA funding opportunities, and noted: “I strongly encourage people to consider applying for one or more of these things. Given how quick applying often is and how impactful funded projects often are, applying is often worthwhile in expectation even if your odds of getting funding aren’t very high. (I think the same basic logic applies to job applications.)”

These funding opportunities could be used to support a very wide range of activities, such as research, career exploration and planning, community building, and entrepreneurship.

You could apply for coaching from Effective Thesis or check our their resources if you're working on a thesis or other piece of academic research or considering doing so.

You could subscribe to the Effective Altruism Newsletter to stay up to date with the effective altruism community and get updates about new job opportunities.

Other sources of career advice

80,000 Hours have a lot of valuable guidance on having more social impact with your career. In particular, I often suggest people check out their in-depth process and template for career planning and apply for their free career advising.

If you're reading this post, you've almost certainly heard of 80,000 Hours and know they have a career planning process and a career advising service. But I still find that many people who are keen to talk to me for career advice haven't actually worked through 80k's process or applied for their advising service. In most such cases, I strongly suggest actually doing so!

In 2...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: More funding is really good, published by Zach Stein-Perlman on June 25, 2022 on The Effective Altruism Forum.

Inspired by Matt Yglesias

For $5,000, you can save one child from malaria.

For $200, you can have a one-trillionth chance of preventing an existential catastrophe from pandemics.

For $5,000–$10,000, you can fund events, books, food, t-shirts, etc. for a small EA group for a year.

For $10,000, you can fund the marginal AI safety researcher for a month.

For $1, you can invest it to double its expected influence over the world in less than 20 years, assuming nothing crazy happens before then.

Let's say money is part of "EA funding" if the person/system directing it is roughly aiming at doing as much good as possible and considering options like these. Then marginal EA funding goes to interventions that the person/system directing it believes are at least as good as interventions like these. These interventions are really good. Therefore marginal EA funding is prima facie really good.

As long as there exist cost-effective interventions to throw money at, EA is funding constrained.

Actually, I can't find GiveWell's marginal cost-effectiveness estimates, but my sense is that they've found interventions with average cost-effectiveness better than $4,500 per child saved and that scale without much cost increase.

Open Philanthropy's last dollar project.

Assuming you can get at least 8% expected real returns per year, and the wealth of the rest of the world grows at at most 4%, and influence is proportional to your share of the world's wealth.

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.