Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: The Wicked Problem Experience, published by Holden Karnofsky on March 2, 2022 on The Effective Altruism Forum.

Cross-posted from Cold Takes

I’ve spent a lot of my career working on wicked problems: problems that are vaguely defined, where there’s no clear goal for exactly what I’m trying to do or how I’ll know when or whether I’ve done it.

In particular, minimal-trust investigations - trying to understand some topic or argument myself (what charity to donate to, whether civilization is declining, whether AI could make this the most important century of all time for humanity), with little reliance on what “the experts” think - tend to have this “wicked” quality:

I could spend my whole life learning about any subtopic of a subtopic of a subtopic, so learning about a topic is often mostly about deciding how deep I want to go (and what to skip) on each branch.

There aren’t any stable rules for how to make that kind of decision, and I’m constantly changing my mind about what the goal and scope of the project even is.

This piece will narrate an example of what it’s like to work on this kind of problem, and why I say it is “hard, taxing, exhausting and a bit of a mental health gauntlet.”

My example is from the 2007 edition of GiveWell. It’s an adaptation from a private doc that some other people who work on wicked problems have found cathartic and validating.

It’s particularly focused on what I call the hypothesis rearticulation part of investigating a topic (steps 3 and 6 in my learning by writing process), which is when:

I have a hypothesis about the topic I’m investigating.

I realize it doesn’t seem right, and I need a new one.

Most of the things I can come up with are either “too strong” (it would take too much work to examine them satisfyingly) or “too weak” (they just aren’t that interesting/worth investigating).

I need to navigate that balance and find a new hypothesis that is (a) coherent; (b) important if true; (c) maybe something I can argue for.

After this piece tries to give a sense for what the challenge is like, a future piece will give accumulated tips for navigating it.

Flashback to 2007 GiveWell

Context for those unfamiliar with GiveWell:

In 2007, I co-founded (with Elie Hassenfeld) an organization that recommends evidence-backed, cost-effective charities to help people do as much good as possible with their donations.

When we started the project, we initially asked charities to apply for $25,000 grants, and to agree (as part of the process) that we could publish their application materials. This was our strategy for trying to find charities that could provide evidence about how much they were helping people (per dollar).

This example is from after we had collected information from charities and determined which one we wanted to rank #1, and were now trying to write it all up for our website. Since then, GiveWell has evolved a great deal and is much better than the 2007 edition I’ll be describing here.

(This example is reconstructed from my memory a long time later, so it’s probably not literally accurate.)

Initial “too strong” hypothesis. Elie (my co-founder at GiveWell) and I met this morning and I was like “I’m going to write a page explaining what GiveWell’s recommendations are and aren’t. Basically, they aren’t trying to evaluate every charity in the world. Instead they’re saying which ones are the most cost-effective.” He nodded and was like “Yeah, that’s cool and helpful, write it.”

Now I’m sitting at my computer trying to write down what I just said in a way that an outsider can read - the “hypothesis articulation” phase.

I write, “GiveWell doesn’t evaluate every charity in the world. Our goal is to save the most lives possible per dollar, not to create a complete ranking or catalogue of charities. Accordingly, our research is oriented around identi...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: [Intro to brain-like-AGI safety] 6. Big picture of motivation, decision-making, and RL, published by Steve Byrnes on March 2, 2022 on The AI Alignment Forum.

6.1 Post summary / Table of contents

Part of the “Intro to brain-like-AGI safety” post series.

Thus far in the series, Post #1 set out some definitions and motivations (what is “brain-like AGI safety” and why should we care?), and Posts #2 & #3 split the brain into a Learning Subsystem (telencephalon and cerebellum) that “learns from scratch” using learning algorithms, and a Steering Subsystem (hypothalamus and brainstem) that is mostly genetically-hardwired and executes innate species-specific instincts and reactions.

Then in Post #4, I talked about the “short-term predictor”, a circuit which learns, via supervised learning, to predict a signal in advance of its arrival, but only by perhaps a fraction of a second. Post #5 then argued that if we form a closed loop involving both a set of short-term predictors in the Learning Subsystem and a corresponding set of hardwired circuits in the Steering Subsystem, we can get a “long-term predictor”. I noted that the “long-term predictor” circuit is closely related to temporal difference (TD) learning.

Now in this post, we fill in the last ingredients—roughly the “actor” and “model” parts of model-based actor-critic reinforcement learning (RL)—to get a whole big picture of motivation and decision-making in the human brain. (I’m saying “human brain” to be specific, but it would be a similar story in any other mammal, and to a lesser extent in any vertebrate.)

The reason I care about motivation and decision-making is that if we eventually build brain-like AGIs (cf. Post #1), we’ll want to build them so that they have some motivations (e.g. being helpful) and not others (e.g. escaping human control and self-reproducing around the internet). Much more on that topic in later posts.

Teaser for upcoming posts: The next post (#7) will walk through a concrete example of the model in this post, where we can watch an innate drive lead to the formation of an explicit goal, and to the adoption and execution of a plan to accomplish that goal. Then starting in Post #8 we’ll switch gears, and from then on you can expect substantially less discussion of neuroscience and more discussion of AGI safety (with the exception of one more primarily-neuroscience post towards the end).

Unless otherwise mentioned, everything in this post is “things that I believe right now”, as opposed to neuroscience consensus. (Pro tip: there is never a neuroscience consensus.) Relatedly, I will make minimal effort to connect my hypotheses to others in the literature, but I’m happy to chat about that in the comments section or by email.

Table of contents:

In Section 6.2, I’ll present a big picture of motivation and decision-making in the human brain, and walk through how it works. The rest of the post will go through different parts of that picture in more detail. If you’re in a hurry, I suggest reading to the end of Section 6.2 and then quitting.

In Section 6.3, I’ll talk about the so-called “Thought Generator”, comprising the dorsolateral prefrontal cortex, sensory cortex, and other areas. (For ML readers familiar with “actor-critic model-based RL”, the Thought Generator is more-or-less a combination of the “actor” and the “model”.) I’ll talk about the inputs and outputs of this module, and briefly sketch how its algorithm relates to neuroanatomy.

In Section 6.4, I’ll talk about how values and rewards work in this picture, including the Reward Prediction Error (RPE) signal that drives learning and decision-making in the Thought Generator.

In Section 6.5, I’ll go into a bit more detail about how and why thinking and decision-making needs to involve not only simultaneous comparisons (i.e., a mechanism for generat...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: AGI x-risk timelines: 10% chance (by year X) estimates should be the headline, not 50%., published by Greg Colbourn on March 1, 2022 on The Effective Altruism Forum.

Artificial General Intelligence (AGI) poses an existential risk (x-risk) to all known sentient life. Given the stakes involved (the whole world/future light cone), we should regard timelines of ≥10% probability of AGI in ≤10 years as crunch time, and -- given that there is already an increasingly broad consensus around this -- be treating AGI x-risk as an urgent immediate priority (not something to mull over leisurely as part of a longtermist agenda).Of course it's not just time to AGI that is important. It's also the probability of doom given AGI. A recent survey of people working on AI risk gives a median of 30% for "level of existential risk" from "AI systems not doing/optimizing what the people deploying them wanted/intended".

To borrow from Stuart Russell's analogy: if there was a 10% chance of aliens landing in the next 10-15 years, humanity would be doing a lot more than we are currently doing. AGI is akin to an alien species more intelligent than us that is unlikely to share our values.

Note that Holden Karnofsky's all-things-considered (and IMO conservative) estimate for the advent of AGI is >10% chance in (now) 14 years. Anecdotally, the majority of people I've spoke to on the current AGISF course have estimates for 10% chance of 10 years or less. Yet most people in EA at large seem to put more emphasis on the 50% estimates that are in the 2050-2060 range.

I originally wrote "..probability of doom given AGI. I think most people in AI Alignment would regard this as >50% given our current state of alignment knowledge and implementation","Correct me if you think this is wrong; would be interesting to see a recent survey on this", and was linked to a recent survey! Note that there is a mismatch with the framing in my post in that the survey implicitly incorporates time to AGI, for which the median estimate amongst those surveyed is presumably significantly later than 10 years. This suggests that P(doom|AGI in 10 years) would be estimated to be higher. It would be good to have a survey of the following questions:1. Year with 10% chance of AGI.2. P(doom|AGI in that year).

This is different to the original analogy, which was an email saying: "People of Earth: We will arrive on your planet in 50 years. Get ready." Say astronomers spotted something that looked like a space-craft, heading in approximately our direction, and estimated there was 10% chance that it was indeed a spacecraft heading to Earth.

Although perhaps we wouldn't. Maybe people would endlessly argue about whether the evidence is strong enough to declare a >10% probability. Or flatly deny it.

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Ukraine Situation Report 2022/03/01, published by lsusr on March 2, 2022 on LessWrong.

I love Russia, I love Russian and I love Russians. My favorite physics professor was a nuclear scientist from the Soviet Union. I studied Russian in college where a classmate gave me my first copy of Foreign Affairs Journal. Catherine the Great is #4 on my list of heroes.

I don't know much about Ukraine or Ukrainians, but it is hard not to love them too after the swagger and humor they have exhibited since the Russian invasion.

A week ago I wrote "The Russian Armed Forces is among the three most capable militaries in the world". Since then, I have been astonished by the incompetence of the Russian Armed Forces.

Russian Equipment

When I found out Ukraine issued 10,000 automatic rifles to civilians my first thoughts were "Those rifles could arm an insurgency but using them in conventional battle against Russian forces would be suicide". That's because there's more to fielding an effective soldier than giving a man or woman a gun. He or she needs to know what direction to point it.

Here is an interesting comment thread between people who (I assume) are mostly American infantry.

Having heard over the past 18-24 months about LSCO [Large Scale Combat Operations] and russia, I have to admit they have been exposed pretty badly. Also one of my friends noted during all of the combat footage we have not seen any night infantry combat, let alone any IR lasers. Now objectively I haven’t seen every single piece of combat footage and presumably there is night combat. ―regularguyofthenorth

I saw a report from Ukraine that said that Russian forces wait until dawn to attack because they don't have NVGs, nor the training on using them, not sure how accurate that is. ―IllustriusDot1866

Everything I've seen has said the same. No NODs, no optics, for anyone but leadership positions in infantry units. That's insane to me. I can't imagine life in a combat zone without NODs or optics. ―bang_the_drums

That was something that stuck out to me too - even for the UA. I see maybe 1:20 with anything but an iron sight and a dream. ―Roastage

I've seen an AS VAL [a Soviet assault rifle] and a VSS Vintorez [a silenced Soviet sniper rifle] as "combat trophies" so far, neither one of them had optics, which basically defeats the point of a suppressed gun shooting subsonic ammo that can penetrate armor.

I also saw the VDV [Russian Airborne Forces] guys at the Hostomel airport, one of them had an AK-12 with no optic, no light, no laser, nothing. "Next gen" AK with ironsights. WTF? ―xyolikesdinosaurs

Yep more or less the same for me. Expected for the Ukrainian Militia but surprising to see so few amongst the Russian infantry. There is a photo on combatfootage of some SF [Special Forces] guys who were fully kitted out with MCXs [gun] and all the jewellery. Unclear if they were FOG but there was more glass in that photo than I'd seen in all the footage to date. ―Roastage

―reddit thread

"IR" stands for "infrared [nightvision]". "NVG" stands for "Night Vision Goggles". "NOD" stands for "Night Observation Device". These American troops are shocked to discover their Russian counterparts cannot see at night. My bicycle has better optics than Russia's paratroopers. American door-kickers in Iraq and Afghanistan operated at night because the best time to do battle is when you can see and the enemy can't.

How can Russia's frontline troops be missing such basic equipment? Because Russia has a GDP the size of Florida's. Subtract oligarchic graft and what's left is Soviet hardware.

When I found out the Ukrainians were defacing signs I thought "that's the right thing to do but I expect it won't do much because Russians can navigate via GPS instead.

It appears unlikely that the majority of Russian military armored vehicles are using GPS, ...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Musings on the Speed Prior, published by Evan Hubinger on March 2, 2022 on The AI Alignment Forum.

Thanks to Paul Christiano, Mark Xu, Abram Demski, Kate Woolverton, and Beth Barnes for some discussions which informed this post.

In the ELK report, Paul, Mark, and Ajeya express optimism about penalizing computation time as a potentially viable way to select the direct translator over the human imitator:

Human imitation requires doing inference in the entire human Bayes net to answer even a single question. Intuitively, that seems like much more work than using the direct translator to simply “look up” the answer.

Compared to all our previous counterexamples, this one offers much more hope. We can’t rule out the possibility of a clever dataset where the direct translator has a large enough computational advantage to be preferred, and we leave it as an avenue for further research.

I am more skeptical—primarily because I am more skeptical of the speed prior's ability to do reasonable things in general. That being said, the speed prior definitely has a lot of nice things going for it, and I do think it's worth taking a careful look at both the good and the bad that the speed prior has to offer. Conceptually, what we want to pay attention to when evaluating a prior from an AI safety perspective is threefold: it needs to favor good models over bad models (e.g. the direct translator over the human imitator), it needs to be competitive to implement, and it needs to favor models with good generalization over models with bad generalization (e.g. the resulting models need to themselves be performance competitive).

Before I do that, however, an important preliminary: there are multiple different forms/types of speed priors, so when I say “the speed prior,” I really mean a class of priors including:

various combinations of circuit-size and circuit-depth complexity and

various combinations of Turing machine time and description complexity, where Turing machine time complexity can be measured either by taking the max number of steps taken across all inputs or the average number of steps taken on a particular distribution of inputs.

The basic structure of this post will be a dialog of sorts between a pro-speed-prior and an anti-speed-prior perspective. I'll start with some of the arguments in favor of the speed prior and how the anti-speed-prior perspective might respond, then give some arguments against the speed prior and how the pro-speed-prior perspective might respond.

Why you should love the speed prior

As in the ELK report, there is a plausible argument for why there exists a speed prior that would prefer the direct translator to the human imitator. Naively, the problem with the speed prior here is that the computation required for the human imitator is proportional to the size of the human's Bayes net, whereas the computation required for the direct translator is proportional to the size of the model's Bayes net—and in the superhuman limit we should expect the latter to be substantially larger than the former. The argument in the ELK report, however, is that while this argument is valid in the limit, there's reason to believe it might be invalid for all the finite cases that we care about. That's because perfect inference in either Bayes net, and thus perfect loss, shouldn't be possible in any finite case.

Thus, the performance of the ontology mapping function, and thus its loss, should be proportional to how much computation it puts into its inference task—for which the direct translator has a big advantage, since it gets to reuse the computation performed by the model.

Response: The obvious response here, and the response that is given in the ELK report, is that the above argument is very fragile—it relies on inference in the human's Bayes net being too hard to always get r...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Have You Tried Hiring People?, published by rank-biserial on March 2, 2022 on LessWrong.

The purpose of this post is to call attention to a comment thread that I think needs it.

Wait, I thought EA already had 46$ billion they didn't know where to spend, so I should prioritize direct work over earning to give?/

I thought so too. This comment thread on ACX shattered that assumption of mine. EA institutions should hire people to do "direct work". If there aren't enough qualified people applying for these positions, and EA has 46 billion dollars, then their institutions should (get this) increase the salaries they offer until there are.

To quote EY:

There's this thing called "Ricardo's Law of Comparative Advantage". There's this idea called "professional specialization". There's this notion of "economies of scale". There's this concept of "gains from trade". The whole reason why we have money is to realize the tremendous gains possible from each of us doing what we do best.

This is what grownups do. This is what you do when you want something to actually get done. You use money to employ full-time specialists.

Excerpts From The ACX Thread:[1]

Daniel Kokotajlo:

For those trying to avert catastrophe, money isn't scarce, but researcher time/attention/priorities is. Even in my own special niche there are way too many projects to do and not enough time. I have to choose what to work on and credences about timelines make a difference.

Daniel Kirmani:

I don't get the "MIRI isn't bottlenecked by money" perspective. Isn't there a well-established way to turn money into smart-person-hours by paying smart people very high salaries to do stuff?

Daniel Kokotajlo:

My limited understanding is: It works in some domains but not others. If you have an easy-to-measure metric, you can pay people to make the metric go up, and this takes very little of your time. However, if what you care about is hard to measure / takes lots of time for you to measure (you have to read their report and fact-check it, for example, and listen to their arguments for why it matters) then it takes up a substantial amount of your time, and that's if they are just contractors who you don't owe anything more than the minimum to.

I think another part of it is that people just aren't that motivated by money, amazingly. Consider: If the prospect of getting paid a six-figure salary to solve technical alignment problems worked to motivate lots of smart people to solve technical alignment problems... why hasn't that happened already? Why don't we get lots of applicants from people being like 'Yeah I don't really care about this stuff I think it's all sci-fi but check out this proof I just built, it extends MIRI's work on logical inductors in a way they'll find useful, gimme a job pls." I haven't heard of anything like that ever happening. (I mean, I guess the more realistic case of this is someone who deep down doesn't really care but on the exterior says they do. This does happen sometimes in my experience. But not very much, not yet, and also the kind of work these kind of people produce tends to be pretty mediocre.)

Another part of it might be that the usefulness of research (and also manager/CEO stuff?) is heavy-tailed. The best people are 100x more productive than the 95th percentile people who are 10x more productive than the 90th percentile people who are 10x more productive than the 85th percentile people who are 10x more productive than the 80th percentile people who are infinitely more productive than the 75th percentile people who are infinitely more productive than the 70th percentile people who are worse than useless. Or something like that.

Anyhow it's a mystery to me too and I'd like to learn more about it. The phenomenon is definitely real but I don't really understand the underlying causes.

Melvin:

Consid...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: RFC WWIII, published by adamzerner on March 1, 2022 on LessWrong.

You should ignore the news unless it's of historic import. Russia's invasion of Ukraine constitutes an event of historic import.

Russia has Invaded Ukraine

One could argue for an even stronger position: you should ignore the news unless it 1) affects you and 2) there is something that you could do about it. I'm trying to think about whether 1 and 2 are true. Like most of us, I have some thoughts, but ultimately I'm not a geopolitics person and don't really know what I'm talking about. And so, this post is a request for comments[1], not an authoritative write-up.

May as well start now

Suppose we have the best case scenario: the war ends, tensions disappear, and we all go back to our lives. How long will that last? How long until tensions get serious again? 1 year? 5? 10? 25? 50? 100?

I lean towards the earlier end of that spectrum. 80,000 Hours would "guess the chance of a nuclear war is 2-20% in the next 200 years". Using that as a jumping off point, the chance of tensions developing enough where the threat is notable seems like it'd be a lot higher, especially given the current invasion.

Even if your estimate is towards the later end of this spectrum, it still seems like the sort of thing we will need to deal with in our lifetimes at some point. So then, any effort spent educating oneself and preparing right now probably won't be wasted. It's like buying honey for your pantry: it doesn't expire, and you know you will use eventually.

Decide what level of risk you are ok with ahead of time

There has been some talk about this in the context of covid. That you should decide ahead of time at what level of case counts you'd be ok with returning to eg. outdoor dining? What level for indoor dining? What level for large indoor gatherings?

Because if you don't, you risk some sort of status quo bias. And similarly, you risk the boiling frog thing happening to you. The case counts just slowly get lower and lower and lower, but each change is too small/gradual to get you to take action.

I think a similar principle makes sense in the context of nuclear risk. I could see tensions escalating, and escalating, and escalating, and escalating, but each escalation seems too small to justify an action like moving from the city to the country. To guard against this, I think it'd make sense to have some, preferably quantitative, idea in your mind about when you would take various actions.

What can an individual actually do to lower their risk?

I don't have a great grasp on this question, but I can think of a few things.

Moving to a remote location

I'm thinking of remote islands like Tristan da Cunha that are aparently inhabitable and sustainable. Making a move like this would be a pretty extreme option, so the risk would have to justify it. But it's hard for me to imagine a remote location like this getting hit by a nuke. And who knows, if you look on the bright side, maybe it'd be a good change of pace and life experience.

Moving to a different country

What I have in mind here are places like Iceland and South Africa. It wouldn't be quite as safe as living in a remote location, but it seems to me that countries like these are significantly safer than places like the US and western Europe.

Moving to the countryside

As an example, I live in Portland, Oregon right now. Which is a city. If I moved out east to the countryside, or even to a place like Bend, OR, judging by maps like these and these, I'd be a lot safer, because Portland is more of a target. This probably wouldn't be as safe as moving to a different country, but moving to a different country would be a lot more inconvenient.

Building a bomb shelter

Imagine that you are interested in self-defense. You could learn how to punch and kick and wrestle and fight. That wou...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Introducing 80k After Hours, published by 80000 Hours on March 1, 2022 on The Effective Altruism Forum.

This is a linkpost for/

As The 80,000 Hours Podcast has developed over the last few years, we’ve found ourselves experimenting more with different types of content.

Things like audio versions of some of our best articles, or less formal chats between team members, or novel topics (like our episode on having a successful career with depression, anxiety, and imposter syndrome).

But different types of content have different audiences, and some people who love classic episodes with Rob will understandably have no interest in experimental episodes with other folks on the team (or vice versa: maybe some people love everyone else on the team but can’t stand Rob!)

That’s why we’ve created 80k After Hours.

It’s a new podcast that includes a much wider variety of content than you’ve come to expect from the original feed.

It’ll mostly still explore the best ways to do good — and some episodes will be much more laser-focused on careers than most original episodes — but we’re going to feel more comfortable with throwing things up there just because they’re fun or entertaining too.

We’ll also feel fine with producing some content for much narrower audiences.

One of our inaugural episodes with Alex Lawsen won’t be for everyone. But if you’re a student, a teacher, or a parent, it might be one of the highest-value things we can offer.

And sometimes we’ll release content that’ll be hard to follow unless you’ve seriously engaged with effective altruism — whereas we usually try to make original episodes much more widely accessible.

So rather than being “extras from the original 80,000 Hours Podcast,” 80k After Hours is its own thing. We’re excited to hear what you think.

We've launched with 3 episodes:

Alex Lawsen on his advice for students

Rob and Keiran on the philosophy of The 80,000 Hours Podcast

An audio version of ‘Be more ambitious: a rational case for dreaming big’

You can subscribe to it anywhere you listen to The 80,000 Hours Podcast, including:

Apple, Spotify, Google, and Podcast Addict.

We’ll be unusually open to requests for new formats or topics for 80k After Hours, so if you have any feedback or suggestions, please reach out to [email protected].

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: .01% Fund - Ideation and Proposal, published by Linch on March 1, 2022 on The Effective Altruism Forum.

Pronounced: Basis Fund. Alt: Point-O-One-Percent Fund

Summary

The basic idea is for a new funding agency, or a subproject within existing funding agencies, to solicit proposals for research or startup ventures that, assuming everything goes well, can reduce existential risk by > 0.01 percentage points (pp) at a price point of between 100M-1B/0.01% of absolute xrisk reduced (including both financial costs and reasonable models of human capital).

The two main advantages of having a specific fund or subfund that focuses on this are:

Memetic call to increase ambition and quantification: Having an entire fund focused on projects that can reduce existential risk by >0.01% can help encourage people to look really hard and actively for potentially really amazing projects, rather than “settle” for mediocre or unscalable ones.

Assessor/grantmaker expertise: Having a fund that focuses on specific quantitative models of this form helps to develop individual and institutional expertise in evaluating the quality of grants a) quantitatively and b) at a specific (high) tier of ambition.

More specifically, we give initial seed funding in the between 30k - 10M region for projects that

have within-model validity,

where we don’t detect obvious flaws, and

seem healthily devoid of very large (EV) downside risks.

The idea is that initially successful projects will have an increasingly strong case for impact later, and then can move on to larger funders like Open Phil, Future Fund, SFF, Longview, etc.

We don’t care as much about naive counterfactual impact at the initial funding stage, as (for example) projects that can save other people’s time/$s to do other important things are also valuable. We will preferentially fund projects that other people either a) are not doing or b) not doing well.

The fund will primarily fund areas with more strategic clarity than AI safety or AI governance, as the “reduce x-risk by 0.01 percentage points” framing may be less valuable for those ventures. We may also fund projects dedicated towards increasing strategic clarity, especially if we think the arguments for their research directions feel qualitatively and quantitatively compelling.

What does this funding source do that existing LT sources don’t?

Point people heavily at a clear target.

I feel like existing ventures I see, including both public EAIF/LTFF grants and the small number of grants/research questions that come across my desk at RP to (implicitly) evaluate, rarely have clear stories and never have numbers for “assuming this goes extremely well, we’d clearly reduce xrisk by X% through ABC”

Having a clear target to aim for is often helpful

Force quantification

Quantification is often good, see (ironically) qualitative arguments here.

Provide both a clear source of funding and specialization/diversification of grantmakers.

Grantmakers here can focus on assessing quantitative claims of a certain ambition bar for seed funding, while leaving qualitative claims, non-seed funding, and grant claims of much lower ambition to other grantmakers.

How will we determine if something actually reduces x-risk by 0.01pp if everything goes well?

We start with high-level estimates of xrisk from different sources, such as as Michael Aird’s Database of existential risk estimates or Ord’s book, as well as someone (maybe Linch? maybe a reader?) who’s good at evaluating quantitative models to look at the within-model validity of each promising grant. A process such as the one outlined here could also be used when attempting quantification.

As the fund progresses and matures, we may have increasingly more accurate and precise high-level quantitative estimates for levels of xrisk from each source (through a network of a...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Announcing CEA’s University Group Accelerator Program (UGAP), published by jessica mccurdy on March 1, 2022 on The Effective Altruism Forum.

The University Group Accelerator Program (UGAP) is a program run by the Centre For Effective Altruism that aims to take a university group from a couple of interested organizers to a mid-sized new EA group. It offers stipends for main organizers, regular meetings with an experienced mentor, trainings, and useful resources to run your first intro seminar (or fellowship) program.

The next round of the program will begin in September and we expect to open applications in the early summer. However, we think it is great for potential organizers to start thinking early about whether they would like to participate and take action this semester to prepare the group to apply.

Further information can be found on this page which includes information on:

How to prepare to be ready to participate in UGAP

How to express interest

Stipends

Qualifications

And more!

We are also looking for additional experienced organizers to serve as mentors. You can find out more about and express interest in becoming a UGAP mentor here.

30 groups have participated this semester across 5 continents and we are eager to expand this Summer!

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.