Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: The curious case of Pretty Good human inner/outer alignment, published by PavleMiha on July 5, 2022 on LessWrong.

I have been convinced to believe that looking at the gap between human inner and outer alignment is a good way to think about potential inner/outer alignment problems in artificial general intelligences:

We have an optimisation process (evolution) trying to propagate genes, that created a general intelligence (me/you). For millions of years our inner goals of feeling really good would also satisfy evolution’s outer goal of propagating genes, because one of the things that feels the best is having sex. But eventually that intelligent agent figured out how to optimise for things that the outer optimisation process didn’t want, such as having protected sex or watching VR porn, thus satisfying the inner goal of feeling really good, but not the outer goal of propagating genes. This is often told as a cautionary tale: we only know of one General Intelligence and it’s misaligned. One day we will create an Artificial General Intelligence (AGI) and we will give it some sort of (outer) goal, and it might then develop an inner goal that doesn’t directly match what we intended. I think this only tells half the story.

Even though our general intelligence has allowed us to invent condoms and have sex without the added cost of children, a surprising amount of people decide to take them off because they find it fun and meaningful to have children.

In a world where we could choose to spend all our time having protected sex or doing drugs, a lot of us choose to have a reasonable number of kids and spend our time on online forums discussing AI safety, all of which seem to satisfy a more longer term version of “propagate your genes” than simply wanting to have sex because it feels good. More than that, we often choose to be nice in situations where being nice is even detrimental to the propagation of our own genes. People adopt kids, try to prevent wars, work on wildlife conservation, spend money on charity buying malaria nets across the world, and more.

I think there are two questions here: why are human goals so sophisticatedly aligned with propagating our genes and why are humans so nice.

Most people want to have kids, not just have sex. They want to go through the costly and painful process of childbirth and child rearing, to the point where many will even do IVF. We’ve used all of our general intelligence to bypass all the feels-nice-in-the-moment bits and jump straight to the “propagate our genes” bit. We are somehow pretty well aligned with our unembodied maker’s wishes.

Humans, of course, do a bunch of stuff that seems unrelated to spreading genes, such as smoking cigarettes and writing this blog post. Our alignment isn’t perfect, but inasmuch as we have ended up a bit misaligned, how did we end up so pleasantly, try-not-to-kill-everything misaligned?

The niceness could be explained by the trivial fact that human values are aligned with what I, a human, think is nice: humans are pretty nice because people do things I understand and empathize with. But there is something odd about the fact that most people, if given the chance, would pay a pretty significant cost to help a person they don’t know, keep tigers non-extinct, or keep Yosemite there looking pretty. Nice is not a clear metric, but why are people so unlike the ruthless paperclip maximisers we fear artificially intelligent agents will immediately become?

Hopefully I’ve convinced you that looking at human beings as an example of intelligent agent development is more interesting than purely as an example of what went wrong, it is also an interesting example of some things going right in ways that I believe current theory on AI safety wouldn’t predict. As for the reasons why human existence has gone as well as it...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Carrying the Torch: A Response to Anna Salamon by the Guild of the Rose, published by moridinamael on July 6, 2022 on LessWrong.

In a recent comment and followup post, Anna Salamon described some of the rocks upon which the Center for Applied Rationality has run aground, and invited anyone interested in the project of improving human rationality to pick up the torch.

We wanted to take this opportunity to remind you that the Guild of the Rose is here to carry that torch onward.

On a personal note, I figured I ought to also explicitly mention that the Guild is, in no small part, the result of many parties from all over the world reading and responding to the bat signal that I shined into the sky in 2018. In that post, an alien visitor with a rationality textbook exhorted us to dream a little bigger regarding the potential of the rationality movement. If that post spoke to you, then you ought to know it spoke to a lot of people, and, well, the Guild is what we're doing about it together.

And I think we're doing a pretty good job! Anna outlined a list of problems that she felt CFAR ran into, and I figured this would be a good place to describe how the Guild dealt with each of those problems.

Wait a minute – dealt with those problems? Anna just posted a few weeks ago!

When we started the Guild in 2020, we looked to CFAR as both an example to emulate and also differentiate ourselves from. We diagnosed many of the same problems that Anna describes in her post, though not necessarily in the same framing. We designed our organization to avoid those problems. We are grateful to CFAR for having pioneered this path.

Problem 1:

Differentiating effective interventions from unfalsifiable woo.

The Guild focuses on actions and habits, not psychotherapy. I think ~0% of what we teach can be called unfalsifiable woo. Even when we tread more esoteric ground (e.g. the decision theory course) we focus on the practical and implementable.

To sketch a perhaps not-totally-generous metaphor, imagine there are two martial arts schools you're trying to choose between:

One of these schools is esoteric, focuses on dogmatic cultivation of Warrior Spirit, and demands intensive meditation. This school promises a kind of transcendent self-mastery and hints that ki-blasts are not entirely off the table.

The other school focuses on punching bags, footwork drills, takedowns, and sparring. This school promises that if you need to throw hands, you'll probably come out of it alive.

I think the vast majority of Human Potential Movement-adjacent organizations look more like the first school. Meanwhile, boring-looking organizations like the Boy Scouts of America, which focus almost entirely on the pragmatic practices of how to tie knots and start fires using sticks, probably succeed more at actually cultivating the "human potential" of their members.

Thus, the Guild focuses on the pragmatic. Our workshops cover effective, "boring" interventions like better nutrition, using your speaking voice more effectively, improving your personal financial organization, emergency preparedness, and implementing a knowledge management system, among many others. There is a new workshop almost every week.

Of course, we also teach what could be considered explicit rationality training. We have workshops focusing on epistemics, and on practical decision theory. But it's our belief that one way to become exceptional is to simply not be below-average at anything important. It is sort of embarrassing to focus extremely hard on "having correct beliefs" while not having a basic survival kit in your car.

Problem 2:

Instructors having impure motives, and mistakes amplifying when it comes to rewiring your brain.

We have put some policies in place to mitigate against this kind of thing.

We mitigate against the risks of rewiring members' b...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: My vision of a good future, part I, published by Jeffrey Ladish on July 6, 2022 on LessWrong.

A future where everyone is okay

I grew up with the future feeling pretty uncertain. God was supposed to return at some point, but maybe there were going to be The End Times before that? Like, a great persecution of the Christians, and maybe something about the bottomless pit and the oceans turning to blood. I no longer believe that, but I can still relate to a lot of angst I see people express about the future, especially in regards to AI, in my own circles, and climate change, in circles a little further out.And yes, I’m quite worried about existential risks from AI. I think the default outcome is that we build systems vastly smarter and more powerful than us, and those systems destroy us because our survival is not conducive to their divergent goals. But I do not believe our destruction is guaranteed. We might succeed at aligning the goals of these vastly more powerful intelligences with our own, and it’s that outcome that inspires me to fight for the future I believe in.

We might succeed at building AI systems that possess both vast amounts of optimization power and a finely tuned concern for human well being and flourishing. If we get this right, the future could be almost unimaginably good. But not entirely unimaginable.

I want to try to imagine it, concretely, because I care about what I’m fighting for. I’m not just fighting against extinction, I’m fighting for something that could really come to pass, something amazing. So I want to know what that might look like.

In brief, the good world:

In the good future I imagine, everyone is okay. Every person has enough good food to eat and water to drink. No one has dementia, cancer, or any disease. There’s no material scarcity in the sense that there is today; anyone can live a billionaire’s lifestyle if they want to. The root causes of social problems have been treated. People still have difficulties from time to time, but everyone is set up to succeed, find romantic partners, and have a rich and fulfilling social life. People tend to be really happy most of the time.

And that’s just the beginning. Once we’ve solved the major problems, that’s where the fun begins. I have quite a list of things I want to do in the future. Countless countries, cities, and bioregions and I want to explore. I want to write science fiction novels. I want to speak new languages, meet new people, build things with my friends and family. The wild thing is, I think we just get to do all this if we succeed at kicking off a good future. And most of the things I’ve imagined have been normal human things, on our earth. The universe is vast and, as far as we know, empty. We can fill the universe with a staggering amount of new things, new kinds of minds and experiences, wonders beyond our limited comprehension. I’ll write about these possibilities in the next post. First, I want to imagine getting to a world where everyone is okay.

The problems to solve

He will wipe every tear from their eyes. Death will be no more; mourning and crying and pain will be no more, for the first things have passed away.” Revelation 21:4

I did not grow up believing that death and suffering were natural conditions of the world. I was taught these were a corruption, the result of sin entering our world after Adam and Eve sinned. I don’t believe this anymore, but I still believe these things are awful, that vast quantities of death and suffering mar our world.

I relate to suffering in a few different ways. The personal ways are often most salient. I’ve struggled with loneliness and stress, and I’ve seen a great deal of friends struggle with depression, anxiety, and loss. I see homeless people on the streets in the city, suffering in the cold, ignored by most, just barely survivin...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Introducing the Fund for Alignment Research (We're Hiring!), published by AdamGleave on July 6, 2022 on The Effective Altruism Forum.

Cross-posted to LessWrong

The Fund for Alignment Research (FAR) is hiring research engineers and communication specialists to work closely with AI safety researchers. We believe these roles are high-impact, contributing to some of the most interesting research agendas in safety. We also think they offer an excellent opportunity to build skills and connections via mentorship and working closely with researchers at a variety of labs.

We were inspired to start FAR after noticing that many academic and independent researchers wanted to work with more collaborators, but often lacked the institutional framework and hiring pipeline to do so. On the other hand, there are a large number of talented people who would like to contribute to AI safety, but lack a stepping stone into the field. Our hope is that FAR can help fill this gap, both directly accelerating valuable research and helping to address the talent pipeline.

In the remainder of this post we'll give a bit more information about FAR and our current collaborators, and then summarize our current openings. Please consider applying or forwarding them to a friend who might be interested! We are also actively brainstorming other ways that FAR could be useful to the community. If you have any ideas, we’d love to hear from you!

About Us

FAR is a non-profit led by Ethan Perez, Adam Gleave, Scott Emmons, and Claudia Shi: a group of AI safety researchers looking to reduce existential risk from artificial intelligence. Ethan recently graduated from the PhD program at New York University, Adam and Scott are PhD candidates at UC Berkeley, and Claudia is a PhD candidate at Columbia University.

FAR provides services to AI safety researchers to accelerate their research agendas. We are currently focused on supporting the agendas of Ethan, Adam, Scott, and Claudia. We are also trialing a collaboration with the labs of David Kruger and Jacob Steinhardt, professors at the University of Cambridge and UC Berkeley.

Our services are currently provided free-of-charge to recipients out of FAR's general support funding. In the future we plan to charge partners who use large quantities of our services on an at-cost basis. This could be paid for from a partner's existing grant, or we can also assist the partner with fundraising for this purpose.

We anticipate supporting many of the same people that BERI currently works with. However, our organisations have differing emphases. First, our core services are different: to a first approximation, BERI provides "operations as a service" whereas FAR provides "a technical team as a service". That is, FAR recruits, manages and trains our own team; whereas BERI hires people primarily at the request of their partner's. Second, FAR works primarily with individual researchers whereas BERI works primarily with entire labs, although this distinction may blur in the future. Finally, FAR is more opinionated than BERI: if we have more demand for our services than our team can support, then we will prioritize based on our internal view of which agendas are most promising.

Although FAR is a new organization, our research has already led to a method for learning from language feedback as a data-efficient alternative to RL from human feedback. We have analyzed challenges associated with treating a language model as an RL policy and launched a competition on inverse scaling for language models. We are currently pursuing several other early-stage AI safety projects. Once we have beta-tested our model, we plan to expand the number of partners we work with. Feel free to get in touch at [email protected] if you think you might be a good fit!

Operations Manager

We are seeking an Operat...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: A Pattern Language For Rationality, published by Vaniver on July 5, 2022 on LessWrong.

There's a lens to looking at the rationality project that I've been finding enlivening recently, and I think it's reached the point where more eyes and hands might be useful, while not being anywhere near complete yet. First, some background.

Christopher Alexander was a designer and architect; his thinking, and focus on patterns in particular, were influential in programming; wikis were first invented to facilitate the collaborative creation and modification of 'patterns' in the style he described. He wrote lots of books, but I'll focus on this trio:

The Timeless Way of Building (about the ‘quality without a name’, and good vs. bad buildings)

A Pattern Language: Towns, Buildings, Construction (about the 'patterns' they’ve identified for physical buildings and towns)

The Oregon Experiment (about the University of Oregon, and how it could replace a ‘master plan’ with principles)

The three books, while each on a different subject or layer of 'design', all work together and depend on each other. The first identifies the target, why it would even be good to pursue, and how you know whether or not you've found it. The second is a detailed description of the patterns they've found useful in approaching the target. The third is what it looks like to organize systems to deliberately and durably organize themselves according to this design.

Just like this approach was profitably translated to programming (and other areas of design), I think it's worth looking at the 'rationality' project as a way to design decisions, habits, and thinking, and attempting to deliberately incorporate Alexander's approach and strategy. This post is an attempt to get started in each of the three directions, rather than fully lay them out; depending on how things go, I might turn this post into a sequence / flesh out individual sections.

The Timeless Way of Living

The Tao that can be told is not the eternal Tao; The name that can be named is not the eternal name.

[Consider also 'thinking', 'deciding', 'doing', and 'being' in place of 'living', for both this section and the next.]

Alexander talks about the “Quality Without A Name”, so-called because no existing English word was a good fit, altho he could point towards it pretty effectively in a few pages; you should read the start of The Timeless Way of Building (Amazon, pdf) to get his sense of it. I think “equilibrium” comes somewhat close–a design has the Quality Without A Name if all of the forces present, both internal and external, are balanced. This isn’t the true name because many equilibria we see don’t have the Quality Without A Name, because for them only some of the forces are active in determining the level. A house might be at ‘equilibrium’ according to the windows and the thermostat, but not according to the human inside who’s not happy with the situation and wants to do something about it.

When talking about buildings, he talks about whether they're 'alive' or 'dead'. His overall sense is that many design features are 'obvious' or 'natural'; while there might be lots of detail to the model of how to make things that are alive, most of the ability to detect whether or not a building is alive or dead is already 'baked in' to being a human.

I think it's relatively easy to point to good buildings and bad buildings, and somewhat harder to point to good lives and bad lives, mostly because we can walk around the inside of building made by others but not their minds. Nevertheless, it seems possible to collect pictures of what it is that makes life worth living, what properties good decisions have, what virtues we might like to embody. This is, historically, a place I think the rationality project has done pretty well.

You may try to name the highest princip...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Introducing the Fund for Alignment Research (We're Hiring!), published by AdamGleave on July 6, 2022 on The AI Alignment Forum.

Cross-posted to the EA Forum

The Fund for Alignment Research (FAR) is hiring research engineers and communication specialists to work closely with AI safety researchers. We believe these roles are high-impact, contributing to some of the most interesting research agendas in safety. We also think they offer an excellent opportunity to build skills and connections via mentorship and working closely with researchers at a variety of labs.

We were inspired to start FAR after noticing that many academic and independent researchers wanted to work with more collaborators, but often lacked the institutional framework and hiring pipeline to do so. On the other hand, there are a large number of talented people who would like to contribute to AI safety, but lack a stepping stone into the field. Our hope is that FAR can help fill this gap, both directly accelerating valuable research and helping to address the talent pipeline.

In the remainder of this post we'll give a bit more information about FAR and our current collaborators, and then summarize our current openings. Please consider applying or forwarding them to a friend who might be interested! We are also actively brainstorming other ways that FAR could be useful to the community. If you have any ideas, we’d love to hear from you!

About Us

FAR is a non-profit led by Ethan Perez, Adam Gleave, Scott Emmons, and Claudia Shi: a group of AI safety researchers looking to reduce existential risk from artificial intelligence. Ethan recently graduated from the PhD program at New York University, Adam and Scott are PhD candidates at UC Berkeley, and Claudia is a PhD candidate at Columbia University.

FAR provides services to AI safety researchers to accelerate their research agendas. We are currently focused on supporting the agendas of Ethan, Adam, Scott, and Claudia. We are also trialing a collaboration with the labs of David Kruger and Jacob Steinhardt, professors at the University of Cambridge and UC Berkeley.

Our services are currently provided free-of-charge to recipients out of FAR's general support funding. In the future we plan to charge partners who use large quantities of our services on an at-cost basis. This could be paid for from a partner's existing grant, or we can also assist the partner with fundraising for this purpose.

We anticipate supporting many of the same people that BERI currently works with. However, our organisations have differing emphases. First, our core services are different: to a first approximation, BERI provides "operations as a service" whereas FAR provides "a technical team as a service". That is, FAR recruits, manages and trains our own team; whereas BERI hires people primarily at the request of their partner's. Second, FAR works primarily with individual researchers whereas BERI works primarily with entire labs, although this distinction may blur in the future. Finally, FAR is more opinionated than BERI: if we have more demand for our services than our team can support, then we will prioritize based on our internal view of which agendas are most promising.

Although FAR is a new organization, our research has already led to a method for learning from language feedback as a data-efficient alternative to RL from human feedback. We have analyzed challenges associated with treating a language model as an RL policy and launched a competition on inverse scaling for language models. We are currently pursuing several other early-stage AI safety projects. Once we have beta-tested our model, we plan to expand the number of partners we work with. Feel free to get in touch at [email protected] if you think you might be a good fit!

Operations Manager

We are seeking an Operation...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Emphasizing emotional altruism in effective altruism, published by MichelJusten on July 5, 2022 on The Effective Altruism Forum.

Thank you to Emma Abele, Lennart Justen, Lara Thurnherr, Nikola Jurković, Thomas Woodside, Aris Richardson, Jemima Jones, Harry Taussig, Lucas Moore, and James Aung for inspiration and feedback. All mistakes are mine.

Summary

I don’t think powerful emotions related to the desire to do good in the world (e.g. feelings of awe, outrage, or fear) surface enough in EA relative to how important they can be for some people.

I argue that we should give such feelings of ‘emotional altruism’ more consideration.

Possible benefits of a greater emphasis on emotional altruism include: Improving internal community health; Accelerating EA engagement; Reaffirming healthy motivations; Creating an inspiring work setting; Conveying a more sympathetic community image; and Instilling personal well-being/ purpose.

Ways to affirm emotional altruism include: More within community discussion; Talking more about beneficiaries in outreach; Crafting experiences that allow powerful emotions to bubble up; More powerful writing; More videos and virtual reality content; New creative artwork; and Deliberately having standout conversations.

Introduction

Effective altruism grapples with profound themes. Themes like suffering at unimaginable scales, futures that transcend our notions of what “good” could look like, and the end of everything we’ve ever known do not occupy the realm of ordinary thoughts. And for many, myself included, these themes do not inspire ordinary emotions. Rather, they are entangled with deep, visceral emotions – emotions that can cast us into deep reflection, that can bring us to tears, and that may underlie the very reason we are in EA.

The feelings I’m gesturing at are delicate, and so is the art of conveying their immensity. Unfortunately, I’m not here to offer them the poetic justice they deserve. Rather, I’m here to claim that these emotions don’t get enough attention, and discuss why I think giving them more attention could benefit the EA community.

What emotions am I talking about?

I’ll give you a sense of what emotions I’m talking about when I use terms like “visceral motivation” and “emotional altruism” by sharing what thoughts can, at times, invoke these feelings for me. But a caveat first: I’m not referring to any one emotion. I’m thinking about a constellation of different emotions, all sharing an ability to inspire good. Feelings that come to mind for me include:

A sense of awe: Holy shit, so much is at stake. The lives of billions, wait trillions, of beings may be at stake in this century. My care-o-meter simply doesn’t go high enough to feel these numbers anywhere near where they should register. But, every once in a while, I get a glimpse of the moral vastness such a number demarcates– and it’s a vastness I can’t overlook.

A sense of horror: What if we fail? What if we, humanity, fall short and the lives of billions are ended and foreclosed? It’s not written in the stars that humanity goes on and flourishes— what if we live in a world where that doesn’t happen?

A sense of guilt: who am I to have been born so lucky? At no expense of my own, I’m among the most well-off human beings that have ever existed. What did I do to deserve this, and how can I possibly repay my good fortune?

A sense of loving-kindness: Think of the billions of conscious creatures walking this world right now, at the whims of their environment. What a peculiar condition we find ourselves in, craving to make sense of the world, yet so often victims to it. If I could relieve the suffering of all beings, I would. May all find peace and joy in their cosmic blip.

A sense of outrage: How can so many of us act like everything is fine? Can’t you see all the people dying only ...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: An update on GiveWell’s funding projections, published by GiveWell on July 5, 2022 on The Effective Altruism Forum.

As little as six months ago, we were in the position of having more funding available than we could spend on opportunities that met our very high cost-effectiveness bar. Today, the opposite is true—we don’t expect to have enough funding to support all the cost-effective opportunities we find.

In this post we will:

provide an update on GiveWell’s projected funding position,

explain how we have been successful in identifying cost-effective opportunities, and

share our initial thoughts about what this update means for GiveWell’s forward-looking grantmaking strategy.

The state of funding

We wrote last year that we would roll over approximately $110 million in funding from 2021 to spend this year. We ultimately rolled over substantially less because we were imprecise in calculating our projected funds in and out (more details available on our mistakes page). But at a high level, it remained true that we received more money than we chose to spend on highly cost-effective funding opportunities.

We expected to be in a similar position this year, rolling over approximately $110 million. We now believe that we will be funding constrained. There are two core reasons for this:

We found a lot more cost-effective opportunities that need funding. Based on our current research pipeline, we think we’ll be able to recommend up to approximately $750 million in grants that are at least 6x as cost-effective as cash transfers.[1] Last year, we identified about $500 million in grants, most of which were at least 8x.

We think we will receive less money than we projected due to recent declines in financial markets. We currently expect to receive $500 to $600 million (though the range of possible outcomes is wide). This is driven by a revision in Open Philanthropy’s 2022 allocation ($350 million, up from $300 million in 2021 but down from the tentatively planned $500 million[2]), as well as expected declines from other donors.

How we have been able to identify more funding opportunities

Our researchers have identified more opportunities across two streams of work:

Core interventions: Core interventions are the programs we know best because we have recommended them for a long time (e.g., malaria nets, deworming, etc.). They are the most cost-effective programs that we know of and are often delivered at scale. This stream of grantmaking currently accounts for approximately three-quarters of our spending and is most commonly spent on top charities. We have been able to grow spending in this stream by encouraging organizations to expand their programs into locations that are underserved. For example, we supported the Against Malaria Foundation’s expansion into Nigeria, the country with the largest malaria burden in the world.[3]

New interventions: New interventions are programs that are newer to us and that we have funded for shorter periods of time. We source new interventions by reading academic papers, talking to subject matter experts, and hearing directly from organizations. We build cost-effectiveness models to determine how to prioritize further work, and make grants to a small number of programs that meet our cost-effectiveness bar. This stream of grantmaking currently accounts for about one-quarter of our spending. Some grantees in this stream may be on a pathway to become a top charity, but this is not the goal of this stream of work—the goal is to fund opportunities if their expected value reaches or exceeds our funding bar. We have grown the number of opportunities in this stream by adding research capacity to investigate more programs.

In addition to considering programs we haven’t evaluated before, we have updated our views on cause areas following the release of new a...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: I interviewed Sam Bankman-Fried on my podcast!, published by Dwarkesh Patel on July 5, 2022 on The Effective Altruism Forum.

Hey everybody!

I interviewed Sam Bankman Fried on The Lunar Society Podcast!

Watch on YouTube, or listen on Spotify, Apple Podcasts, or any other podcast platform. Read the full transcript here.

I asked a ton of questions about effective altruism and SBF's giving, such as:

Why he created a new EA philanthropic org instead of just donating to Open Philanthropy.

How he deals with adverse selection in the projects he gets proposed?

What would be the scenario in which he feels Future Fund didn't live up to its potential?

When is it best to donate directly and when is it better to influence the government to make something happen?

Are EA causes multiplicative?

How does he hedge against the possibility that utilitarianism is wrong in his giving?

Should we think of his charitable giving as a yearly contribution of a billion dollars or as a 30 billion hedge against the possibility that there's going to be some existential risk that requires a large pool of liquid wealth?

I also asked a bunch of questions about crypto, FTX, leadership, hiring, and scaling.

Hope you enjoy it! If so, please share it with your friends and on social media! Here is the announcement tweet. Thanks!

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Goal Factoring, published by CFAR!Duncan on July 5, 2022 on LessWrong.

Epistemic status: Anecdotally strong

The goal factoring technique as taught by CFAR is not derived from any particular body of psych research, but is instead a straightforward and general application of the principle of reductionism. It was developed and refined through iteration, and has been useful to large numbers of alumni.

Imagine that you are sitting at home at the end of a long, hard week, preparing to make plans for the evening. One of your friends is throwing a party, and has urged you to come. At the same time, there are delicious leftovers in the fridge and a movie you’ve been hoping to watch, and you’re feeling pretty lethargic.

When making this sort of decision, most people do some form of weighing— whether explicitly, with System 2, or viscerally/intuitively, with System 1— comparing the pros and cons of each option and selecting the one with the highest net “goodness.” You may consider things like who is likely to be at the party, or whether your friend would be offended if you didn’t show up; you may do some internal measuring of your energy levels, to see if you’re in dire need of some rest and relaxation. The decision might come from balancing a bunch of little things, or be based on one crucial factor.

Ultimately, though, most people end up picking one or the other—we either go out, or stay in. Occasionally, we might come up with a sort of com- promise option—such as going to the party for half an hour and then coming back home—but we rarely reach outside of the A, B, or A&B framework.

The goal factoring technique asks that we approach these sorts of problems a little differently. Instead of simply comparing one choice to another, goal factoring encourages us to adopt a “third path” mentality—to assume, for the sake of argument, that there might be a way to get everything we want, and achieve all of the good with none of the bad.

The Parable of the Orange

Sometimes, of course, there is no way to get everything. Sometimes, we really are constrained, and have to make tradeoffs and compromises. But we tend to feel constrained more often than we really are, thanks to social imperatives and longstanding habits and assumed entanglements between various obligations. Often, there’s a lot of wiggle room that we aren’t aware of, especially if it’s been a while since we stepped back and took a fresh look from a broader perspective.

In the parable of the orange, two individuals both reached for the last orange at the market, and thus began an argument. On the surface, this appeared to be a classic A-or-B situation; the orange was going to go with one person or the other (and a compromise, such as cutting the orange in half, would leave both individuals dissatisfied).

However, when the farmer asked the question “What do you want the orange for?” the situation suddenly changed. “Want orange” is not an atomic, fundamental drive, after all—a person’s desire for an orange is usually instrumental in some way or another. Most likely the two individuals wanted to eat it, yes—but maybe they wanted to make juice, or to get seeds for planting, or to use the zest in a recipe; there are any number of other possibilities that are actually reasonably likely.

As it happened, one of the would-be buyers was hoping to make mulled wine, which requires an orange peel, but not the actual flesh of the fruit. Since the other buyer simply wanted to eat, it was possible for both of them to get all of what they wanted, by dividing the orange appropriately.

Had the farmer not spoken up, though—had the three of them not investigated the possibility of a third path—the situation would have ended with at least one person being disappointed.

Goal factoring asks that we play the role of the farmer in our own decision-makin...