The Nonlinear Library

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Are our Top Charities saving the same lives each year?, published by GiveWell on June 18, 2024 on The Effective Altruism Forum.

Author: Adam Salisbury, Senior Research Associate

In a nutshell

We've had a longstanding concern that some of our top charity programs, including insecticide-treated nets, seasonal malaria chemoprevention (SMC), and vitamin A supplementation (VAS), may have less impact than we've estimated due to "repetitive saving." These programs provide health interventions to the same children under 5 years old annually or every 3 years. Our cost-effectiveness models currently assume that different lives are saved each year from these interventions.

We think it's possible the programs are actually saving the same, high-risk children over and over. In a worst-case scenario, this could mean the programs are saving 80% fewer cumulative lives than we thought.

Based on a shallow review of empirical evidence and talking to experts, our best guess is that we're only overstating the total lives saved by these programs by around 10%, because:

Under-5 deaths from malaria, pneumonia, diarrhea, etc., are concentrated in the first year or two of life. Even if the mortality risk were concentrated among the same children each year, the impact comes from the initial couple of years of treatment, which leaves less scope for repetitive saving in later years.

The highest-risk group of children appears to vary somewhat randomly each year, reducing overlap. For example, malaria hotspots can shift over time.

This broad conclusion also matches the opinions of two epidemiology experts we consulted.

Our main uncertainties are:

What about survival after age 5? We focus primarily on survival through age 5, but it's possible children whose deaths we avert in the under-5 window have lower life expectancy beyond age 5. Though we've thought less about this, we think there are both empirical and intuitive reasons to expect longer-term survival benefits from these programs.

We also think accounting for life expectancy post-treatment raises difficult moral questions: we'd worry about implicitly weighing lives saved in more deprived contexts as less valuable than others.

How persistent are at-risk populations? Ideally, we'd directly identify and track the children saved by the programs to see their future risk. In reality, we have to use proxies like children discharged from the hospital after severe malaria. These may not be perfect stand-ins if they developed some immunity or have lingering complications.

Summary

What's the issue?

Within GiveWell's research team, there's been a long standing question: are we overstating the impact of our top charities by failing to account for "repetitive saving", or averting the same deaths each year? (more)

3 of the programs our top charities support - insecticide-treated net campaigns, seasonal malaria chemoprevention (SMC), and vitamin A supplementation (VAS) - provide health commodities repetitively to children under 5 (e.g., they provide a round of SMC to children in one region of Burkina Faso in 2025, then provide another round to those same children in 2026, and so on).

Our cost-effectiveness models of these programs make the (implicit) assumption that we're saving a different set of lives each time we provide treatment. What our SMC model says, in effect, is: "a subset of children are saved in year 1 and returned to average life expectancy; in year 2, a different subset of children are saved and returned to average life expectancy".

This could be wrong if we're saving the same set of children each year - for instance, if SMC only delays deaths by a year, rather than returning children to average life expectancy (as we assume).

In a worst-case scenario, the 5-year SMC program we support might be saving 80% fewer cumulative lives than what we est...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Expressions of Interest: Starting an Islamic Effective Giving Org, published by Kaleem on June 18, 2024 on The Effective Altruism Forum.

TLDR: Gathering expressions of interest for people who want to contribute to a new org working on making zakat more effective

Hi,

I've been researching the feasibility of launching a new org which (either entirely, or as one of its functions) works to redirect zakat to (the most) effective charities. You can see some thoughts I've had whilst doing this work here: 1, 2, 3 .

I'm gathering expressions of interest from anyone who is interest in being part of this new org, either full time or in some bit-part capacity. Ideally I'd find a great co-founder to work with, but there are lots of other ways to contribute.

Feel free to share the form link with anyone you think might be interested, and feel free to comment or DM with any questions or suggestions.

Thanks !

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Announcing AI Welfare Debate Week (July 1-7), published by Toby Tremlett on June 18, 2024 on The Effective Altruism Forum.

July 1-7 will be AI Welfare Debate Week on the EA Forum. We will be discussing the debate statement: "AI welfare [1] should be an EA priority[2]". The Forum team will be contacting authors who are well-versed in this topic to post, but we also welcome posts, comments, quick takes and link-posts from any Forum user who is interested.

Feedback on this post may influence the exact wording of the debate statement and its footnotes[3], but it will be held fixed enough to ensure that someone writing a post immediately after this announcement is published can be confident that it will still be relevant to the debate on July 1st.

We will be experimenting with a banner where users can mark how strongly they agree or disagree with the debate statement, and a system which uses the posts we record as changing your mind to produce a list of the most influential posts.

Should AI welfare be an EA priority?

AI welfare - the capacity of digital minds to feel pleasure, pain, happiness, suffering, satisfaction, frustration, or other morally significant welfare states - appears in many of the best and worst visions of the future. If we consider the value of the future from an impartial welfarist perspective, and if digital minds of comparable moral significance to humans are far easier to create than humans, then the majority of future moral patients may be digital.

Even if they don't make up the majority of minds, the total number of digital minds in the future could be vast.

The most tractable period to influence the future treatment of digital minds may be limited. We may have decades or less to advocate against the creation of digital minds (if that were the right thing to do), and perhaps not much longer than that to advocate for proper consideration of the welfare or rights of digital minds if they are created.

Therefore, gaining a better understanding of the likely paths in front of us, including the ways in which the EA community could be involved, is crucial. The sooner, the better.

My hopes for this debate

Take these all with a pinch of salt, the debate is for you, these are my (Toby's) opinions.

I'd like to see discussion focus on digital minds and AI welfare rather than AI in general.

There will doubtless be valuable discussion comparing artificial welfare to other causes, but the most interesting arguments are likely to focus on the merits or demerits of this cause. In other words, it'd be less interesting (for me at least) to see familiar arguments that one cause should dominate EA funding or that another cause should not be funded by EA, even though both arguments would be ways to push towards agree or disagree on the debate statement.

I'd rather we didn't spend too high a percentage of the debate on the question of whether AI will ever be sentient, although we will have to decide how to deal with the uncertainty here.

FAQs

How does the banner work?

The banner will show the distribution of the EA Forum's opinion on the debate question. Users can place their icon anywhere on the axis to indicate their opinion, and can move it as many times as they like during the week.

How are the "most influential posts" calculated?

Under the banner, you'll be able to see a leaderboard of "most influential posts". These are ranked based on a metric I'm calling "delta-points". You get delta points when someone changes their mind- moving their marker along the agree/ disagree line which will appear at the bottom of your post, if it is tagged "digital minds debate week".

Do I have to write in the style of a debate?

No. The aim of this debate week is to elicit interesting content which changes the audience's mind. This could be in the form of a debate-style argument for a...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Animal advocates should campaign to restrict AI precision livestock farming, published by Zachary Brown on June 18, 2024 on The Effective Altruism Forum.

Here's a fairly safe prediction: most of the potential harm from AI is potential harm to nonhuman animals.

It's a safe[1] prediction because there are so many more nonhuman animals than there are humans. Many of these animals presently live in conditions where their lives are influenced or controlled by human activity. Since AI will increasingly substitute for human activity, many animals' lives might therefore someday be controlled or influenced by AI.

Indeed, while there are just over 8 billion people, there are around a zillion[2] animals farmed each year, the majority on factory farms, where their lives are nearly entirely determined by human choices. Factory farms, then, are an enormous and terrifying arena for AI-caused harm.

Accordingly, when we talk about AI governance and risks from AI, risks to nonhuman animals should be a major part of that conversation, alongside e.g. biorisks and discrimination. Unfortunately, this is basically never the case.

For instance, the mostly commendable recent International Scientific Report on the Safety of Advanced AI: Interim Report, a long and comprehensive document, with a long and comprehensive name, written by committee and designed to achieve global consensus on AI risks, does not even mention animal welfare risks - does not even mention animals, or agriculture, or anything like that - at all.

This blog post is a small attempt to rectify the imbalance. I want to share my thoughts about a specific AI technology: precision livestock farming. I'll argue that, while I am uncertain about whether AI precision farming technology will be good or bad for animal welfare on a per-animal basis, I think we can be much more confident that it will be bad on a total population basis, since it will likely increase the total number of farmed animals.

Largely in view of this argument, but also for political feasibility reasons, I think we should consider advocating for restrictions on precision livestock farming in industrial settings.

'AI' can mean a lot of different things

When we talk about the effects of AI on animal welfare, it is essential to distinguish different types of AI systems. 'How will AI affect animals?' is uselessly broad. There are a number of different AI technologies we should distinguish between: precision livestock farming, AI genetics and breeding programs, language models used by the general public, AI adoption by activists, AI adoption by alternative protein producers, models used to decode animal thoughts, and others.

All quite different, and it's hard to know what the aggregate effect of these technologies will be.

A lot of these technologies are important, but my focus here is on precision livestock farming.

What is precision livestock farming?

Precision livestock farming (PLF), as I understand it, basically means putting in place a bunch of sensors and instruments that gather data about individual farmed animals, and then having some algorithm - this could be a neural network, but might be something much more basic - manage features of the animals' environment, based on the gathered data, in order to 'optimise' farming.

The classic example is an automated feeding system, which gathers data about individual animals' feeding behaviour and health and stuff and automatically distributes food in response to the gathered data.

In theory, systems like this can be designed to make sure all animals are having a great time. In practice, systems like this will often be designed, I expect, to further intensify animal agriculture.

What do we know about the particular technologies that are currently being implemented?

According to this paper (Schillings et al, 2021), the f...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: D&D.Sci Alchemy: Archmage Anachronos and the Supply Chain Issues Evaluation & Ruleset, published by aphyer on June 18, 2024 on LessWrong.

This is a follow-up to last week's D&D.Sci scenario: if you intend to play that, and haven't done so yet, you should do so now before spoiling yourself.

There is a web interactive here you can use to test your answer, and generation code available here if you're interested, or you can read on for the ruleset and scores.

RULESET

There are two steps to brewing a potion:

STEP 1: MAGICAL POTENCY

Any ingredient that doesn't exist in the mundane world is Magical, while any ingredient that exists in the mundane world is not:

Magical

Not Magical

Angel Feather

Badger Skull

Beholder Eye

Beech Bark

Demon Claw

Crushed Diamond

Dragon Scale

Crushed Onyx

Dragon Spleen

Crushed Ruby

Dragon Tongue

Crushed Sapphire

Dragon's Blood

Eye of Newt

Ectoplasm

Ground Bone

Faerie Tears

Oaken Twigs

Giant's Toe

Powdered Silver

Troll Blood

Quicksilver

Vampire Fang

Redwood Sap

The first step of potion-brewing is to dissolve the magical potency out of the Magical Ingredients to empower your potion. This requires the right amount of Magical Ingredients: too few, and nothing magical will happen and you will produce Inert Glop, while too many and there will be an uncontrolled Magical Explosion.

If you include:

0-1 Magical Ingredients: 100% chance of Inert Glop.

2 Magical Ingredients: 50% chance of Inert Glop, 50% chance OK.

3 Magical Ingredients: 100% chance OK.

4 Magical Ingredients: 50% chance OK, 50% chance Magical Explosion.

5+ Magical Ingredients: 100% chance Magical Explosion.

If your potion got past this step OK, move on to:

STEP 2: DIRECTION

Some ingredients are used to direct the magical power into the desired resulting potion. Each potion has two required Key Ingredients, both of which must be included to make it:

Potion

Key Ingredient 1

Key Ingredient 2

Barkskin Potion*

Crushed Onyx

Ground Bone

Farsight Potion

Beholder Eye

Eye of Newt

Fire Breathing Potion

Dragon Spleen

Dragon's Blood

Fire Resist Potion

Crushed Ruby

Dragon Scale

Glibness Potion

Dragon Tongue

Powdered Silver

Growth Potion

Giant's Toe

Redwood Sap

Invisibility Potion

Crushed Diamond

Ectoplasm

Necromantic Power Potion*

Beech Bark

Oaken Twigs

Rage Potion

Badger Skull

Demon Claw

Regeneration Potion

Troll Blood

Vampire Fang

*Well. Sort of. See the Bonus Objective section below.

Some ingredients (Angel Feather, Crushed Sapphire, Faerie Tears and Quicksilver) aren't Key Ingredients for any potion in the dataset. Angel Feather and Faerie Tears are nevertheless useful - as magical ingredients that don't risk creating any clashing potion, they're good ways to add magical potential to a recipe. Crushed Sapphire and Quicksilver have no effect, including them is entirely wasteful.

If you've gotten through Step 1, the outcome depends on how many potions you've included both the Key Ingredients of:

0 potions: with nothing to direct it, the magical potential dissolves into an Acidic Slurry.

1 potion: you successfully produce that potion.

2 or more potions:

Sometimes (1/n of the time, where n is # of potions you included) a random one of the potions will dominate, and you will produce that one.

The rest of the time, the clashing directions will produce Mutagenic Ooze.

So, for example, if you brew a potion with:

Dragon Spleen, Dragon Scale, Dragon Tongue and Dragon's Blood:

You have included 4 magical ingredients, and the Key Ingredients of one potion (Fire Breathing).

50% of the time you will get a Magical Explosion, 50% of the time you will get a Fire Breathing Potion.

Badger Skull, Demon Claw, Giant's Toe, Redwood Sap.

You have included 2 magical ingredients, and the Key Ingredients of two potions (Rage and Growth).

50% of the time you will get Inert Glop, 25% of the time Mutagenic Ooze, 12.5% of the time G...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: I would have shit in that alley, too, published by Declan Molony on June 18, 2024 on LessWrong.

After living in a suburb for most of my life, when I moved to a major U.S. city the first thing I noticed was the feces. At first I assumed it was dog poop, but my naivety didn't last long.

One day I saw a homeless man waddling towards me at a fast speed while holding his ass cheeks. He turned into an alley and took a shit. As I passed him, there was a moment where our eyes met. He sheepishly averted his gaze.

The next day I walked to the same place. There are a number of businesses on both sides of the street that probably all have bathrooms. I walked into each of them to investigate.

In a coffee shop, I saw a homeless woman ask the barista if she could use the bathroom. "Sorry, that bathroom is for customers only." I waited five minutes and then inquired from the barista if I could use the bathroom (even though I hadn't ordered anything). "Sure! The bathroom code is 0528."

The other businesses I entered also had policies for 'customers only'. Nearly all of them allowed me to use the bathroom despite not purchasing anything.

If I was that homeless guy, I would have shit in that alley, too.

I receive more compliments from homeless people compared to the women I go on dates with

There's this one homeless guy - a big fella who looks intimidating - I sometimes pass on my walk to the gym. The first time I saw him, he put on a big smile and said in a booming voice, "Hey there! I hope you're having a blessed day!" Without making eye contact (because I didn't want him to ask me for money), I mumbled "thanks" and quickly walked away.

I saw him again a few weeks later. With another beaming smile he exclaimed, "You must be going to the gym - you're looking fit, my man!" I blushed and replied, "I appreciate it, have a good day." He then added, "God bless you, sir!" Being non-religious, that made me a little uncomfortable.

With our next encounter, I found myself smiling as I approached him. This time I greeted him first, "Good afternoon!" His face lit up with glee. "Sir, that's very kind of you. I appreciate that. God bless you!" Without hesitation I responded, "God bless you, too!" I'm not sure the last time I've uttered those words; I don't even say 'bless you' after people sneeze.

We say hi to each other regularly now. His name is George.

Is that guy dead?

Coming home one day, I saw a disheveled man lying facedown on the sidewalk.

He's not moving. I crouched to hear if he's breathing. Nothing.

I looked up and saw a lady in a car next to me stopped at a red light. We made eye contact and I gestured towards the guy as if to say what the fuck do we do? Her answer was to grip the steering wheel and aggressively stare in front of her until the light turned green and she sped off.

Not knowing if I needed to call an ambulance, I asked him, "Hey buddy, you okay?" I heard back a muffled, "AYE KENT GEEUP!"

Well, at least he's not dead.

"Uhh, what was that? You doing okay?" This time a more articulate, "I CAN'T GET UP," escaped from him. Despite his clothes being somewhat dirty and not wanting to touch him, I helped him to his feet.

With one look on his face I could tell that he wasn't all there. I asked him if he knew where he was or if he needed help, but he could only reply with gibberish. It could have been drugs; it could have been mental illness. With confirmation that he wasn't dead and was able to walk around, I went home.

Who's giving Brazilian waxes to the homeless?

I was walking behind a homeless man the other day. He was wearing an extra long flannel and sagging his pants low.

Suddenly, he noticed his (one and only) shoe was untied and fixed it promptly by executing a full

standing pike. I wasn't expecting him to have the flexibility of a gymnast.

In doing so, his flannel lifted u...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Join GiveWell as a Research Analyst, published by GiveWell on June 18, 2024 on The Effective Altruism Forum.

GiveWell is hiring Research Analysts: If you're looking for a high-impact entry-level position with no experience requirements, we'd love for you to apply! We are looking to hire one to two more Research Analysts and will be accepting applications through July 15 at the latest, but may close the application sooner if we make our last hire before then.

Have a huge impact with your career

Each year, GiveWell's researchers spend more than 50,000 hours looking for and funding cost-effective programs that save and improve lives. In 2022 (the most recent year for which we have data), we directed more than $430 million to programs around the world - and we expect that this funding will avert more than 56,000 deaths![1] Your work as a Research Analyst will increase that impact.

With additional research capacity, we could look for even more promising programs and gain even more confidence that we're funding the most cost-effective opportunities.

As a Research Analyst on the Commons subteam, your work will focus on the highest-impact investigations currently underway. The Commons subteam provides flexible research capacity to all of the other research subteams, each of which focus on a specific area of our grantmaking (malaria, water quality, vaccines, etc.). By providing flexible support, our team allows the research team as a whole to direct its efforts toward the highest-priority projects.

In addition to opening up capacity for senior researchers, you'll have opportunities for more direct impact. For example, Research Analysts are responsible for verifying the accuracy of our work and ensuring consistency in the way we're modeling cost-effectiveness across different types of programs. Finding errors and inconsistencies in our work has the potential to change our funding decisions and improve the cost-effectiveness of our grantmaking.

You'll also have the opportunity to influence GiveWell's decisions beyond your own workstreams. New staff are often surprised at the openness of GiveWell leadership to engaging with critiques of our work, including from junior staff.

Gain skills and experience to jump-start your career in global health and development

You'll gain a breadth of knowledge about global health and development. Because Research Analysts on the Commons team work with all the other research subteams, you'll be in a unique position to learn broadly about GiveWell's research.

In a given week, you might be researching the options a maize farmer in Malawi has for selling their product, checking the accuracy of the formulas in a cost-effectiveness model for the co-delivery of vitamin A supplementation and seasonal malaria chemoprevention in Nigeria, and drafting a page explaining our reasoning for recommending a grant to support the installation of chlorine dispensers near communal water points.

You'll get an inside view of the grantmaking process. You'll see the GiveWell research pipeline in action, as we move from shallow reviews of a large number of programs to more intensive reviews of promising programs and potential implementers, focusing on the strength of evidence, cost-effectiveness, and capacity to effectively use additional funding.

In addition to working on individual projects at various stages in the process, you may also have opportunities to embed in a subteam for the duration of a grant investigation and gain deeper insight into a particular intervention.

You'll have support to grow and develop. We want to help you build a career, so we provide time, money, and ongoing guidance to help you achieve your professional development goals. Research Analysts on the Commons team also receive substantial support from the Commons Lead and Commons Coordinator to...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Why so many "racists" at Manifest?, published by Austin on June 18, 2024 on The Effective Altruism Forum.

Manifest 2024 is a festival that we organized last weekend in Berkeley. By most accounts, it was a great success. On our feedback form, the average response to "would you recommend to a friend" was a 9.0/10.

Reviewers said nice things like "one of the best weekends of my life" and "dinners and meetings and conversations with people building local cultures so achingly beautiful they feel almost like dreams" and "I've always found tribalism mysterious, but perhaps that was just because I hadn't yet found my tribe."

Arnold Brooks running a session on Aristotle's Metaphysics. More photos of Manifest here.

However, a recent post on The Guardian and review on the EA Forum highlight an uncomfortable fact: we invited a handful of controversial speakers to Manifest, whom these authors call out as "racist". Why did we invite these folks?

First: our sessions and guests were mostly not controversial - despite what you may have heard

Here's the schedule for Manifest on Saturday:

(The largest & most prominent talks are on the left. Full schedule here.)

And here's the full list of the 57 speakers we featured on our website: Nate Silver, Luana Lopes Lara, Robin Hanson, Scott Alexander, Niraek Jain-sharma, Byrne Hobart, Aella, Dwarkesh Patel, Patrick McKenzie, Chris Best, Ben Mann, Eliezer Yudkowsky, Cate Hall, Paul Gu, John Phillips, Allison Duettmann, Dan Schwarz, Alex Gajewski, Katja Grace, Kelsey Piper, Steve Hsu, Agnes Callard, Joe Carlsmith, Daniel Reeves, Misha Glouberman, Ajeya Cotra, Clara Collier, Samo Burja, Stephen Grugett, James

Grugett, Javier Prieto, Simone Collins, Malcolm Collins, Jay Baxter, Tracing Woodgrains, Razib Khan, Max Tabarrok, Brian Chau, Gene Smith, Gavriel Kleinwaks, Niko McCarty, Xander Balwit, Jeremiah Johnson, Ozzie Gooen, Danny Halawi, Regan Arntz-Gray, Sarah Constantin, Frank Lantz, Will Jarvis, Stuart Buck, Jonathan Anomaly, Evan Miyazono, Rob Miles, Richard Hanania, Nate Soares, Holly Elmore, Josh Morrison.

Judge for yourself; I hope this gives a flavor of what Manifest was actually like. Our sessions and guests spanned a wide range of topics: prediction markets and forecasting, of course; but also finance, technology, philosophy, AI, video games, politics, journalism and more. We deliberately invited a wide range of speakers with expertise outside of prediction markets; one of the goals of Manifest is to increase adoption of prediction markets via cross-pollination.

Okay, but there sure seemed to be a lot of controversial ones…

I was the one who invited the majority (~40/60) of Manifest's special guests; if you want to get mad at someone, get mad at me, not Rachel or Saul or Lighthaven; certainly not the other guests and attendees of Manifest.

My criteria for inviting a speaker or special guest was roughly, "this person is notable, has something interesting to share, would enjoy Manifest, and many of our attendees would enjoy hearing from them". Specifically:

Richard Hanania - I appreciate Hanania's support of prediction markets, including partnering with Manifold to run a forecasting competition on serious geopolitical topics and writing to the CFTC in defense of Kalshi. (In response to backlash last year, I wrote a post on my decision to invite Hanania, specifically)

Simone and Malcolm Collins - I've enjoyed their Pragmatist's Guide series, which goes deep into topics like dating, governance, and religion. I think the world would be better with more kids in it, and thus support pronatalism. I also find the two of them to be incredibly energetic and engaging speakers IRL.

Jonathan Anomaly - I attended a talk Dr. Anomaly gave about the state-of-the-art on polygenic embryonic screening. I was very impressed that something long-considered scien...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Fat Tails Discourage Compromise, published by niplav on June 17, 2024 on LessWrong.

Say that we have a set of options, such as (for example) wild animal welfare interventions.

Say also that you have two axes along which you can score those interventions: popularity (how much people will like your intervention) and effectiveness (how much the intervention actually helps wild animals).

Assume that we (for some reason) can't convert between and compare those two properties.

Should you then pick an intervention that is a compromise on the two axes - that is, it scores decently well on both - or should you max out on a particular axis?

One thing you might consider is the distribution of options along those two axes: the distribution of interventions can be normal on for both popularity and effectiveness, or the underlying distribution could be lognormal for both axes, or they could be mixed (e.g. normal for popularity, and lognormal for effectiveness).

Intuitively, the distributions seem like they affect the kinds of tradeoffs we can make, how could we possibly figure out how?

…

…

…

It turns out that if both properties are normally distributed, one gets a fairly large Pareto frontier, with a convex set of options, while if the two properties are lognormally distributed, one gets a concave set of options.

(Code here.)

So if we believe that the interventions are normally distributed around popularity and effectiveness, we would be justified in opting for an intervention that gets us the best of both worlds, such as sterilising stray dogs or finding less painful rodenticides.

If we, however, believe that popularity and effectiveness are lognormally distributed, we instead want to go in hard on only one of those, such as buying brazilian beef that leads to Amazonian rainforest being destroyed, or writing a book of poetic short stories that detail the harsh life of wild animals.

What if popularity of interventions is normally distributed, but effectiveness is lognormally distributed?

In that case you get a pretty large Pareto frontier which almost looks linear to me, and it's not clear anymore that one can't get a good trade-off between the two options.

So if you believe that heavy tails dominate with the things you care about, on multiple dimensions, you might consider taking a barbell strategy and taking one or multiple options that each max out on a particular axis.

If you have thin tails, however, taking a concave disposition towards your available options can give you most of the value you want.

See Also

Being the (Pareto) Best in the World (johnswentworth, 2019)

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Getting 50% (SoTA) on ARC-AGI with GPT-4o, published by ryan greenblatt on June 17, 2024 on LessWrong.

I recently got to 50%[1] accuracy on the public test set for ARC-AGI by having GPT-4o generate a huge number of Python implementations of the transformation rule (around 8,000 per problem) and then selecting among these implementations based on correctness of the Python programs on the examples (if this is confusing, go here)[2]. I use a variety of additional approaches and tweaks which overall substantially improve the performance of my method relative to just sampling 8,000 programs.

[This post is on a pretty different topic than the usual posts I make about AI safety.]

The additional approaches and tweaks are:

I use few-shot prompts which perform meticulous step-by-step reasoning.

I have GPT-4o try to revise some of the implementations after seeing what they actually output on the provided examples.

I do some feature engineering, providing the model with considerably better grid representations than the naive approach of just providing images. (See below for details on what a "grid" in ARC-AGI is.)

I used specialized few-shot prompts for the two main buckets of ARC-AGI problems (cases where the grid size changes vs doesn't).

The prior state of the art on this dataset was 34% accuracy, so this is a significant improvement.[3]

On a held-out subset of the train set, where humans get 85% accuracy, my solution gets 72% accuracy.[4] (The train set is significantly easier than the test set as noted here.)

Additional increases of runtime compute would further improve performance (and there are clear scaling laws), but this is left as an exercise to the reader.

In this post:

I describe my method;

I analyze what limits its performance and make predictions about what is needed to reach human performance;

I comment on what it means for claims that François Chollet makes about LLMs. Given that current LLMs can perform decently well on ARC-AGI, do claims like "LLMs like Gemini or ChatGPT [don't work] because they're basically frozen at inference time. They're not actually learning anything." make sense? (This quote is from here.)

Thanks to Fabien Roger and Buck Shlegeris for a bit of help with this project and with writing this post.

What is ARC-AGI?

ARC-AGI is a dataset built to evaluate the general reasoning abilities of AIs. It consists of visual problems like the below, where there are input-output examples which are grids of colored cells. The task is to guess the transformation from input to output and then fill out the missing grid. Here is an example from the tutorial:

This one is easy, and it's easy to get GPT-4o to solve it. But the tasks from the public test set are much harder; they're often non-trivial for (typical) humans. There is a reported MTurk human baseline for the train distribution of 85%, but no human baseline for the public test set which is known to be significantly more difficult.

Here are representative problems from the test set[5], and whether my GPT-4o-based solution gets them correct or not.

Problem 1:

Problem 2:

Problem 3:

My method

The main idea behind my solution is very simple: get GPT-4o to generate around 8,000 python programs which attempt to implement the transformation, select a program which is right on all the examples (usually there are 3 examples), and then submit the output this function produces when applied to the additional test input(s). I show GPT-4o the problem as images and in various ascii representations.

My approach is similar in spirit to the approach applied in AlphaCode in which a model generates millions of completions attempting to solve a programming problem and then aggregates over them to determine what to submit.

Actually getting to 50% with this main idea took me about 6 days of work. This work includes construct...