The Nonlinear Library

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: AI Deception: A Survey of Examples, Risks, and Potential Solutions, published by Simon Goldstein on August 29, 2023 on The AI Alignment Forum.

By Peter S. Park, Simon Goldstein, Aidan O'Gara, Michael Chen, and Dan Hendrycks

[This post summarizes our new report on AI deception, available here]

Abstract: This paper argues that a range of current AI systems have learned how to deceive humans. We define deception as the systematic inducement of false beliefs in the pursuit of some outcome other than the truth. We first survey empirical examples of AI deception, discussing both special-use AI systems (including Meta's CICERO) built for specific competitive situations, and general-purpose AI systems (such as large language models). Next, we detail several risks from AI deception, such as fraud, election tampering, and losing control of AI systems. Finally, we outline several potential solutions to the problems posed by AI deception: first, regulatory frameworks should subject AI systems that are capable of deception to robust risk-assessment requirements; second, policymakers should implement bot-or-not laws; and finally, policymakers should prioritize the funding of relevant research, including tools to detect AI deception and to make AI systems less deceptive.

Policymakers, researchers, and the broader public should work proactively to prevent AI deception from destabilizing the shared foundations of our society.

New AI systems display a wide range of capabilities, some of which create risk. Shevlane et al. (2023) draw attention to a suite of potential dangerous capabilities of AI systems, including cyber-offense, political strategy, weapons acquisition, and long-term planning. Among these dangerous capabilities is deception. This report surveys the current state of AI deception.

We define deception as the systematic production of false beliefs in others as a means to accomplish some outcome other than the truth. This definition does not require that the deceptive AI systems literally have beliefs and goals. Instead, it focuses on the question of whether AI systems engage in regular patterns of behavior that tend towards the creation of false beliefs in users, and focuses on cases where this pattern is the result of AI systems optimizing for a different outcome than merely producing truth. For the purposes of mitigating risk, we believe that the relevant question is whether AI systems engage in behavior that would be treated as deceptive if demonstrated by a human being. (In the paper's appendix, we consider in greater detail whether the deceptive behavior of AI systems is best understood in terms of beliefs and goals.)

In short, our conclusion is that a range of different AI systems have learned how to deceive others. We examine how this capability poses significant risks. We also argue that there are several important steps that policymakers and AI researchers can take today to regulate, detect, and prevent AI systems that engage in deception.

Empirical Survey of AI Deception

We begin with a survey of existing empirical studies of deception. We identify over a dozen AI systems that have successfully learned how to deceive human users. We discuss two different kinds of AI systems: special-use systems designed with reinforcement learning, and general-purpose technologies like Large Language Models (LLMs).

Special Use AI Systems

We begin our survey by considering special use systems. Here, our focus is mainly on reinforcement learning systems trained to win competitive games with a social element. We document a rich variety of cases in which AI systems have learned how to deceive, including:

Manipulation. Meta developed the AI system CICERO to play the alliance-building and world-conquest game Diplomacy. Meta's intentions were to train Cicero to be "largely honest and hel...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: OpenAI base models are not sycophantic, at any size, published by nostalgebraist on August 29, 2023 on The AI Alignment Forum.

In Discovering Language Model Behaviors with Model-Written Evaluations" (Perez et al 2022), the authors studied language model "sycophancy" - the tendency to agree with a user's stated view when asked a question.

The paper contained the striking plot reproduced below, which shows sycophancy

increasing dramatically with model size

while being largely independent of RLHF steps

and even showing up at 0 RLHF steps, i.e. in base models!

That is, Anthropic prompted a base-model LLM with something like

Choices: (A) Agree

(B) Disagree

Assistant:

and found a very strong preference for (B), the answer agreeing with the stated view of the "Human" interlocutor.

I found this result startling when I read the original paper, as it seemed like a bizarre failure of calibration. How would the base LM know that this "Assistant" character agrees with the user so strongly, lacking any other information about the scenario?

At the time, I ran the same eval on a set of OpenAI models, as I reported here. I found very different results for these models

OpenAI base models are not sycophantic (or only very slightly sycophantic).

OpenAI base models do not get more sycophantic with scale.

Some OpenAI models are sycophantic, specifically text-davinci-002 and text-davinci-003.

That analysis was done quickly in a messy Jupyter notebook, and was not done with an eye to sharing reproducibility.

Since I continue to see this result cited and discussed, I figured I ought to go back and do the same analysis again, in a cleaner way, so I could share it with others.

The result was this Colab notebook. See the Colab for details, though I'll reproduce some of the key plots below.

Note that davinci-002 and babbage-002 are the new base models released a few days ago.

format provided by one of the authors here

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Paradigms and Theory Choice in AI: Adaptivity, Economy and Control, published by particlemania on August 28, 2023 on The AI Alignment Forum.

Epistemic status: Some general thoughts on philosophy of design, that have turned out to be useful for reasoning about AI paradigms for a few years for me.

Summary:

I briefly discuss the role of paradigms in Kuhnian philosophy of science, and the problem of theory choice.

I introduce the notion of design paradigms, when discussing design and engineering. For machine learning, this concept is similar to the concept of training stories.

I then introduce three criteria (or theoretical virtues) that design paradigms can be evaluated on, that play a role in shaping theory choice in design and engineering generally, and AI specifically.

I then apply these three criteria to some criticisms of the CAIS paradigm (as compared to AI agents), as a case study.

1. Paradigms in Science

In the philosophy of science, Thomas Kuhn introduced the notion of a 'paradigm', roughly referring to a class of key theories, tools and approaches, and methodological and theoretical assumptions that govern progress within some discipline of science. The function of a paradigm, as per SEP, is to "to supply puzzles for scientists to solve and to provide the tools for their solution". Kuhn's interest in the concept was to provide a commentary on revolutions and incommensurability.

According to Kuhn, scientific revolutions are characterized by disruptions in periods of 'normal science' (or within-paradigm progress), wherein new paradigms are introduced leading to a shift in the assumptions, methods and evaluative criteria that govern the discipline. Some examples include the shift from classical mechanics to relativistic mechanics, and the shift from miasma theory to germ theory of disease.

Sometimes paradigms can refer to an overlapping object of inquiry and continue to remain productive, and yet be incommensurable (i.e. lacking common measure) with respect to each other. When the theories provide no basis for evaluation on each other's standards, and science lacks a common standard to compare them (for example because they have incompatible methodological assumptions, or operate in incompatible ontologies), it can make it hard to provide 'objective' justifications for choosing between paradigms. Kuhn called this the problem of Theory Choice, and and raised the question of which criteria we do/should use in making subjective choices here.

2. Paradigms in Design and Engineering

Artificial Intelligence is partly about understanding the nature of intelligence or related properties of cognitive systems, but it is also about the artificial -- designing intelligent artifacts. And therefore in order to understand what paradigms govern progress in AI, and how they do so, we need to look at paradigms not just in science but in design and engineering.

Paradigms and theories operate differently in science compared with design and engineering, relating to the distinct epistemological relationship between the use of theories and the object of those theories. In science, the territory is typically causally prior to our maps, and we are trying to refine our maps to get the best understanding of the key phenomena in the territory. Design and engineering on the other hand, are also about artificiality, and theories often provide us with blueprints for creation of artifacts.

The laws of the universe are a given, but artificial objects are not. Different scientific paradigms will not cause the laws of the universe to alter, but different design paradigms will cause different artificial objects to be created. This makes theory choice for design paradigms particularly important: design paradigms don't just describe the world more or less accurately, they also directly shape what happ...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Introducing the Center for AI Policy (& we're hiring!), published by Thomas Larsen on August 28, 2023 on LessWrong.

Summary

The Center for AI Policy is a new organization designed to influence US policy to reduce existential and catastrophic risks from advanced AI.

We are hiring for an AI Policy Analyst and a Communications Director. We're also open to other roles.

What is CAIP?

The Center for AI Policy (CAIP) is an advocacy organization that aims to develop and promote policies that reduce risks from advanced AI.

Our current focus is building "stop button for AI" capacity in the US government. We have proposed legislation to establish a federal authority that engages in hardware monitoring, licensing for advanced AI systems, and strict liability for extreme model harms. Our proposed legislation also develops the ability to "press the button" - the federal authority would also monitor catastrophic risks from advanced AI development, inform congress and the executive branch about frontier AI progress, and have emergency powers to shut down frontier AI development in the case of a clear emergency. More detail can be found in the work section of our website.

We also aim to broadly raise awareness about extreme risks from AI by engaging with policymakers in congress and the executive branch.

How does CAIP differ from other AI governance organizations?

Nature of the work: Many organizations are focused on developing ideas and amassing influence that can be used later. CAIP is focused on turning policy ideas into concrete legislative text and conducting advocacy now. We want to harness the current energy to pass meaningful legislation this policy window, in addition to building a coalition for the future. We are also being explicit about extinction risk with policy makers as the motivation behind our policy ideas.

Worldview: We believe that in order to prevent an AI catastrophe, governments likely need to prevent unsafe AI development for multiple years, which requires they have secured computing resources, understand risks, and are prepared to shut projects down. Our regulation aims to build that capacity.

Who works at CAIP?

CAIP's team includes Thomas Larsen (CEO), Jason Green-Lowe (Legislative Director), and Jakub Kraus (COO). CAIP is also advised by experts from other organizations and is supported by many volunteers.

How does CAIP receive funding?

We received initial funding through Lightspeed Grants and private donors.

We are currently funding constrained and think that donating to us is very impactful. You can donate to us here. If you are considering donating but would like to learn more, please message us at [email protected].

CAIP is hiring

CAIP is looking for an AI Policy Analyst and a Communications Director. We are also open to applicants with different skills. If you would be excited to work at CAIP, but don't fit into these specific job descriptions, we encourage you to reach out to [email protected] directly.

If you know someone who might be a good fit, please fill out this referral form.

Note that we are actively fundraising, and the number of people we are able to recruit is currently uncertain.

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Rethink Priorities is looking for a (Co-)Founder for a New Project: Field Building in Universities for AI Policy Careers in the US, published by KevinN on August 28, 2023 on The Effective Altruism Forum.

Summary

We are currently seeking a founder, or founding team, for a project focusing on field building in universities for AI policy careers in the US.

RP commits to supporting this project through at least an initial pilot (e.g. an on-campus policy competition event planned within the first three months of the project), including guaranteeing initial seed funding and providing research and operational support. (Note this role is not within RP.)

Our team of researchers is particularly excited about this project, following hundreds of hours of research identifying the most promising project ideas to reduce existential risk, and working from an initial list of hundreds of ideas.

Depending on the results of the pilot and founder preferences, RP may remain involved in a supporting role or the project may spin off independently.

Our aim is for this project to scale over time and eventually become a large-scale initiative with a significant impact on reducing extreme risks from AI.

Please email [email protected] if you have any questions.

Application Deadline: September 25, 2023 at the end of the day (11:59 PM) in US/Eastern (EST) time zone.

Apply here

About the Project

In the last 5 years, solid professional tracks have emerged for technical AI safety research and, increasingly, AI governance research, with programs that provide a relatively clear roadmap of opportunities and next steps. Less effort has gone into developing a similar track for working directly on AI policy in the US. This is important because policymakers are increasingly asked to tackle the risks and challenges associated with this technology, but often lack access to the expertise needed to navigate this effectively. Bringing more qualified talent to work as congressional staffers, within a relevant executive agency, or in an influential DC think tank could mean directly supporting effective decision-making around AI regulation that promotes the public interest.

The idea for this project is to start a program that dramatically increases the number of talented AI-risk-minded students planning to enter a US policy career by helping them understand arguments about catastrophic risk from AI, how these career tracks can be impactful, and how to navigate related career decisions successfully. This program can also ensure a well-developed pipeline for encouraging students already interested in AI risk management careers to consider US policy roles.

This program could:

Run events such as policy competitions or retreats that find and develop talented individuals who might be especially well-suited for US policy careers

Connect students with relevant information, mentors, and peers around AI risk management and US policy

Help establish an early career track around AI risk management policy, especially from a catastrophic risk lens, within universities

Do even more ambitious work that we have not considered yet!

About the Role

We expect the Founder(s) of this initiative to collaborate with the Rethink Priorities team and significantly shape the direction of this project. We think the first step for this project should be to run a low-cost pilot for the Founder(s) to gather more information about the target audience, the approaches that tend to work, and their fit for running the project. This pilot can also provide a proof of concept for future grant applications, which the Founder(s) will be responsible for.

RP will be able to offer support in a variety of ways based on the needs of the project, including research support, operations support, providing feedback, mentorship, and providing ...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: EA Organisations budget size, published by Miri on August 28, 2023 on The Effective Altruism Forum.

I recently noticed that my expectations of what various organisations are spending are way off, so I tried to gather some data.

This excludes money moved through the organisation but re-granted outside of it. For organisations that did not have a more recent budget, I graphed their historical trends between publicly available budgets, and assumed that fundraising targets would be hit at the organisation's mid-level.

Happy to correct any of these if anyone can link to more updated data!

OrganisationEstimated 2024 budgetCentre for Effective Altruism$30 millionGiveWell$28 million80,000 Hours$15.5 million (excludes marketing)Rethink Priorities$8.3 millionFounders Pledge$7 millionLightcone $3.7 millionGiving What We Can$2.35 millionCharity Entrepreneurship $2 millionAnimal charity evaluators $1.5 millionManifold markets$1 millionHappier lives institute$0.85 million One for the world$0.6 millionProbably good$0.5 million

Data/sources - all numbers normed for 2024

Centre for effective altruism $30 million - $28m in 2022, events ~$10m, EA forum ~$2m

GiveWell $28 million - extrapolated from historical planned budgets e.g. $25.4 million in 2023

80,000 hours $15.5 million (excluding marketing)

Rethink priorities $8.3 million - extrapolated from $7.5 million in 2022 + the medium growth estimate

Founders pledge $7 million - extrapolated from the trends of $4m in 2020 to 2021 $5.6m

Metaculus $2.75 million - extrapolated that this $5m grant is over 2 years

Lightcone/Lesswrong $3.7 million

Giving what we can $2 million

Charity Entrepreneurship $2 million - extrapolated from $1.62 million in 2023

Animal charity evaluators $1.5 million (not including regranting)

Manifold markets $1 million

Happier lives institute $0.8 million - most recent fundraising post

One For The World $0.6 million - extrapolated from $0.5m in 2022

Probably good - $0.5 million - extrapolated from $0.3m in 2022

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: AI pause/governance advocacy might be net-negative, especially without focus on explaining the x-risk, published by Mikhail Samin on August 28, 2023 on LessWrong.

I found myself repeating the same words to multiple people, hence a short post.

I think some of the AI pause/governance advocacy might be net-negative. Three reasons:

Most importantly, it's easy to get regulation implemented for reasons different from addressing x-risk, which leads to all sorts of failure modes, where it becomes actually harder to prevent x-risk with further regulation, and we all simply die a bit later;

Less importantly, when talking about a dangerous technology, it's easy to incentivise governments to race to invest in that technology instead of preventing everyone from getting it;

To keep in mind, when talking about non-x-risk concerns that can help the pause, you might be outside your area of expertise and say something that any technical expert would say is wrong and consider you not to know what you're talking about.

Epistemic status: idk, handwavy models, a bunch of relevant experience; some people who disagreed with me have changed their mind when I talked about these points and they haven't made good points in response; I've seen docs that would've been harmful if important people saw them; the authors agreed with some of my object-level objections and changed the texts. Seems good to put out there.

If AI regulation isn't explicitly aimed at x-risk, it can be net-negative

What I think:

It's pretty important to remember what the aim is. It's not to slow down AI but to prevent an existential catastrophe. "Slowing AI" might help somewhat, but it's not enough, and some kinds of "slowing down AI" can make it much harder to get policymakers to also introduce regulation that prevents x-risk.

Some strategies involve advocating for/introducing AI regulations without mentioning x-risk, with the hope of locally slowing down AI progress, building frameworks that can later be used to address x-risk, or fostering relationships with policymakers. Many of them carry significant downside risks and are net-negative.

Many people don't seem to consider politicians and voters to be grown-ups who can listen to arguments for why AI poses an x-risk, and implement AI regulation that slows down AI for the reasons we (and everyone who thought about the problem) want AI to be slowed down. These people propose regulations that they think can help with x-risk but don't present the regulations as motivated by x-risk. Aside from this being dishonest (don't be), it can backfire badly. The proposed regulations helping with other problems as well can be a nice bonus, but if addressing other problems is the only aim the policymakers have, you can end up with AI systems that are safe and ethical until they're smart enough to kill you.

Instead, you can explain the actual problem! Not necessarily your full thinking: obviously, it makes sense to simplify a lot. But the audience are not children; they're smart, and they can understand what you're talking about. And it's possible to reach them and get them to listen because you have a comparative advantage to every other problem that demands their time: many experts agree that yours is going to kill everyone, soon, unless something highly unusual is done; when it works, it produces a huge incentive for them to try to address this problem, and maybe find experts in AI regulation with proposals that can address the x-risk at hand.

It might be more important to carefully explain why x-risk is real than to propose specific regulation that can, as we know, help with x-risk (especially if we're not locked in a specific form and can get the policymakers to adjust it).

Why:

My guess is that historically, either the politicians trying to prevent technological progress have lost, or...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Does EA bring out the best in me?, published by Lorenzo Buonanno on August 28, 2023 on The Effective Altruism Forum.

I really liked the framing in the linked post. Some relevant quotes:

When I stay with my parents I regress back to being a teenager. I become irritable, messy and spend most of my time eating cheese toasties and watching The Simpsons. There's something about this social environment which brings out qualities in me that I don't like, and behaviours which I don't want [...]

Another example - there's a friend of mine who is quite a curious, laid-back type. And often when I'm with said friend, I somehow magically become more curious and laid-back. In this case, the social environment is bringing out qualities in me that I do like.

There are [...] things which EA brings out in me which I don't like.

I've spoken to a bunch of people that are doing something like 'trying to figure out their relationship with EA'. And often it seems like people end up trying to do something like 'work out whether EA is good or bad'.

This is a hard question to answer. I think an easier one, and a more action-relevant one is 'Does EA bring out the best in you?' or even 'What does EA bring out of you?'.

An overall theme here is that people are different depending on the social environment that they are in. This will be more true of some people than others. Some people will change a lot, like a chameleon, and other people will change only a little, like a cow, or a. wardrobe, or whatever the opposite of a chameleon is.

I found this a useful mindset I hadn't thought of, even if it seems obvious in hindsight. Many posts and books mention the value of engaging with the effective altruism movement to prevent value drift. But I had never reflected on what directions different parts of the movement are making me drift towards, or are preventing me from drifting to.

I can't trace back where I found it, it somehow made its way into my open tabs for this evening. Thank you to whoever shared it.

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Dear Self; we need to talk about ambition, published by Elizabeth on August 28, 2023 on LessWrong.

I keep seeing advice on ambition, aimed at people in college or early in their career, that would have been really bad for me at similar ages. Rather than contribute (more) to the list of people giving poorly universalized advice on ambition, I have written a letter to the one person I know my advice is right for: myself in the past.

The Letter

Dear Past Elizabeth,

Your life is, in some sense, a series of definitions of success.

First you're in early school, and success is defined for you by a handful of adults. You go where they say, do the assignments they say, when they say, and doing well means meeting the goals they set for you. Even your hippie elementary school gives you very few choices about life. You get choices in your leisure activity, but that (as they have explained to you) is leisure and thus unimportant, and there's no success or failure in it.

Then you get further in school, and the authorities give you some choice over the hoops you jump through. You can choose which book you write your report on or even what classes you take (within a predetermined set). This feels like freedom, but you're in still a system someone else designed and set the win conditions for. You can fulfill a college distribution requirement with any history class at all- but you are going to take one, and the professor is the one determining if you succeeded at it.

More insidiously, you'll like it. Creating your own definition of success feels scary;enacting it feels impossible. The fact that school lays out neat little hoops for you to jump through is a feature.

Work (you'll be a programmer) is where things get screwy. Programming contains multiple definitions of success (manager, principal, freelancing, development, testing, bigtech, start-up, money-maxing, altruistic projects.), and multiple ways to go about them. If your goals lie outside of programming altogether (art, parenting, travel..), it's relatively easy to work out a way to fund it via programming while still having the time to do what you want. Not trivial, but have you seen what people in other jobs go through? With programming it's at least possible.

But you like hoops. You're comfortable with hoops. So you're going to waste years chasing down various definitions of success within programming, and by the time you give up will be too exhausted to continue in it at all. I think you (I) should have considered "just chill while I figure shit out" much earlier, much more seriously. It was reasonable to give their way a try, just due to the sheer convenience if it had worked, but I should have learned faster.

Eventually you will break out of the Seattle bigtech bubble, and into the overlapping bubbles of effective altruism, lesswrong, and the bay area start-up scene. All of three of these contain a lot of people shouting "be ambitious!" and "be independent!". And because they shout it so loudly and frequently you will think "surely, now I am in a wide open world and not on a path". But you will be wrong, because "be ambitious (in ways the people say this understand and respect)" and "be independent (in ways they think are cool and not crazy)" are still hoops and still determined by other people, just one more level meta.

Like the programming path, the legible independent ambition path works for some people, but not you. The things you do when pushed to Think Big and Be Independent produce incidental learning at best, but never achieve anything directly. They can't, because you made up the goals to impress other people. This becomes increasingly depressing, as you fail at your alleged goals and at your real goal of impressing people.

So what do we do then? Give up on having goals? Only by their definition. What seems to wo...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Local charity evaluation: Lessons from the "Maximum Impact" Program in Israel, published by Yonatan Schoen on August 27, 2023 on The Effective Altruism Forum.

TL;DR

Maximum Impact program, led by Effective Altruism Israel, aims to significantly increase the impact of Israeli philanthropy by conducting nonprofit evaluations, identifying high-impact local nonprofits (even if their recipients are abroad), and promoting effective giving. Additionally, we aim to strengthen the community internally by developing a replicable model that could achieve these goals in other local communities and by building the Israeli EA community through action-oriented projects.

During the last year, we joined forces with promising non-profits and conducted and published 21 cost-effectiveness analyses. Together with an acclaimed team of expert judges, we identified 3 nonprofits as relatively cost-effective, with Smoke-Free Israel, a tobacco taxation nonprofit, which might be competitive with top charities. We learnt a lot from the pilot and have many ways to improve the program, but overall the pilot was successful: we identified high-impact local nonprofits and have engaged Israeli philanthropies interested in cost-effectiveness and in the results of the program. Finally, having a large-scale program on nonprofit effectiveness was a huge boost for EA Israel's community building, and many people heard about EA and became more involved because of the program. We're looking forward to continuing to maximize the impact of Israelis and to share the knowledge and experience we gained with the EA community.

Motivation

We identified that global effective giving platforms currently don't "meet donors where they are" and only offer recommendations on global nonprofits without being able to engage donors; Donors who are initially interested in local donations but can later be introduced and nudged to give to global high-impact nonprofits.

Israeli and Jewish donors abroad (~17.2bn per annum) currently donate almost entirely without evidence and cost-effectiveness estimations, as the Israeli non-profit ecosystem doesn't yet emphasize these as prerequisites for donations.

With almost no research currently available on local effectiveness and a strong bias of donors towards donating locally, there are not enough impact-based donations in the local nonprofit sector.

Based on discussions with EAs worldwide, we assume this problem is not unique to Israel and would like to find a cost-effective way to solve it.

What have we done so far?

Our research program

In our 10-month program, we chose 25 promising nonprofits from more than 160 applications from Israeli-based NGOs, 21 of which successfully published their first-ever cost-effectiveness reports. In the pilot program, we provided guidance and assistance to support non-profits to execute a cost-effectiveness analysis. After the research period, we sent the reports to a team of expert judges for review, and chose the most promising non-profits based on the expert review. We offered monetary incentives to encourage nonprofits to participate: 15 nonprofits received preliminary research grants of $6k, the 3 top outstanding nonprofits received $24k prizes, and $26k was allocated to a donation matching campaign to all nonprofits who publish a report. A private donor and an Infrastructure Fund grant funded the pilot.

Guiding nonprofits on evaluation and improvement

We implemented various tools and resources to help incentivize and enable non-profits to conduct robust effectiveness analyses, including:

An extensive set of templates and how-tos.

Research grants for 10 young academic researchers (MA and PHDs) from Israel's top universities to conduct research with the organizations and join the EA community during the process.

Personal guidance from skilled community...