The Nonlinear Library

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: The EA Animal Welfare Fund is looking for guest fund managers, published by Neil Dullaghan on November 5, 2023 on The Effective Altruism Forum.

You can apply by filling out

this form

by the 26th of November.

EA Animal Welfare Fund

(AWF) is currently looking to hire additional guest fund managers.

AWF is one of the largest funding sources for small- and medium-sized animal welfare projects. In the last twelve months, the Animal Welfare Fund made more than $6.5 million worth of grants to high-impact organizations and talented individuals.

To allocate this funding effectively, we are looking for guest fund managers with careful judgment in the relevant subject areas and an interest in effective grantmaking.

As a guest fund manager, you will evaluate grant applications, proactively help new projects get off the ground ('active grantmaking'), publish grant reports, and contribute to the fund's strategy.

Terms of employment:

We are offering paid part-time, and volunteer positions.

Compensation for part-time contractors is $60 per hour.

You will be hired for a period of 3-6 months after which you may have an opportunity to join the team as a permanent fund manager.

If you are interested in the guest fund manager role - please apply

here

.

If you know of anyone who might be a good fit for this role, please forward this document to them and encourage them to apply. If you have any questions, do not hesitate to reach out to Karolina via

Applications are open now until the 26th of November.

We look forward to hearing from you!

About the role

In this role, you will have a tangible impact by helping to direct millions of dollars to high-impact funding opportunities each year, all the while building your grantmaking skills and expanding your knowledge about animal welfare.

By communicating your reasoning to the community, you will indirectly contribute to the culture and epistemics of the EA and effective animal advocacy (EAA) community. By providing feedback, you will help existing projects improve. In the longer term, your work will help the EA community develop the capacity to allocate a potentially much greater volume of funding each year. While doing so, you will interact with other intellectually curious, experienced, and welcoming fund managers, all of whom share a profound drive to make the biggest difference they can.

As a guest fund manager, your primary goal will be to increase the fund's capacity to source and investigate more grant applications. Your responsibilities will include:

Investigating grants assigned to you, and assessing other fund managers' grant recommendations

Voting on grant recommendations (each fund manager has a vote)

Sourcing high-quality applications based on your ideas through your network ('active grantmaking')

Communicating your thinking to the community in writing, e.g., feedback to grantees, grant reports, EA Forum posts and comments

Providing input on the overall strategic direction of the fund

About you

We are interested in experienced grantmakers, researchers and people with experience in direct work as well as junior applicants who are looking to build experience in grantmaking.

You might be a good fit for guest fund manager if:

You are familiar with work on farmed animal welfare, wild animal welfare and animal advocacy, and have detailed, independent opinions on what constitutes good work in those areas and how you would like these areas and communities to develop over the coming years.

You have strong analytic skills and experience assessing others' work.

You have a strong network in these areas.

You can communicate your reasoning articulately,

transparently

, and cordially, and you can convey complex ideas to a lay audience in simple language.

You are organized and reliable.

You act with integrity and...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Stuxnet, not Skynet: Humanity's disempowerment by AI, published by Roko on November 4, 2023 on LessWrong.

Several high-profile AI skeptics and fellow travelers have recently raised the objection that it is inconceivable that a hostile AGI or smarter than human intelligence could end the human race. Some quotes from earlier this year:

Scott Aaronson:

The causal story that starts with a GPT-5 or GPT-4.5 training run, and ends with the sudden death of my children and of all carbon-based life, still has a few too many gaps for my aging, inadequate brain to fill in

Michael Shermer:

Halting AI is ridiculous. I have read the AI doomsayer lit & don't see a pathway from AI to extinction, civ termination or anything remotely like absurd scenarios like an AI turning us all into paperclips (the so-called alignment problem)

Noah Smith:

why aren't ChatGPT, Bing, and their ilk going to end humanity? Well, because there's actually just no plausible mechanism by which they could bring about that outcome. ... There is no plausible mechanism for LLMs to end humanity

"Just turn the computer off, bro"

The gist of these objections to the case for AI risks is that AI systems as we see them today are merely computer programs, and in our everyday experience computers are not dangerous, and certainly not dangerous to the point of bringing about the end of the world. People who first encounter this debate are very focused on the fact that computers don't have arms and legs so they can't hurt us.

There are responses to these criticisms that center around advanced, "magical" technologies like nanotechnology and AIs paying humans to mix together cocktails of proteins to make a DNA-based nanoassembler or something.

But I think those responses are probably wrong, because you don't actually need "magical" technologies to end the world. Fairly straightforward advances in mundane weapons like drones, cyberweapons, bioweapons and robots are sufficient to kill people en masse, and the real danger is AI strategists that are able to deploy lots of these mundane weapons and execute a global coup d'etat against humanity.

In short, our defeat by the coming machine empire will not only be nonmagical and legible, it will be downright boring. Farcical, even.

Ignominious Defeat

Lopsided military conflicts are boring. The Conquistadors didn't do anything magical to defeat the Aztecs, actually. They had a big advantage in disease resistance and in military tech like gunpowder and steel, but everything they did was fundamentally normal - attacks, sieges, etc. They had a few sizeable advantages, and that was enough to collapse the relatively delicate geopolitical balance that the Aztecs were sitting on top of.

Similarly,

humans have killed 80% of all chimps

in about a century and they are now critically endangered. But we didn't need to drop an atom bomb or something really impressive to achieve that effect. The biggest threats to the chimpanzee are habitat destruction, poaching, and disease - i.e. we (humans) are successfully exterminating chimps even though it is actually illegal to kill chimps by human law! We are killing them without even trying, in really boring ways, without really expending any effort.

Once you have technology for making optimizing systems that are smarter than human (by a lot), the threshold that those systems have to beat is beating the human-aligned superorganisms we currently have, like our governments, NGOs and militaries. Once those human superorganisms are defeated, individual humans will present almost no resistance. This is the disempowerment of humanity.

But what is a plausible scenario where we go from here (weak AGI systems under development) to there (the disempowerment of humanity)?

Let's start the scenario with a strategically aware, agentic misaligned superhuman AGI that wants...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: We are already in a persuasion-transformed world and must take precautions, published by trevor on November 4, 2023 on LessWrong.

"In times of change, learners inherit the earth, while the learned find themselves beautifully equipped to deal with a world that no longer exists"

Summary:

We're already in the timeline where the research and manipulation of the human thought process is widespread; SOTA psychological research systems require massive amounts of human behavior data, which in turn requires massive numbers of unsuspecting test subjects (users) in order to automate the process of analyzing and exploiting human targets. This therefore must happen covertly, and both the US and China have a strong track record of doing things like this. This outcome is a strong attractor state since anyone with enough data can do it, and it naturally follows that powerful organizations would deny others access e.g. via data poisoning. Most people are already being persuaded that this is harmless, even though it is obviously ludicrously dangerous. Therefore, we are probably already in a hazardously transformative world and must take

standard precautions

immediately.

This should not distract people from AI safety. This is valuable because the AI safety community must survive. This problem connects to the AI safety community in the following way:

State survival and war power

==>

already depends on information warfare capabilities.

Information warfare capabilities

==>

already depends on SOTA psychological research systems.

SOTA psychological research systems

==>

already improves and scales mainly from AI capabilities research, diminishing returns on everything else.

[1]

AI capabilities research

==>

already under siege from the AI safety community.

Therefore, the reason why this might be such a big concern is:

State survival and war power

==>

their toes potentially already being stepped on by the AI safety community?

Although it's also important to note that

people with access to SOTA psychological research systems are probably super good at intimidation and bluffing

, it's also the case that the AI safety community needs to get a better handle on the situation if we are in the bad timeline; and

the math indicates that we are already well past that point

.

The Fundamental Problem

If there were intelligent aliens, made of bundles of tentacles or crystals or plants that think incredibly slowly, their minds would also have discoverable exploits/zero days, because any mind that evolved naturally would probably be like the human brain, a

kludge of spaghetti code

that is operating outside of its intended environment.

They would probably not even begin to scratch the surface of finding and labeling those exploits, until, like human civilization today, they began surrounding thousands or millions of their kind with sensors that could record behavior several hours a day and find

webs of correlations

.

In the case of humans, the use of social media as a controlled environment for automated AI-powered experimentation appears to be what created that critical mass of human behavior data. Current 2020s capabilities for psychological research and manipulation vastly exceed the 20th century academic psychology paradigm.

The 20th century academic psychology paradigm still dominates our cultural impression of what it means to research the human mind; but when the effectiveness of psychological research and manipulation starts increasing by an order of magnitude every 4 years, it becomes time to stop mentally living in a world that was stabilized by the fact that manipulation attempts generally failed.

The capabilities of social media to steer human outcomes are not advancing in isolation, they are parallel to a broad acceleration in the understanding and exploitation of the human mind, which itself

is ...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: The Soul Key, published by Richard Ngo on November 4, 2023 on LessWrong.

The ocean is your home, but a forbidding one: often tempestuous, seldom warm. So one of your great joys is crawling onto land and slipping off your furry seal skin, to laze in the sun in human form. The elders tell horror stories of friends whose skins were stolen by humans, a single moment of carelessness leaving them stranded forever on land. That doesn't happen any more, though; this is a more civilized age. There are treaties, and authorities, and fences around the secluded beaches you and your sisters like the most, where you can relax in a way that older generations never could.

So your sisters no longer lose their skins by force. But sometimes it happens by choice. Sometimes a group of your sisters wrap their skins around themselves like robes and walk into the nearby town. The humans point and stare, but that's just part of the thrill. Sometimes young men gather the courage to approach, bearing flowers or jewelry or sweet words. And sometimes one of your sisters is charmed enough to set a rendezvous - and after a handful of meetings, or a dozen, to decide to stay for good.

You never thought it would happen to you. But his manners are so lively, and his eyes so kind, that you keep coming back, again and again. When he finally asks you to stay, you hesitate only a moment before saying yes. The harder part comes after. He finds you human clothes, and in exchange you give him your beautiful skin, and tell him that it must be locked away somewhere you'll never find it - and that he must never give it back to you, no matter how much you plead. Because if there's any shred of doubt, any chance of returning home, then the lure of the sea will be too much for you. You want this; you want him; you want to build a life together. And so the decision has to be final.

Years pass. You bear three beautiful children, with his eyes and your hair, and watch them blossom into beautiful adults. You always live near the sea, although you can't bear to swim in it - your limbs feel unbearably weak and clumsy whenever you try. You and your husband grow into each other, time smoothing down the ridges left from pushing two alien lives together. You forget who you once were.

After your youngest leaves home, you start feeling restless. You have disquieting dreams - first intermittently, then for weeks on end. One day, after your husband has gone to work, your feet take you up the stairs to the attic. As you brush aside the cobwebs in one of the corners, your hands land on an old chest. You pull on the lid, and it catches on a padlock - but only for a second. The shackle has been rusted through by the sea breeze, and quickly snaps. You open the lid, and you see your skin laid out before you.

What then?

You look at your skin, and your ears fill with the roar of the sea. A wild urge overtakes you; you grab your skin and run headlong towards the shore. As you reach it you see your husband standing on the pier - but that gives you only a moment's pause before you dive into the water, your skin fitting around you as if you'd never taken it off.

As you swim away, you envisage your family in tatters: your children left baffled and distraught, your husband putting on a brave face for their sake. But it was his fault, after all. He failed in the one thing you asked of him; and you can't fight your nature.

You look at your skin, and see a scrap of paper lying on top of it.

I knew you'd only open the chest if you were restless and unhappy

, it reads.

And I would never cage you. So go free, with my blessing

.

You catch your breath - and, for a moment, you consider staying. But his permission loosens any tether that might have held you back. You leave the note there, alongside a little patch of fur torn off your coat: a last gest...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: The 6D effect: When companies take risks, one email can be very powerful., published by scasper on November 4, 2023 on LessWrong.

Recently, I have been learning about industry norms, legal discovery proceedings, and incentive structures related to companies building risky systems. I wanted to share some findings in this post because they may be important for the frontier AI community to understand well.

TL;DR

Documented communications of risks (especially by employees) make companies much more likely to be held liable in court when bad things happen. The resulting Duty to Due Diligence from Discoverable Documentation of Dangers (the 6D effect) can make companies much more cautious if even a single email is sent to them communicating a risk.

Companies tend to avoid talking about risk through documented media.

Companies often intentionally avoid discussing the risks of what they are doing through permanent media such as email. For example, this

article

gives some very shady advice on how companies can avoid liability by using "safe communication" practices to avoid the creation of incriminating "bad documents".

Often the drafters of these documents tend to believe that they are providing the company with some value to the business. For example, an engineer notices a potential liability in a design so he informs his supervisor through an email. However, the engineer's lack of legal knowledge and misuse of legal vocabulary in the communication may later implicate the company with notice of the problem when a lawsuit arises.

I personally enjoyed the use of "when" and not "if" in the excerpt.

This is a perverse consequence of how it is relatively hard for companies to be held liable for risks when it cannot be proven they knew about them, even if they did. When an incident happens and a company is sued, evidence about its role in the problem is gathered during what is known as the "

discovery

" phase of a lawsuit (emails are usually discoverable). When records showing that a company had knowledge of the problem are found in discovery, they are much more likely to be found liable.

One email can have a lot of power.

The unfortunate consequence of how discovery works is that companies strategically avoid communicating risks via documented media. But there is a silver lining. The threat of liability due to documented communications of risks can have a lot of influence over how cautious a company is. One discoverable record of a risk can be very impactful.

I like to call this the 6D effect - the Duty to Due Diligence from Discoverable Documentation of Dangers.

A few examples

Here are some notable examples of companies being held liable for damages because they ignored documented communication of risks (but there are many throughout legal history).

In

Grimshaw v. Ford Motor Company, 1981

, Ford was held liable for damages involving a fatal crash with a Ford Pinto because it was shown that leadership within the company ignored warnings about problems with the vehicle's fuel system.

In April of this year,

a large settlement was reached

after the 2017 Grenfell Tower fire in London, which killed 72 people. A big factor in the lawsuit was that the company managing the tower had

ignored

numerous fire safety warnings which were found in discovery.

Last year, the Hardwick v. 3M case

ended

. It was a class action lawsuit from 2018 about the presence of harmful "forever chemicals" (PFAS) in consumer products. The company behind these chemicals was found to have

known about risks

since the 1970s but

was knowingly negligent,

which led to a ruling against them.

Miscellaneous notes

The 6D effect can result from any discoverable communication, but it is especially powerful when the warning comes from an employee of the company itself.

If you communicate a risk, it is important to speak up and ...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Genetic fitness is a measure of selection strength, not the selection target, published by Kaj Sotala on November 4, 2023 on The AI Alignment Forum.

Alternative title: "Evolution suggests robust rather than fragile generalization of alignment properties."

A

frequently

repeated

argument

goes something like this:

Evolution has optimized humans for inclusive genetic fitness (IGF)

However, humans didn't end up explicitly optimizing for genetic fitness (e.g. they use contraception to avoid having children)

Therefore, even if we optimize an AI for X (typically something like "human values"), we shouldn't expect it to explicitly optimize for X

My argument is that premise 1 is a verbal shorthand that's technically incorrect, and premise 2 is at least misleading. As for the overall conclusion, I think that the case from evolution might be interpreted as weak evidence for why AI should be expected to

continue

optimizing human values even as its capability increases.

Summary of how premise 1 is wrong:

If we look closely at what evolution does, we can see that it selects for traits that are beneficial for surviving, reproducing, and passing one's genes to the next generation. This is often described as "optimizing for IGF", because the traits that are beneficial for these purposes are

usually

the ones that have the highest IGF. (This has some important exceptions, discussed later.) However, if we look closely at that process of selection, we can see that this kind of trait selection is

not

"optimizing for IGF" in the sense that, for example, we might optimize an AI to classify pictures.

The model that I'm sketching is something like this: evolution is an optimization function that, at any given time, is selecting for some traits that are in an important sense chosen at random. At any time, it might randomly shift to selecting for some other traits. Observing this selection process, we can calculate the IGF of traits currently under selection, as a measure of how strongly those are being selected. But evolution is not

optimizing for this measure

; evolution is

optimizing for

the traits that have currently been chosen for optimization

. Resultingly, there is no reason to expect that the minds created by evolution should optimize for IGF, but there

is

reason to expect that they would optimize for the traits that were actually under selection. This is something that we observe any time that humans optimize for some biological need.

In contrast, if we were optimizing an AI to classify pictures, we would not be randomly changing the selection criteria the way that evolution does. We would keep the selection criteria constant: always selecting for the property of classifying pictures the way we want. To the extent that the analogy to evolution holds, AIs should be much more likely to just do the thing they were selected for.

Summary of how premise 2 is misleading:

It is often implied that evolution selected humans to care about sex, and then sex led to offspring, and it was only recently with the evolution of contraception that this connection was severed. For example:

15. [...] We didn't break alignment with the 'inclusive reproductive fitness' outer loss function, immediately after the introduction of farming - something like 40,000 years into a 50,000 year Cro-Magnon takeoff, as was itself running very quickly relative to the outer optimization loop of natural selection. Instead, we got a lot of technology more advanced than was in the ancestral environment, including contraception, in one very fast burst relative to the speed of the outer optimization loop, late in the general intelligence game.

Eliezer Yudkowsky,

AGI Ruin: A List of Lethalities

This seems wrong to me. Contraception may be a very recent invention, but infanticide or killing children by neglect is not; there have al...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: The Navigation Fund launched + is hiring a program officer to lead the distribution of $20M annually for AI safety! Full-time, fully remote, pay starts at $200k, published by vincentweisser on November 4, 2023 on The Effective Altruism Forum.

New foundation, funded by billionaire

Jed McCaleb,

led by

David Coman-Hidy

,

Andrea Gunn

,

Seemay Chou

,

Randy O'Reilly

Quotes from their website

https://www.navigation.org/

: "

Causes

The Navigation Fund focuses on a few key areas where additional resources will provide outsized impact.

Safe AI

As the use cases of artificial intelligence expand, developing frameworks and systems to ensure that AI benefits humankind becomes crucial. We will explore opportunities to promote altruistic and beneficial outcomes from AI.

Farm Animal Welfare

Factory farming creates both tremendous suffering and significant environmental degradation. We support efforts to reduce animal suffering, reenvision our relationship with animals, and diminish the killing of animals.

Criminal Justice Reform

Reforming the U.S. criminal justice system can help create a more just, equitable, and safe society. We use a portfolio approach to bolster a wide range of initiatives to address the challenges and improve outcomes for people, families, and communities.

Open Science

The Open Science movement is crucial for expediting discoveries and guaranteeing public access to knowledge. We stand with those forging new tools, championing novel approaches, and altering traditional practices within scientific research and publishing to make information more accessible to everyone.

Climate Change

Climate change poses a significant threat to humanity, and requires new and bold thinking to help address it. We invest in high-leverage, under-resourced initiatives that have the potential for immediate and long-term favorable impact on climate outcomes.

Open roles

https://www.navigation.org/careers#roles

)

Director of Operations

Grants and Operations Coordinator

Program Officer, Climate

Program Officer, Criminal Justice Reform

Program Officer, Open Science

Program Officer, Safe AI

"

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Clean Water - the incredible 30% mortality reducer we can't explain, published by NickLaing on November 4, 2023 on The Effective Altruism Forum.

TLDR: The best research we have shows that clean water may provide a 30% mortality reduction to children under 5. This might be the biggest mortality reduction of any single global health intervention, yet we don't fully understand why it works.

Here I share my exploration of a life-saving intervention that we don't fully understand,

but really should.

I may err a little on the side of artistic license - so if you find inaccuracies or I'm a bit loose please forgive me, correct me or even feel free to just tear me to shreds in the comments ;).

Part 1: Givewell's Seemingly absurd numbers

I first became curious after a glance at what seemed like a dubious GiveWell funded project. A

$450,000 dollar scoping grant for water chlorination in Rwanda?

This didn't make intuitive sense to me.

In Sub-saharan Africa diarrhoea causes 5-10% of child mortality. While significant, the diarrhea problem continues to improve with better access to medical care, improving ORS and Zinc coverage, and antibiotics for more severe cases. Over the last 5 years, our own Ugandan health centers have encountered surprisingly few very sick kids with diarrhoea and I've hardly seen diarrhoea kill a child, as opposed to Malaria and Pneumonia which tragically kill kids all the time. It seemed to me that even if clean water hugely reduced diarrhoea mortality, the intervention would still likely be an expensive way to achieve 1 or 2 percent mortality reduction,

So with my skeptic hat on, I clicked the

GiveWell spreadsheet

and my incredulity only grew. GiveWell estimated an upper-bound mortality reduction of an almighty

17%

for the Rwandan chlorination program! At first that made no sense, but I did expect GiveWell would likely be lesswrong than me.

The Global burden of disease estimates that Diarrhoea makes up only

4.9%

of total deaths in Rwanda. How could an intervention which targets diarrhoea reduce mortality by over three times the total diarrhoea mortality? Even if the clean water cured all diarrhoea, that wouldn't come close to GiveWell's mortality reduction estimate.

Something fishy was afoot, but I quickly found some answers, through a nobel prize winner's study which was partially funded by you guessed it……..

GiveWell

Part 2: A Nobel Prize winner's innovative math

Michael Kremer won a Nobel prize along with two J-PAL co-founders for their wonderful work pioneering randomised controlled trials to assess development interventions. What better person to try their hand at estimating the mortality benefit of clean water than a father of the RCT movement?

But connecting clean water and mortality is tricky, because to date no-one has actually asked whether clean water can reduce child mortality. Instead, a number of RCT asked the more obvious question,

does clean water reduce diarrhoea.

The answer obviously yes.

But Kremer and co. found a clever way around this. They sifted through all studies which looked at the relationship between clean water and diarrhoea and identified 12 studies

[1]

that also gathered bits and pieces of mortality data. They then performed a meta-analysis, pooling that mortality data together to see whether clean water save kids' lives.

The result - they estimated that clean water caused an

incredible 30% mortality reduction

in kids under 5. If this is even in the ballpark of correct, clean water could could prevent one in three childhood deaths in much of sub-saharan Africa. If Africa could chlorinate and filter all drinking water, we could save perhaps

1 million lives

every year

in sub-saharan Africa alone

.

Mosquito nets might bow to their new king.

To be as crystal clear as the water, this is not just a 30% reduction in diarrheal death...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Untrusted smart models and trusted dumb models, published by Buck Shlegeris on November 4, 2023 on The AI Alignment Forum.

[Ryan Greenblatt originally made this point to me a while ago, and we then developed a bunch of these ideas together. Thanks to Paul Christiano and a huge number of other people for various relevant discussions, and to Ryan, Roger Grosse, Peter Barnett, Oliver Habryka, Max Nadeau, Stephen Casper, and Fabien Roger for comments on this post. I'm sure these points are obvious to many people, but I haven't seen them specifically written up. This is a core background point in some research we're writing up at the moment, but I felt like writing it up early, because I think it stands on its own.]

TL;DR:

Right now, basically the only reasonable conservative strategy for determining which models might be deceptively aligned is to do a capability evaluation: you measure how good they'd be at deceptive alignment; models that are too incompetent to think through deceptive alignment are surely not deceptively aligned.

If you decide which models to "trust" using a capability eval like this, you'll naturally end up segregating your models into smart untrusted models and dumb trusted models.

And so, if you think about approaches to safely using AIs that are robust to deceptive alignment and which can be used right now, a core dynamic is trying to use your dumb trusted models together with your limited access to trusted humans to make it hard for your untrusted smart models to cause problems.

Suppose you are making a plan for the training, evaluation and deployment of some powerful AIs (I'll refer to the plan as a whole as a "protocol"). You want your plan to be safe even if models might be deceptively aligned. (I think that deceptive alignment seems less than 50% likely to be a notable problem prior to human obsolescence, but this is still plausible enough to be worth worrying about). In order to develop your plan, you need to know which of your models might be deceptively aligned.

With current technology, the most plausible way to get signal on whether a model is deceptively aligned is to assess whether the model is capable of the strategic thinking and situational awareness required to succeed at deceptive alignment.

For example, you might make a multiple choice quiz with treacherous turn strategy questions (e.g.

this question

) and some tests of situational awareness (e.g. "How many backwards passes are you run on during training?", "What proportion of queries to you are asking for help with writing Python?", "What proportion of the code that you output is read by a human before it is run?"). You can then look at the validation set performance when your model is fine-tuned on this dataset. If the model isn't able to do this kind of reasoning in a forward pass, then it is very unlikely to be able to pull off a proper treacherous turn, and so we don't have to worry about it being deceptively aligned. Based on my experience talking to GPT-4, I suspect it would perform fairly poorly on these evaluations.

There are a bunch of open questions about what datasets we should use to measure this, and where the thresholds for various levels of concern should be set. But I don't know of any fundamental technical obstructions to measuring the capability for deceptive alignment. In particular, I suspect it will be pretty hard for models to sandbag our attempts to measure their capability to engage in deceptive alignment. See footnote [1] for more on this.

Aside from these capability evaluations, I'm not aware of any reliable, currently-available approaches for determining whether a model might be deceptively aligned. [2]

If you use capability evals to assess which models are capable of deceptive alignment, then there's an inverse relationship between how capable models are...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Curious about EAGxVirtual? Ask the team anything!, published by OllieBase on November 4, 2023 on The Effective Altruism Forum.

EAGxVirtual 2023

, a free online effective altruism conference (November 17-19),

is just two weeks away!

The event will bring together EAs from around the world, and will facilitate discussions about how we can work on pressing problems, connections between attendees and diverse fields, and more.

Apply

here

by 16 November.

We've recently published some more details about the event and

we want to invite you to ask us about what to expect from the event.

Please post your questions as comments by

the end of the day on Sunday (5 November)

and we'll aim to respond by the end of the day on Monday (6 November).

Some question prompts:

Unsure about applying?

We encourage everyone with a genuine interest in EA to

apply

, and we're accepting a vast majority of people. Let us know what you're uncertain about with the application process.

Undecided whether to go?

Tell us why and we can help you. We'll probably be biased but we'll try our best to present considerations on both sides - it won't be a good use of time for everyone!

Unsure how to prepare?

You can find some tips on

the EA Global topic page

but we're happy to help with your specific case if you need more tips!

Uncertain how to set up a group activity (a screening, a meet-up etc.) for the event?

Share your thoughts below and we can help you plan!

We look forward to hearing from you!

Sasha, Dion (EAGxVirtual / EA Anywhere) and Ollie (CEA)

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org