The Nonlinear Library

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: AI, Animals, and Digital Minds 2024 - Retrospective, published by Constance Li on June 19, 2024 on The Effective Altruism Forum.

This is a retrospective of the AIADM 2024 Conference, Retreat, and Co-working in London.

Tl;dr: ~130 people joined together over the span of three days to learn, connect, and make progress towards making AI safe for nonhumans.

Background

This event followed in the footsteps of the

October 2023 Artificial Intelligence, Conscious Machines, and Animals: Broadening AI Ethics conference held at Princeton by Peter Singer, Tse Yip Fai, Leonie Bossert and Thilo Hagendorff. It was planned in a [formerly private] AI Coalition channel on the

Hive Slack that many of the attendees of the original conference were invited to for the purposes of continuing conversation. It was here that I discovered that running the conference in 2024 would be highly counterfactual because Peter Singer was retiring from Princeton and none of the previous organizers were planning on repeating it. We were able to get permission to hold the second iteration of the conference and even got a

promotional endorsement by Peter Singer. The process of planning took place over several months and was independently funded.

Objective

The goal for this event was to explore how we can develop AI technologies in a way that protects and benefits nonhuman animals and potentially sentient AI. We had the dual purpose of increasing the salience of the field of AI and Nonhumans and also getting potential leaders to network with one another.

To advance the former goal, the content and programming needed to be highly accessible so we made it hybrid, recorded as much as practical, created

website pages and

social media assets for speakers to help with pre-conference promotion and future SEO for the speakers, and added more event space as the RSVPs crept up. It was open to anyone who could offer value to furthering this field, including, but not limited to, thought leaders, researchers, industry workers, funders and hopeful future contributors.

Timing and location

All in-person events took place in London immediately after EAG London (May 31 - June 2)

June 3: Conference (hybrid: onsite - live room and overflow room, offsite - livestreaming, and virtual)

June 4-5: Retreat (virtual and in-person options that only synced up during

hour of livestreamed lightning talks)

June 6-11: Co-working (in person)

Attendees

There were 260+ applications for attendance. A directory was created of people who consented to have their application answers shared with other attendees. These included answers to questions such as: What is your experience or demonstrated interest in the topic? Why do you want to attend? What can you offer to other attendees?

Conference

The 1st day was a hybrid conference:

It took place at the London School of Economics (LSE), split between 2 rooms which had around 50-60 total attendees.It was also streamed live to Newspeak House which had 10-15 attendees. Around 50-70 people attended via zoom.

We had a very tight schedule and prioritized having more time for Q&A so each talk was only 10 - 20 minutes long and we grouped similar talks together as panels.

Sessions

AI for Animals

Towards Ethical AI: Collaborative Efforts for Animal Welfare under the EU's AI Act by Dr Fakhra Ikhlaq and Dr Saeed Ahmad (

watch session |

view slides)

AI's Potential to Map Animal Suffering: Insights from the Welfare Footprint Framework by Wladimir Alonso (

watch session |

view slides)

Animal-friendly AI = Misaligned AI by Bob Fischer (

watch session |

view slides)

Building Pro-Animal AI Systems panel by Sam Tucker and Sankalpa Ghose (

watch session |

view

slides)

AI in Farming panel by Virginie Simoneau-Gilbert, Amber Elise Sheldon, and Walter Veit (

watch session |

view

all

slides)

Digital Minds

Changing attitudes to...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Surviving Seveneves, published by Yair Halberstadt on June 19, 2024 on LessWrong.

Contains spoilers for the first couple of chapters of Seveneves

Highly speculative on my part, I know very little about most of these topics

In Seveneves Neal Stephenson does the classic sci-fi trick of assuming that exactly one thing in the universe is different, and seeing where that takes us. In his case that one thing is the moon has somehow exploded.

And where that takes us is the complete destruction of the earth. As the initially huge chunks of moon rock hit into each other they break into smaller and smaller pieces, and take up more and more space. Eventually this process increases exponentially, the loosely held collection of rocks that was the moon disperses into a planetary ring, and earth is bombarded by lunar leavings for 5000 years:

There will be so many [meteors] that they will merge into a dome of fire that will set aflame anything that can see it. The entire surface of the Earth is going to be sterilized. Glaciers will boil. The only way to survive is to get away from the atmosphere. Go underground, or go into space.

They have only two years to prepare. Which option should they take? The choice seems obvious!

But they respond with the absolutely batshit insane solution. They go into space. And not to mars, or some other friendly location. Low Earth Orbit..

This is a terrible choice for all sorts of reasons:

1. They are even more at risk of meteor collision there, since all meteors that hit earth pass through LEO, but at least the atmosphere protects earth from the small ones.

2. There's simply no way to get people up there at scale. No matter how you slice it, at most an insignificant fraction of people can get to LEO. We simply don't have the capacity to send rockets at scale, and two years is not enough time to develop and scale the technology enough to make a dent in the 7 billion people on earth.

3. To prepare as well as possible in two years, the earth economy will have to keep running and sending stuff up to space. But if people know they are going to die, and don't have any real chance of being one of the lucky survivors, why would they bother? I would expect the economy to collapse fairly rapidly, followed by looting, and the collapse of government structures.

4. There's a thousand things that can kill you in space, and just staying alive requires lots of advanced technology. If society isn't able to keep a highly technologically advanced society going in space, everyone will die.

5. Keeping a technologically advanced society going with a small number of people is essentially impossible.

6. Earth technology and processes often don't work in space since they rely on gravity. New technological processes will need to be developed just for space, but with only a tiny number of people able to work on them and extremely limited resources.

7. There are no new resources in LEO. There'll have to be 100% perfect recycling of the resources sent up from earth. But propellant has to be expelled every time they manoeuvre to avoid meteors, so this is impossible.

Stephenson works with these constraints, and comes up with what are IMO wildly optimistic assumptions about how society could function. Whatever. But the obvious solution, is to just go underground, which doesn't suffer from any of these problems:

1. The ground + atmosphere protects them from all but the largest of meteors.

2. Digging is well understood technology, and we can do it at scale. There's no reason why we wouldn't be able to create enough space underground for hundreds of millions, or even billions, of people in two years if everyone's lives depended on it.

3. Since people know they can survive, there are strong incentives to keep working, especially if money will be needed to buy one of the spaces in the un...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Ilya Sutskever created a new AGI startup, published by harfe on June 19, 2024 on LessWrong.

[copy of the whole text of the announcement on ssi.inc, not an endorsement]

Safe Superintelligence Inc.

Superintelligence is within reach.

Building safe superintelligence (SSI) is the most important technical problem of our time.

We have started the world's first straight-shot SSI lab, with one goal and one product: a safe superintelligence.

It's called Safe Superintelligence Inc.

SSI is our mission, our name, and our entire product roadmap, because it is our sole focus. Our team, investors, and business model are all aligned to achieve SSI.

We approach safety and capabilities in tandem, as technical problems to be solved through revolutionary engineering and scientific breakthroughs. We plan to advance capabilities as fast as possible while making sure our safety always remains ahead.

This way, we can scale in peace.

Our singular focus means no distraction by management overhead or product cycles, and our business model means safety, security, and progress are all insulated from short-term commercial pressures.

We are an American company with offices in Palo Alto and Tel Aviv, where we have deep roots and the ability to recruit top technical talent.

We are assembling a lean, cracked team of the world's best engineers and researchers dedicated to focusing on SSI and nothing else.

If that's you, we offer an opportunity to do your life's work and help solve the most important technical challenge of our age.

Now is the time. Join us.

Ilya Sutskever, Daniel Gross, Daniel Levy

June 19, 2024

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Suffering Is Not Pain, published by jbkjr on June 19, 2024 on LessWrong.

"Pain is inevitable; suffering is optional."

The motivation of this post is to address the persistent conflation between suffering and pain I have observed from members of the EA community, even amongst those who purport to be "suffering-focused" in their ethical motivations. In order to best address the problem of suffering, it is necessary to be clear about the difference between suffering and mere pain or ordinary displeasure.

The parable of the second arrow

In the Buddhist parable of the second arrow, the Buddha illustrates the distinction between suffering and pain with the tale of a man struck by two arrows. The first arrow represents the pain that life inevitably brings. The second arrow, however, represents the suffering that arises from his reaction to the pain.

The Buddha teaches that while the first arrow (pain) is unavoidable, the second arrow (suffering) is optional, and that by letting go of the resistance to the pain (aversion), one will not suffer the sting of the second arrow.

Defining pain and suffering

Pain: An unpleasant physical sensation or emotional experience.[1]

Suffering: The unsatisfactoriness that arises from craving, aversion, and clinging/attachment to sensations and experiences; dukkha.

I feel it is important to clarify at this point that, while the above definition of suffering derives from historically-Buddhist teachings about dukkha and its cause, I am not endorsing this definition because it is Buddhist but rather because I believe it best identifies suffering as it can actually be observed in phenomenal experience.

For those who are skeptical (possibly deeply so) about the claims and teachings of Buddhism, I ask that you consider the distinction I am advocating with reference to your own experience(s) of pain and suffering. While both pain and suffering are phenomena that "feel bad" experientially, I maintain that the sensations and experiences to which the terms/concepts "pain" and "suffering" respectively refer are actually distinct as differentiated by the above definitions.

As a tradition, Buddhism is almost entirely concerned with suffering, its cause, its cessation, and the way to its cessation, so I do not consider it far-fetched to think that the way(s) in which it describes suffering are quite useful in distinguishing it as it is to be found in actual experience.

Additionally, a distinction between pain and suffering has not only been made in the context of Buddhism. For examples of papers in the context of Western academic philosophy which argue for such a distinction, see Kauppinen (2019) and Massin (2017). Further, empirical work which investigates the effects of meditation on responses to painful experiences, such as Zeidan et al. (2011), Grant et al. (2011), and Perlman et al.

(2010), as well as studies investigating the effectiveness of therapeutic techniques like Cognitive Behavioral Therapy (CBT), such as Thorn et al. (2011), Ehde et al. (2014), and Wetherell et al. (2011), suggest that in changing perceptions of and reactions to pain, individuals may experience a reduction in suffering, even when the physical sensation of pain remains.

Thus, even outside the context of Buddhism, it seems there is strong evidence for there being a difference between pain and suffering as actually experienced.

Defining these terms clearly and accurately is crucial in differentiating between two concepts that are often conflated. By clearly defining pain and suffering, we can better understand their relationship and address suffering more effectively with the identification of its root causes.

The relationship between pain and suffering

Pain is not the cause of suffering. As illustrated by the parable of the second arrow and made clear in the above definitions o...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Apply Now to EAGxBerkeley 2024!, published by Sasha Berezhnoi on June 19, 2024 on The Effective Altruism Forum.

We are excited to announce that

EAGxBerkeley 2024 will take place September 7 - 8 at

Lighthaven in Berkeley, California.

The conference will focus on a diverse range of cause areas, local community building, and knowledge sharing. It will offer networking, workshops, presentations, and small group gatherings for around 500 attendees dedicated to addressing some of the world's greatest challenges.

For more details, you may read the rest of this post and visit our

website.

Who is this conference for?

EAGxBerkeley is an Effective Altruism conference organized especially for EA communities in the Bay Area and the Western USA. We are also open and happy to host people from the rest of the United States or even internationally, but keep in mind we will prioritize applicants who have a stake in our region. But as always - when unsure, please apply.

If you cannot come for all two days, please be aware that it is also possible - and still very valuable - to attend the conference for one day!

This conference is intended both for individuals who are relatively new to EA and those who have already professionally engaged with EA. Some ways to demonstrate a good fit for an effective altruism conference include:

Familiarity with core EA principles through completion of an Introductory Fellowship, reading the EA Handbook, or similar.

Demonstrable Plans for EA Involvement: Applicants should have clear and tangible plans for becoming highly involved in EA. This can be as simple as joining an EA chapter or similar initiatives.

Relevance to

EA's Top Cause Areas: Applicants with shared interests and experience in areas crucial to EA.

Examples: Working on PhD thesis in preventing nuclear proliferation, applying to internships in animal welfare, applying to jobs or starting a personal project or organization in an EA-related field.

Why attend?

Connect: Meet like-minded people passionate about making a positive impact.

Learn: Engage with groundbreaking ideas through workshops, talks, and interviews. More detailed information on event content will be available closer to the conference via updates to this

webpage.

Grow: Map out your professional path, identify next steps, and develop your application skills.

Find work: Introduce yourself to impactful organizations looking for new hires

Applications

Applications are now

open and will close on August 20th.

If you're excited to be a part of the conference but are unsure about applying, please err on the side of applying! We will try our best to admit anyone who can demonstrate in their application that they are interested in learning more about EA and are thoughtful, capable, and highly motivated.

The default ticket price is $100. Tickets are free for students. Discount and free ticket options are available to anyone who finds the ticket price outside their budget, with no questions asked.

Travel Expenses

We are prepared to reimburse travel expenses for some attendees. Travel support funds are limited, so please only request funding if you would not be able to attend without it. Whether or not you request travel funding will have no bearing on the outcome of your application. See our

travel policy for more details.

Contact Us

If you would like to suggest yourself or someone else as a speaker, or have an idea for a workshop for the event, please use

this form. Alternatively, you may reach out to our content lead, Sam Faber-Manning, via

[email protected].

If you have any other questions or comments, please email us at [email protected]

Apply now

We look forward to receiving your application!

Warmly,

the EAGxBerkeley 2024 Organizing Team

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please vis...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Beyond the Board: Exploring AI Robustness Through Go, published by AdamGleave on June 19, 2024 on The AI Alignment Forum.

Last year, we showed that supposedly superhuman Go AIs can be beaten by human amateurs playing specific "cyclic" patterns on the board. Vulnerabilities have previously been observed in a wide variety of sub- or near-human AI systems, but this result demonstrates that even far superhuman AI systems can fail catastrophically in surprising ways.

This lack of robustness poses a critical challenge for AI safety, especially as AI systems are integrated in critical infrastructure or deployed in large-scale applications. We seek to defend Go AIs, in the process developing insights that can make AI applications in various domains more robust against unpredictable threats.

We explored three defense strategies: positional adversarial training on handpicked examples of cyclic patterns, iterated adversarial training against successively fine-tuned adversaries, and replacing convolutional neural networks with vision transformers. We found that the two adversarial training methods defend against the original cyclic attack. However, we also found several qualitatively new adversarial strategies that can overcome all these defenses.

Nonetheless, finding these new attacks is more challenging than against an undefended KataGo, requiring more training compute resources for the adversary.

For more information, see our blog post, project website or paper.

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Meta Charity Funders: Second round retrospective, published by Joey on June 19, 2024 on The Effective Altruism Forum.

Previously:

Launching the meta charity funding circle (MCF),

Summary of Our First Grant Round and Path Forward,

Launching the 2nd round

Meta Charity Funders (MCF) is a funding circle that aims to fund charitable projects working one level removed from direct impact.

We think this post will be relevant for people who want to apply to MCF in the future and people who want to better understand the EA Meta funding landscape. The post is written by the MCF organisers (all authors of this post). Some of our members might not agree with everything said.

Summary

We received 109 applications and ultimately funded 11 projects: "AI Assurance Tech Report", an anonymous AI safety field-building initiative, an anonymous effective giving organization, an anonymous UHNWI initiative, EA Indonesia, EA New Zealand, EA Philippines, EA UK, Founders Pledge, Giving Multiplier, and High Impact Athletes. In total, our members granted $2,043,176 in this round, which is a significant step up from the $686,580 that was given in the last round.

The next grant round will open in late August 2024, and we will welcome similar applications as in the last round. We are especially excited about initiatives targeting ultra-high-net-worth-individuals and are excited to fund such initiatives. However, we want to stress that we believe the skillset to do this well is rare.

Information for the next round

The next round will open in late August, with grants given out in late November.

The application form will remain open but don't expect your application to be processed before September.

If you want to join Meta Charity Funders as a donor, please fill in

this form. Note that the minimum expected annual donation is $100,000. However, there is no obligation to donate if you don't think the opportunities presented are good enough, and during your first year, you are free to observe. If you have any questions, please contact us at [email protected].

Check out our

website to learn more about Meta Charity Funders and stay up-to-date with the new funding round.

How to increase your chances of success when applying

First, read

Summary of Our First Grant Round and Path Forward.

We generally felt that the applications this round were of higher quality than the last round, and we believe this was partly due to the information we shared in the last grant round. Fewer initiatives were desk rejected, and we had a larger pool of grants that we considered during the meetings.

We still want to strongly encourage all applicants to spend time on their

Theories of Change as this is where you set out your path to impact. The clearer you make the connection between us providing you funding and you making a positive impact on the world, the more likely you are to get funded.

Grants we made

Below we will provide a summary of the projects we ended up funding as stated by the applicants themselves, the extent to which we have chosen to fund them, their request, and our reasoning. Note that all funding decisions are made by individual members, and therefore do not necessarily reflect the priorities of the funding circle in its entirety.

General comments:

On local EA Orgs: Due to a re-focus from CEA, it has recently become difficult for smaller EA organizations to get funded. As such, some funders have chosen to cover the budgets of some local EA orgs.

On anonymity: Some organizations we have funded have chosen to remain anonymous. While we prefer to disclose what we fund openly, we made exceptions in these cases, since we agreed that it would be better all things considered.

We include the "of $X" to indicate that the grantees might still be looking for additional funds and encourage other funders to reach...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Loving a world you don't trust, published by Joe Carlsmith on June 18, 2024 on LessWrong.

(Cross-posted from my website. Audio version here, or search for "Joe Carlsmith Audio" on your podcast app.)

This is the final essay in a series that I'm calling "Otherness and control in the age of AGI." I'm hoping that the individual essays can be read fairly well on their own, but see here for a brief summary of the series as a whole. There's also a PDF of the whole series here.

Warning: spoilers for Angels in America; and moderate spoilers for Harry Potter and the Methods of Rationality.)

"I come into the presence of still water..."

~Wendell Berry

A lot of this series has been about problems with yang - that is, with the active element in the duality of activity vs. receptivity, doing vs. not-doing, controlling vs. letting go.[1] In particular, I've been interested in the ways that "deep atheism" (that is, a fundamental mistrust towards Nature, and towards bare intelligence) can propel itself towards an ever-more yang-y, controlling relationship to Otherness, and to the universe as a whole.

I've tried to point at various ways this sort of control-seeking can go wrong in the context of AGI, and to highlight a variety of less-controlling alternatives (e.g. "gentleness," "liberalism/niceness/boundaries," and "green") that I think have a role to play.[2]

This is the final essay in the series. And because I've spent so much time on potential problems with yang, and with deep atheism, I want to close with an effort to make sure I've given both of them their due, and been clear about my overall take. To this end, the first part of the essay praises certain types of yang directly, in an effort to avoid over-correction towards yin.

The second part praises something quite nearby to deep atheism that I care about a lot - something I call "humanism." And the third part tries to clarify the depth of atheism I ultimately endorse. In particular, I distinguish between trust in the Real, and various other attitudes towards it - attitudes like love, reverence, loyalty, and forgiveness. And I talk about ways these latter attitudes can still look the world's horrors in the eye.

In praise of yang

Let's start with some words in praise of yang.

In praise of black

Recall "black," from my essay on green. Black, on my construal of the colors, is the color for power, effectiveness, instrumental rationality - and hence, perhaps, the color most paradigmatically associated with yang. And insofar as I was especially interested in green qua yin, black was green's most salient antagonist.

So I want to be clear: I think black is great.[3] Or at least, some aspects of it. Not black qua ego. Not black that wants power and domination for its sake.[4] Rather: black as the color of not fucking around. Of cutting through the bullshit; rejecting what Lewis calls "soft soap"; refusing to pretend things are prettier, or easier, or more comfortable; holding fast to the core thing.

I wrote, in my essay on sincerity, about the idea of "seriousness." Black, I think, is the most paradigmatically serious color.

And it's the color of what Yudkowsky calls "the void" - that nameless, final virtue of rationality; the one that carries your movement past your map, past the performance of effort, and into contact with the true goal.[5] Yudkowsky cites Miyamoto Musashi:

The primary thing when you take a sword in your hands is your intention to cut the enemy, whatever the means... If you think only of hitting, springing, striking or touching the enemy, you will not be able actually to cut him. More than anything, you must be thinking of carrying your movement through to cutting him.

Musashi (image source here)

In this sense, I think, black is the color of actually caring. That is: one becomes serious, centrally, when there are stak...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Boycott OpenAI, published by PeterMcCluskey on June 18, 2024 on LessWrong.

I have canceled my OpenAI subscription in protest over OpenAI's lack of ethics.

In particular, I object to:

threats to confiscate departing employees' equity unless those employees signed a life-long non-disparagement contract

Sam Altman's pattern of lying about important topics

I'm trying to hold AI companies to higher standards than I use for typical companies, due to the risk that AI companies will exert unusual power.

A boycott of OpenAI subscriptions seems unlikely to gain enough attention to meaningfully influence OpenAI. Where I hope to make a difference is by discouraging competent researchers from joining OpenAI unless they clearly reform (e.g. by firing Altman). A few good researchers choosing not to work at OpenAI could make the difference between OpenAI being the leader in AI 5 years from now versus being, say, a distant 3rd place.

A year ago, I thought that OpenAI equity would be a great investment, but that I had no hope of buying any. But the value of equity is heavily dependent on trust that a company will treat equity holders fairly. The legal system helps somewhat with that, but it can be expensive to rely on the legal system. OpenAI's equity is nonstandard in ways that should create some unusual uncertainty.

Potential employees ought to question whether there's much connection between OpenAI's future profits and what equity holders will get.

How does OpenAI's behavior compare to other leading AI companies?

I'm unsure whether Elon Musk's xAI deserves a boycott, partly because I'm unsure whether it's a serious company. Musk has a history of breaking contracts that bears some similarity to OpenAI's attitude. Musk also bears some responsibility for SpaceX requiring non-disparagement agreements.

Google has shown some signs of being evil. As far as I can tell, DeepMind has been relatively ethical. I've heard clear praise of Demis Hassabis's character from Aubrey de Grey, who knew Hassabis back in the 1990s. Probably parts of Google ought to be boycotted, but I encourage good researchers to work at DeepMind.

Anthropic seems to be a good deal more ethical than OpenAI. I feel comfortable paying them for a subscription to Claude Opus. My evidence concerning their ethics is too weak to say more than that.

P.S. Some of the better sources to start with for evidence against Sam Altman / OpenAI:

a lengthy Zvi post about one week's worth of evidence

Leopold Aschenbrenner

Geoffrey Irving

But if you're thinking of working at OpenAI, please look at more than just those sources.

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: On DeepMind's Frontier Safety Framework, published by Zvi on June 18, 2024 on LessWrong.

On DeepMind's Frontier Safety Framework

Previously: On OpenAI's Preparedness Framework, On RSPs.

The First Two Frameworks

To first update on Anthropic and OpenAI's situation here:

Anthropic's RSP continues to miss the definitions of the all-important later levels, in addition to other issues, although it is otherwise promising. It has now been a number of months, and it is starting to be concerning that nothing has changed. They are due for an update.

OpenAI also has not updated its framework.

I am less down on OpenAI's framework choices than Zac Stein-Perlman was in the other review I have seen. I think that if OpenAI implemented the spirit of what it wrote down, that would be pretty good. The Critical-level thresholds listed are too high, but the Anthropic ASL-4 commitments are still unspecified. An update is needed, but I appreciate the concreteness.

The bigger issue with OpenAI is the two contexts around the framework.

First, there's OpenAI. Exactly.

A safety framework you do not adhere to is worth nothing. A safety framework where you adhere to the letter but not the spirit is not worth much.

Given what we have learned about OpenAI, and their decision to break their very public commitments about committing compute to superalignment and driving out their top safety people and failure to have a means for reporting safety issues (including retaliating against Leopold when he went to the board about cybersecurity) and also all that other stuff, why should we have any expectation that what is written down in their framework is meaningful?

What about the other practical test? Zac points out that OpenAI did not share the risk-scorecard for GPT-4o. They also did not share much of anything else. This is somewhat forgivable given the model is arguably not actually at core stronger than GPT-4 aside from its multimodality. It remains bad precedent, and an indication of bad habits and poor policy.

Then there is Microsoft. OpenAI shares all their models with Microsoft, and the framework does not apply to Microsoft at all. Microsoft's track record on safety is woeful. Their submission at the UK Summit was very weak. Their public statements around safety are dismissive, including their intention to 'make Google dance.' Microsoft Recall shows the opposite of a safety mindset, and they themselves have been famously compromised recently.

Remember Sydney? Microsoft explicitly said they got safety committee approval for their tests in India, then had to walk that back. Even what procedures they have, which are not much, they have broken. This is in practice a giant hole in OpenAI's framework.

This is in contrast to Anthropic, who are their own corporate overlord, and DeepMind, whose framework explicitly applies to all of Google.

The DeepMind Framework

DeepMind finally has its own framework. Here is the blog post version.

So first things first. Any framework at all, even a highly incomplete and unambitious one, is far better than none at all. Much better to know what plans you do have, and that they won't be enough, so we can critique and improve. So thanks to DeepMind for stepping up, no matter the contents, as long as it is not the Meta Framework.

There is extensive further work to be done, as they acknowledge. This includes all plans on dealing with misalignment. The current framework only targets misuse.

With that out of the way: Is the DeepMind framework any good?

In the Framework, we specify protocols for the detection of capability levels at which models may pose severe risks (which we call "Critical Capability Levels (CCLs)"), and articulate a spectrum of mitigation options to address such risks. We are starting with an initial set of CCLs in the domains of Autonomy, Biosecurity, Cybersec...