Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Reading list on AI agents and associated policy, published by Peter Wildeford on August 10, 2024 on The Effective Altruism Forum.

I put together this reading list on AI agents, with a tilt more towards AI governance. I hope it may be of help to others working on this area. If you know of other great resources or have comments on these resources, please say so in the comments!

Top priority papers:

OpenAI

Practices for Governing Agentic AI Systems

OpenAI

Research into Agentic AI Systems

Deepmind

The Ethics of Advanced AI Assistants

Alan Chan et. al

Visibility into AI Agents (

see also)

A bit about the agents that currently exist:

VisualWebArena: EVALUATING MULTIMODAL AGENTS ON REALISTIC VISUAL WEB TASKS

WebVoyager : Building an End-to-End Web Agent with Large Multimodal Models

METR:

An update on our general capability evaluations

Good background:

METR

Autonomous replication threat models

Evaluating Frontier Models for Dangerous Capabilities

Introduction to Cooperative AI

Governing AI Agents

Peter's thoughts on AI loss of control and paths forward

Request for proposals: benchmarking LLM agents on consequential real-world tasks

Lower priority papers (and a tweet):

Foundational Challenges in Assuring Alignment and Safety of Large Language Models p31-35

Regulated Advanced Artificial Agents

Harms from Increasingly Agentic Algorithmic Systems

Frontier AI ethics

https://twitter.com/sebkrier/status/1777081791855186191

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org