Examples of AI safety progress

Training AIs to behave safely and beneficially is difficult. They might learn to game their reward function, deceive human oversight, or seek power. Some argue that researchers have not made much progress in addressing these problems, but here we offer a few examples of progress on AI safety.

Detecting lies in AI outputs. Language models often output false text, but a recent paper suggests they understand the truth in ways not reflected in their output. By analyzing a model’s internals, we can calculate the likelihood that a model believes a statement is true. The finding has been replicated in models that answer [...]

---

Outline:

(00:13) Examples of AI safety progress

(03:56) Yoshua Bengio proposes a ban on AI agents

(07:19) Lessons from Nuclear Arms Control for Verifying AI Treaties

(10:02) Links

---

First published:

Source:

---

Want more? Check out our ML Safety Newsletter for technical safety research.

Narrated by TYPE III AUDIO.

Geoffrey Hinton is concerned about existential risks from AI

Geoffrey Hinton won the Turing Award for his work on AI. Now he says that part of him regrets his life’s work, as he believes that AI poses an existential threat to humanity. As Hinton puts it, “it’s quite conceivable that humanity is just a passing phase in the evolution of intelligence.”

AI is developing more rapidly than Hinton expected. In 2015, Andrew Ng argued that worrying about AI risk is like worrying about overpopulation on Mars. Geoffrey Hinton also used to believe that advanced AI was decades away, but recent progress has changed his views. Now he says [...]

---

Outline:

(00:12) Geoffrey Hinton is concerned about existential risks from AI

(02:32) White House meets with AI labs

(04:22) Trojan Attacks on Language Models

(06:51) Assorted Links

---

First published:

Source:

---

Want more? Check out our ML Safety Newsletter for technical safety research.

Narrated by TYPE III AUDIO.

Cybersecurity Challenges in AI Safety

Meta accidentally leaks a language model to the public. Meta’s newest language model, LLaMa, was publicly leaked online against the intentions of its developers. Gradual rollout is a popular goal with new AI models, opening access to academic researchers and government officials before sharing models with anonymous internet users. Meta intended to use this strategy, but within a week of sharing the model with an approved list of researchers, an unknown person who had been given access to the model publicly posted it online.

How can AI developers selectively share their models? One inspiration could be the film industry, which places watermarks and tracking technology on “screener” copies of movies sent [...]

---

Outline:

(00:11) Cybersecurity Challenges in AI Safety

(02:48) Artificial Influence: An Analysis Of AI-Driven Persuasion

(05:37) Building Weapons with AI

(07:47) Assorted Links

---

First published:

Source:

---

Want more? Check out our ML Safety Newsletter for technical safety research.

Narrated by TYPE III AUDIO.

Policy Proposals for AI Safety

Critical industries rely on the government to protect consumer safety. The FAA approves new airplane designs, the FDA tests new drugs, and the SEC and CFPB regulate risky financial instruments. Currently, there is no analogous set of regulations for AI safety.

This could soon change. President Biden and other members of Congress have recently been vocal about the risks of artificial intelligence and the need for policy solutions.

From guiding principles to enforceable laws. Previous work on AI policy such as the White House Blueprint for an AI Bill of Rights and the NIST AI Risk Management Framework has articulated guiding principles like interpretability, robustness, and privacy. But these recommendations are not enforceable – AI [...]

---

Outline:

(00:09) Policy Proposals for AI Safety

(04:19) Competitive Pressures in AI Development

---

First published:

Source:

---

Want more? Check out our ML Safety Newsletter for technical safety research.

Narrated by TYPE III AUDIO.

ChaosGPT and the Rise of Language Agents

Chatbots like ChatGPT usually only respond to one prompt at a time, and a human user must provide a new prompt to get a new response. But an extremely popular new framework called AutoGPT automates that process. With AutoGPT, the user provides only a high-level goal, and the language model will create and execute a step-by-step plan to accomplish the goal.

AutoGPT and other language agents are still in their infancy. They struggle with long-term planning and repeat their own mistakes. Yet because they limit human oversight of AI actions, these agents are a step towards dangerous deployment of autonomous AI.

Individual bad actors [...]

---

Outline:

(00:12) ChaosGPT and the Rise of Language Agents

(02:49) Natural Selection Favors AIs over Humans

(05:17) AI Safety in the Media

---

First published:

Source:

---

Want more? Check out our ML Safety Newsletter for technical safety research.

Narrated by TYPE III AUDIO.

Growing concerns about rapid AI progress

Recent advancements in AI have thrust it into the center of attention. What do people think about the risks of AI?

The American public is worried. 46% of Americans are concerned that AI will cause “the end of the human race on Earth,” according to a recent poll by YouGov. Young people are more likely to express such concerns, while there are no significant differences in responses between people of different genders or political parties. Another poll by Monmouth University found broad support for AI regulation, with 55% supporting the creation of a federal agency that governs AI similar to how the FDA approves drugs and [...]

---

Outline:

(00:12) Growing concerns about rapid AI progress

(02:53) Plugging ChatGPT into email, spreadsheets, the internet, and more

(05:35) Which jobs could be affected by language models?

---

First published:

Source:

---

Want more? Check out our ML Safety Newsletter for technical safety research.

Narrated by TYPE III AUDIO.