Machine Learning Archives - Software Engineering Daily

OpenAI: Compute and Safety with Dario Amodei


Listen Later

Applications of artificial intelligence are permeating our everyday lives. We notice it in small ways–improvements to speech recognition; better quality products being recommended to us; cheaper goods and services that have dropped in price because of more intelligent production.

But what can we quantitatively say about the rate at which artificial intelligence is improving? How fast are models advancing? Do the different fields in artificial intelligence all advance together, or are they improving separately from each other? In other words, if the accuracy of a speech recognition model doubles, does that mean that the accuracy of image recognition will double also?

It’s hard to know the answer to these questions.

Machine learning models trained today can consume 300,000 times the amount of compute that could be consumed in 2012. This does not necessarily mean that models are 300,000 times better–these training algorithms could just be less efficient than yesterday’s models, and therefore are consuming more compute.

We can observe from empirical data that models tend to get better with more data. Models also tend to get better with more compute. How much better do they get? That varies from application to application, from speech recognition to language translation. But models do seem to improve with more compute and more data.

Dario Amodei works at OpenAI, where he leads the AI safety team. In a post called “AI and Compute,” Dario observed that the consumption of machine learning training runs is increasing exponentially–doubling every 3.5 months. In this episode, Dario discusses the implications of increased consumption of compute in the training process.

Dario’s focus is AI safety. AI safety encompasses both the prevention of accidents and the prevention of deliberate malicious AI application.

Today, humans are dying in autonomous car crashes–this is an accident. The reward functions of social networks are being exploited by botnets and fake, salacious news–this is malicious. The dangers of AI are already affecting our lives on the axes of accidents and malice.

There will be more accidents, and more malicious applications–the question is what to do about it. What general strategies can be devised to improve AI safety? After Dario and I talk about the increased consumption of compute by training algorithms, we explore the implications of this increase for safety researchers.

The post OpenAI: Compute and Safety with Dario Amodei appeared first on Software Engineering Daily.

...more
View all episodesView all episodes
Download on the App Store

Machine Learning Archives - Software Engineering DailyBy Machine Learning Archives - Software Engineering Daily

  • 4.4
  • 4.4
  • 4.4
  • 4.4
  • 4.4

4.4

69 ratings


More shows like Machine Learning Archives - Software Engineering Daily

View all
The Changelog: Software Development, Open Source by Changelog Media

The Changelog: Software Development, Open Source

291 Listeners

Data Skeptic by Kyle Polich

Data Skeptic

478 Listeners

Software Engineering Daily by Software Engineering Daily

Software Engineering Daily

623 Listeners

Talk Python To Me by Michael Kennedy

Talk Python To Me

587 Listeners

Soft Skills Engineering by Jamison Dance and Dave Smith

Soft Skills Engineering

284 Listeners

Super Data Science: ML & AI Podcast with Jon Krohn by Jon Krohn

Super Data Science: ML & AI Podcast with Jon Krohn

301 Listeners

Python Bytes by Michael Kennedy and Brian Okken

Python Bytes

214 Listeners

NVIDIA AI Podcast by NVIDIA

NVIDIA AI Podcast

342 Listeners

Machine Learning Guide by OCDevel

Machine Learning Guide

768 Listeners

Syntax - Tasty Web Development Treats by Wes Bos & Scott Tolinski - Full Stack JavaScript Web Developers

Syntax - Tasty Web Development Treats

984 Listeners

DataFramed by DataCamp

DataFramed

268 Listeners

Practical AI by Practical AI LLC

Practical AI

211 Listeners

AWS Podcast by Amazon Web Services

AWS Podcast

203 Listeners

Google DeepMind: The Podcast by Hannah Fry

Google DeepMind: The Podcast

195 Listeners

This Day in AI Podcast by Michael Sharkey, Chris Sharkey

This Day in AI Podcast

209 Listeners