AI + a16z

Building the Next Generation of Conversational AI


Listen Later

In this episode of AI + a16z, Sesame Cofounder and CTO Ankit Kumar joins a16z general partner Anjney Midha for a deep dive into the research and engineering behind their voice technology. They discuss the technical challenges of real-time speech generation, the trade-offs in balancing personality with efficiency, and why the team is open-sourcing key components of their model. Ankit breaks down the complexities of multimodal AI, full-duplex conversation modeling, and the computational optimizations that enable low-latency interactions. 

They also explore the evolution of natural language as a user interface and its potential to redefine human-computer interaction.
Plus, we take audience questions on everything from scaling laws in speech synthesis to the role of in-context learning in making AI voices more expressive.

Key Takeaways:
How Sesame AI achieves natural voice interactions through real-time speech generation.

  • The impact of open-sourcing their speech model and what it means for AI research.
  • The role of full-duplex modeling in improving AI responsiveness.
  • How computational efficiency and system latency shape AI conversation quality.
  • The growing role of natural language as a user interface in AI-driven experiences.

For anyone interested in AI and voice technology, this episode offers an in-depth look at the latest advancements pushing the boundaries of human-computer interaction.

Learn more:

The Maya + Miles demo

Crossing the uncanny valley of conversational voice

Sesame CSM 1B model

Follow everybody on X:

Ankit Kumar

Anjney Midha

Check out everything a16z is doing with artificial intelligence here, including articles, projects, and more podcasts.

...more
View all episodesView all episodes
Download on the App Store

AI + a16zBy a16z

  • 4.6
  • 4.6
  • 4.6
  • 4.6
  • 4.6

4.6

26 ratings


More shows like AI + a16z

View all
This Week in Startups by Jason Calacanis

This Week in Startups

1,266 Listeners

a16z Podcast by Andreessen Horowitz

a16z Podcast

999 Listeners

The Twenty Minute VC (20VC): Venture Capital | Startup Funding | The Pitch by Harry Stebbings

The Twenty Minute VC (20VC): Venture Capital | Startup Funding | The Pitch

509 Listeners

Invest Like the Best with Patrick O'Shaughnessy by Colossus | Investing & Business Podcasts

Invest Like the Best with Patrick O'Shaughnessy

2,294 Listeners

Y Combinator Startup Podcast by Y Combinator

Y Combinator Startup Podcast

207 Listeners

Practical AI by Practical AI LLC

Practical AI

188 Listeners

The Logan Bartlett Show by by Redpoint Ventures

The Logan Bartlett Show

190 Listeners

web3 with a16z crypto by a16z crypto, Sonal Chokshi, Chris Dixon

web3 with a16z crypto

61 Listeners

No Priors: Artificial Intelligence | Technology | Startups by Conviction

No Priors: Artificial Intelligence | Technology | Startups

127 Listeners

Latent Space: The AI Engineer Podcast by swyx + Alessio

Latent Space: The AI Engineer Podcast

65 Listeners

The AI Daily Brief (Formerly The AI Breakdown): Artificial Intelligence News and Analysis by Nathaniel Whittemore

The AI Daily Brief (Formerly The AI Breakdown): Artificial Intelligence News and Analysis

428 Listeners

The Ben & Marc Show by Marc Andreessen, Ben Horowitz

The Ben & Marc Show

120 Listeners

BG2Pod with Brad Gerstner and Bill Gurley by BG2Pod

BG2Pod with Brad Gerstner and Bill Gurley

432 Listeners

Lightcone Podcast by Y Combinator

Lightcone Podcast

20 Listeners

Training Data by Sequoia Capital

Training Data

37 Listeners