
Sign up to save your podcasts
Or
Ion Stoica helped define the modern data stack. Now he’s coming for AI evaluation. From co-founding Databricks and Anyscale to launching LMArena, Ion has shaped the infrastructure underlying some of the biggest shifts in computing. In this conversation, he unpacks what most people get wrong about model evaluation, the infrastructure challenges ahead for agents and heterogeneous compute, and why he believes the U.S. is structurally disadvantaged in open-source AI compared to China.
(0:00) Intro
(0:49) Launching a New Startup: LMArena
(1:01) The Origin of the Vicuna Model
(1:54) Challenges in Model Evaluation
(6:33) Becoming a Company
(7:47) Expanding Evaluation Capabilities
(13:48) The Importance of Human-Based Evaluations
(18:56) Open Source vs. Proprietary Models
(23:05) Infrastructure and Collaboration Challenges
(28:22) China's Strategic Advantages in Technology
(29:54) Opportunities in AI Infrastructure
(31:50) Challenges in AI Model Optimization
(35:49) The Role of Data in AI Enterprises
(39:31) Reflections on AI Progress and Predictions
(50:40) Quickfire
With your co-hosts:
@jacobeffron
- Partner at Redpoint, Former PM Flatiron Health
@patrickachase
- Partner at Redpoint, Former ML Engineer LinkedIn
@ericabrescia
- Former COO Github, Founder Bitnami (acq’d by VMWare)
@jordan_segall
- Partner at Redpoint
4.9
4949 ratings
Ion Stoica helped define the modern data stack. Now he’s coming for AI evaluation. From co-founding Databricks and Anyscale to launching LMArena, Ion has shaped the infrastructure underlying some of the biggest shifts in computing. In this conversation, he unpacks what most people get wrong about model evaluation, the infrastructure challenges ahead for agents and heterogeneous compute, and why he believes the U.S. is structurally disadvantaged in open-source AI compared to China.
(0:00) Intro
(0:49) Launching a New Startup: LMArena
(1:01) The Origin of the Vicuna Model
(1:54) Challenges in Model Evaluation
(6:33) Becoming a Company
(7:47) Expanding Evaluation Capabilities
(13:48) The Importance of Human-Based Evaluations
(18:56) Open Source vs. Proprietary Models
(23:05) Infrastructure and Collaboration Challenges
(28:22) China's Strategic Advantages in Technology
(29:54) Opportunities in AI Infrastructure
(31:50) Challenges in AI Model Optimization
(35:49) The Role of Data in AI Enterprises
(39:31) Reflections on AI Progress and Predictions
(50:40) Quickfire
With your co-hosts:
@jacobeffron
- Partner at Redpoint, Former PM Flatiron Health
@patrickachase
- Partner at Redpoint, Former ML Engineer LinkedIn
@ericabrescia
- Former COO Github, Founder Bitnami (acq’d by VMWare)
@jordan_segall
- Partner at Redpoint
1,273 Listeners
1,040 Listeners
519 Listeners
217 Listeners
88 Listeners
426 Listeners
186 Listeners
121 Listeners
75 Listeners
461 Listeners
31 Listeners
22 Listeners
43 Listeners
35 Listeners
13 Listeners