Unsupervised Learning with Jacob Effron

Lin Qiao, the co-founder of Fireworks.ai, sits down for a deep dive into the future of AI. Lin ran the PyTorch team at Meta, which developed some of the most fundamental open-source AI software in use today. She’s got a riveting perspective on the AI landscape that is a must-listen.

[0:00] Intro

[1:06] Fireworks: Revolutionizing AI Inference

[2:12] Challenges in AI Model Development

[4:05] The Future of AI: Compound Systems

[4:32] Designing Effective AI Tools

[10:26] Customization and Fine-Tuning in AI

[14:06] Human-in-the-Loop Automation

[16:38] Evaluating AI Models

[19:18] Building Complex AI Systems

[21:18] Function Calling and AI Orchestration

[26:52] AI Infrastructure and Hardware

[31:08] Small Expert Models

[31:27] Hyperscalers and Resource Management

[32:14] Inference Systems and Scalability

[33:08] Running Models Locally: Cost and Privacy

[35:20] Open Source Models and Meta's Role

[36:41] The Evolution of AI Training and Inference

[38:04] Fireworks' Vision and Market Strategy

[40:46] The Impact of Generative AI

[45:18] AI Research and Future Trends

[46:58] Building for a Rapidly Changing AI Landscape

[49:36] Quickfire

With your co-hosts: 

@jacobeffron 

- Partner at Redpoint, Former PM Flatiron Health 

@patrickachase 

- Partner at Redpoint, Former ML Engineer LinkedIn 

@ericabrescia 

- Former COO Github, Founder Bitnami (acq’d by VMWare) 

@jordan_segall 

- Partner at Redpoint

Jacob is an AI investor at Redpoint Ventures. He's led Redpoint's investments in companies like Abridge, Physical Intelligence & Legora. Follow Jacob on Twitter (@jacobeffron).

On Unsupervised Learning we probe the sharpest minds in AI in search for the truth about what's real today, what will be real in the future and what it all means for businesses and the world. If you're a builder, researcher or investor navigating the AI world, this podcast will help you deconstruct and understand the most important breakthroughs and see a clearer picture of reality. Subscribe to this show to stay up to date on our latest episodes.

Noam Brown, renowned AI researcher and key figure at OpenAI, joins us for a deep dive into the o1 release. Recorded just one day before o1’s full public debut, this episode explores the groundbreaking advancements and challenges behind this innovative test-time compute model.

We discuss the technical breakthroughs that set o1 apart, its unique capabilities compared to previous models, and how it disrupts traditional paradigms in AI development. Noam also shares insights into OpenAI’s approach to innovation, the economic realities of scaling AI, and what the future holds for the field.

[0:00] Intro

[0:50] Scaling Model Capabilities and Economic Constraints

[2:48] Excitement Around Test Time Compute

[4:50] Challenges and Future Directions in AI Research

[8:11] Noam Brown's Journey and OpenAI's Research Focus

[16:08] The Role of Specialized Models and Tools

[21:18] Unexpected Use Cases and Future Milestones

[23:44] Proof of Concept: o1's Capabilities

[24:48] The Bitter Lesson: Insights from Richard Sutton

[25:59] Scaffolding Techniques and Their Future

[27:56] Challenges in Academia and AI Research

[30:30] Evaluating AI Models: Metrics and Trends

[34:47] The Role of AI in Social Sciences

[39:39] AI Agents and Emergent Communication

[40:17] Future of AI Robotics

[41:13] Advancing Scientific Research with AI

[43:30] Quickfire

With your co-hosts: 

@jacobeffron 

- Partner at Redpoint, Former PM Flatiron Health 

@patrickachase 

- Partner at Redpoint, Former ML Engineer LinkedIn 

@ericabrescia 

- Former COO Github, Founder Bitnami (acq’d by VMWare) 

@jordan_segall 

- Partner at Redpoint

Jacob is an AI investor at Redpoint Ventures. He's led Redpoint's investments in companies like Abridge, Physical Intelligence & Legora. Follow Jacob on Twitter (@jacobeffron).

On Unsupervised Learning we probe the sharpest minds in AI in search for the truth about what's real today, what will be real in the future and what it all means for businesses and the world. If you're a builder, researcher or investor navigating the AI world, this podcast will help you deconstruct and understand the most important breakthroughs and see a clearer picture of reality. Subscribe to this show to stay up to date on our latest episodes.

For this episode of Unsupervised Learning we spoke with Russ d'Sa, co-founder of LiveKit, a company at the forefront of voice AI technology. Russ thinks of LiveKit as a “nervous system,” powering the sensory interfaces humans use to interact with AI – including the Advanced Voice feature in ChatGPT as well as applications like Character.ai, Spotify and many more. 

Russ talked about when voice makes sense as an interface, the exciting new UX paradigms on the horizon, the intersection of voice and robotics and Anthropic's Computer Use API.

[0:00] Intro

[0:24] Using ChatGPT Voice in Daily Life

[2:26] How LiveKit Works with ChatGPT Voice

[5:16] LiveKit as the Nervous System for AI

[8:11] Future of Work and AI Interfaces

[18:31] Emerging Use Cases for Voice AI

[22:29] AI Models in Customer Support

[23:10] Latency Improvements in AI

[24:37] Challenges in System Integration

[26:01] Multimodal AI and Browser Integration

[29:40] Telephony and AI in Healthcare

[32:11] Humanoid Robotics and On-Device AI

[33:50] Cloud vs. On-Device Inference

[36:50] Quickfire

With your co-hosts: 

@jacobeffron 

- Partner at Redpoint, Former PM Flatiron Health 

@patrickachase 

- Partner at Redpoint, Former ML Engineer LinkedIn 

@ericabrescia 

- Former COO Github, Founder Bitnami (acq’d by VMWare) 

@jordan_segall 

- Partner at Redpoint

Jacob is an AI investor at Redpoint Ventures. He's led Redpoint's investments in companies like Abridge, Physical Intelligence & Legora. Follow Jacob on Twitter (@jacobeffron).

On Unsupervised Learning we probe the sharpest minds in AI in search for the truth about what's real today, what will be real in the future and what it all means for businesses and the world. If you're a builder, researcher or investor navigating the AI world, this podcast will help you deconstruct and understand the most important breakthroughs and see a clearer picture of reality. Subscribe to this show to stay up to date on our latest episodes.

Jonathan Frankle is the Chief AI Scientist at Databricks ($43B), which he joined through the acquisition of MosaicML in July 2023. Databricks has over 12,000 customers on the cutting edge of AI; Jonathan works to anticipate their needs and offer solutions even as the tech is rapidly evolving.

[0:00] Intro

[0:52] Incentives and Team Motivation at Databricks

[2:40] The Evolution of AI Models: Transformers vs. LSTMs

[5:27] Mosaic and Databricks: A Strategic Merger

[7:31] Guidance on AI Model Training and Fine-Tuning

[11:11] Building Effective AI Evaluations

[16:02] Domain-Specific AI Models and Their Importance

[19:37] The Future of AI: Challenges and Opportunities

[25:07] Ethical Considerations and Human-AI Interaction

[29:13] Customer Collaboration and AI Implementation

[30:45] Navigating AI Tools and Techniques

[35:41] The Role of Open Source Models

[36:46] AI Infrastructure and Partnerships

[48:27] Academia's Role in AI Research

[52:09] Ethics and Policy in AI

[57:47] Quickfire

With your co-hosts: 

@jacobeffron 

- Partner at Redpoint, Former PM Flatiron Health 

@patrickachase 

- Partner at Redpoint, Former ML Engineer LinkedIn 

@ericabrescia 

- Former COO Github, Founder Bitnami (acq’d by VMWare) 

@jordan_segall 

- Partner at Redpoint

Jacob is an AI investor at Redpoint Ventures. He's led Redpoint's investments in companies like Abridge, Physical Intelligence & Legora. Follow Jacob on Twitter (@jacobeffron).

On Unsupervised Learning we probe the sharpest minds in AI in search for the truth about what's real today, what will be real in the future and what it all means for businesses and the world. If you're a builder, researcher or investor navigating the AI world, this podcast will help you deconstruct and understand the most important breakthroughs and see a clearer picture of reality. Subscribe to this show to stay up to date on our latest episodes.

I sat down with DeepL cofounder Jarek Kutylowski. DeepL is a comprehensive Language AI platform that enables organizations to communicate effectively across languages, cultures, and markets. Jarek shared a treasure trove of insights on the past, present, and future of AI translation. Here were some standout moments:

[0:00] Intro
[0:38] The Rise of AI and DeepL's Journey
[1:41] DeepL's Competitive Edge and Market Impact
[2:41] Innovations in AI Translation
[4:39] DeepL's Product and Use Cases
[7:35] Challenges and Strategies in AI Translation
[14:29] Human Translators and Data Labeling
[24:39] Building and Scaling AI Infrastructure
[32:52] Evaluating AI Models: Objective vs Subjective
[34:05] Translation Quality Metrics
[35:35] The Debate on AI Moats and Specialized Models
[40:16] The Impact of Real-Time Translation on Business
[45:05] Challenges in Developing Synchronous Speaking Models
[46:49] Adjacent AI Technologies and Their Potential
[48:33] Quickfire

With your co-hosts: 

@jacobeffron 

- Partner at Redpoint, Former PM Flatiron Health 

@patrickachase 

- Partner at Redpoint, Former ML Engineer LinkedIn 

@ericabrescia 

- Former COO Github, Founder Bitnami (acq’d by VMWare) 

@jordan_segall 

- Partner at Redpoint

Jacob is an AI investor at Redpoint Ventures. He's led Redpoint's investments in companies like Abridge, Physical Intelligence & Legora. Follow Jacob on Twitter (@jacobeffron).

On Unsupervised Learning we probe the sharpest minds in AI in search for the truth about what's real today, what will be real in the future and what it all means for businesses and the world. If you're a builder, researcher or investor navigating the AI world, this podcast will help you deconstruct and understand the most important breakthroughs and see a clearer picture of reality. Subscribe to this show to stay up to date on our latest episodes.

Joshua Xu is co-founder and CEO of HeyGen -- the fast-growing AI video creation and translation platform. You can upload a video, or create a new one from a script using an AI avatar as your star, and HeyGen will translate it into 175 languages. HeyGen now serves over 40,000 customers and is generating $35+ million in revenue.

[0:00] Intro  

[0:37] HeyGen's Viral Moments  

[1:23] Creating Magic with AI  

[3:30] The Future of AI in Video Production  

[9:29] HeyGen's Use Cases and Customer Base  

[13:15] AI Avatars  

[25:46] The Future of Content  

[26:43] Competing with Industry Giants  

[27:16] Innovating for New Markets  

[31:24] Enterprise Push: Lessons and Surprises  

[33:07] Trust and Safety in AI  

[37:03] Fundraising and Financial Strategies for AI Startups  

[41:22] Quickfire

With your co-hosts: 

@jacobeffron 

- Partner at Redpoint, Former PM Flatiron Health 

@patrickachase 

- Partner at Redpoint, Former ML Engineer LinkedIn 

@ericabrescia 

- Former COO Github, Founder Bitnami (acq’d by VMWare) 

@jordan_segall 

- Partner at Redpoint

Jacob is an AI investor at Redpoint Ventures. He's led Redpoint's investments in companies like Abridge, Physical Intelligence & Legora. Follow Jacob on Twitter (@jacobeffron).

On Unsupervised Learning we probe the sharpest minds in AI in search for the truth about what's real today, what will be real in the future and what it all means for businesses and the world. If you're a builder, researcher or investor navigating the AI world, this podcast will help you deconstruct and understand the most important breakthroughs and see a clearer picture of reality. Subscribe to this show to stay up to date on our latest episodes.

Percy Liang is a Stanford professor and co-founder of Together AI, driving some of the most critical advances in AI research. 

Percy is also a trained classical pianist, which clearly influences the way he thinks about technology. We explored the evolution of AI from simple token prediction to autonomous agents capable of long-term problem-solving, the problem of interpretability, and the future of AI safety in complex, real-world systems.

[0:00] Intro

[0:46] Discussing OpenAI's O1 Model  

[2:21] The Evolution of AI Agents  

[3:27] Challenges and Benchmarks in AI  

[4:38] Compatibility and Integration Issues  

[6:17] The Future of AI Scaffolding  

[10:05] Academia's Role in AI Research  

[15:17] AI Safety and Holistic Approaches  

[18:32] Regulation and Transparency in AI  

[21:42] Generative Agents and Social Simulations  

[29:14] The State of AI Evaluations  

[32:07] Exploring Evaluation in Language Models  

[35:13] The Challenge of Interpretability  

[39:31] Innovations in Model Architectures  

[43:18] The Future of Inference and Customization  

[46:46] Milestones in AI Research and Reasoning  

[49:43] Robotics and AI: The Road Ahead  

[52:24] AI in Music: A Harmonious Future  

[55:52] AI's Role in Education and Beyond  

[56:30] Quickfire

[59:16] Jacob and Pat Debrief

With your co-hosts: 

@jacobeffron 

- Partner at Redpoint, Former PM Flatiron Health 

@patrickachase 

- Partner at Redpoint, Former ML Engineer LinkedIn 

@ericabrescia 

- Former COO Github, Founder Bitnami (acq’d by VMWare) 

@jordan_segall 

- Partner at Redpoint

Jacob is an AI investor at Redpoint Ventures. He's led Redpoint's investments in companies like Abridge, Physical Intelligence & Legora. Follow Jacob on Twitter (@jacobeffron).

On Unsupervised Learning we probe the sharpest minds in AI in search for the truth about what's real today, what will be real in the future and what it all means for businesses and the world. If you're a builder, researcher or investor navigating the AI world, this podcast will help you deconstruct and understand the most important breakthroughs and see a clearer picture of reality. Subscribe to this show to stay up to date on our latest episodes.

Douwe’s contributions to AI are truly a part of its bedrock foundations. He wrote the first paper on retrieval-augmented generation (RAG) and has raised over $100 million to help enterprises build contextual language models that fit their use cases. Before Contextual he was the head of research at Hugging Face, worked on the Facebook AI research team (i.e. Llama) and remains a professor at Stanford. Douwe was incredibly open about his take on AI’s recent history and where he thinks it’s going.

[0:00] Intro
[0:51] Exploring the Impact of Systems Thinking in AI
[1:49] Latency Constraints and AI Deployments
[2:05] Benchmarks and Real-World Applications
[3:27] Transition to Contextual and Company Vision
[5:12] Challenges and Innovations in Enterprise AI
[8:51] The Evolution and Future of RAG
[15:26] Alignment and Reinforcement Learning in AI
[23:52] Collaborations and the Role of Academia
[29:15] The Evolving Role of AI Developers
[30:19] Changing Perspectives in AI Research
[30:44] Synthetic Data and Agentic Workflows
[33:47] The Future of Multimodal Data
[35:31] Reasoning Capabilities in AI Models
[42:56] The Rise of Multi-Agent Systems
[45:24] Hugging Face and the AI Ecosystem
[46:59] Building Contextual and AI Startups
[49:51] The Future of AI and Personalized Entertainment
[50:41] Quickfire Round: Overhyped and Underhyped AI
[56:25] Final Thoughts and Parting Words

With your co-hosts: 

@jacobeffron 

- Partner at Redpoint, Former PM Flatiron Health 

@patrickachase 

- Partner at Redpoint, Former ML Engineer LinkedIn 

@ericabrescia 

- Former COO Github, Founder Bitnami (acq’d by VMWare) 

@jordan_segall 

- Partner at Redpoint

Jacob is an AI investor at Redpoint Ventures. He's led Redpoint's investments in companies like Abridge, Physical Intelligence & Legora. Follow Jacob on Twitter (@jacobeffron).

On Unsupervised Learning we probe the sharpest minds in AI in search for the truth about what's real today, what will be real in the future and what it all means for businesses and the world. If you're a builder, researcher or investor navigating the AI world, this podcast will help you deconstruct and understand the most important breakthroughs and see a clearer picture of reality. Subscribe to this show to stay up to date on our latest episodes.

If you don’t know Grammarly, it’s a personalized AI assistant for writing that has over 30 Million Daily Active Users. Grammarly has been building AI productivity tooling long before the most recent GenAI wave and has raised over $400M, with a current valuation of $13B. Rahul believes AI will enable everyone to focus on more meaningful, creative interactions by automating the "drudgery" of daily tasks. It was interesting to hear him talk about how he thinks about competition and his longer-term perspective on how AI will be adopted by the enterprise.

[0:00] Intro
[1:03] The Future of AI in Human Communication
[3:47] Grammarly's Evolution and Product Overview
[8:21] Limitations of LLMs
[15:31] The Impact of ChatGPT and Future Prospects
[23:52] Fine-Tuning AI for User Needs
[30:16] Competitive Landscape and Differentiators
[39:14] AI in Education
[46:25] Over-hyped/Under-hyped
[49:57] Most Exciting AI Startups

With your co-hosts: 

@jacobeffron 

- Partner at Redpoint, Former PM Flatiron Health 

@patrickachase 

- Partner at Redpoint, Former ML Engineer LinkedIn 

@ericabrescia 

- Former COO Github, Founder Bitnami (acq’d by VMWare) 

@jordan_segall 

- Partner at Redpoint

Jacob is an AI investor at Redpoint Ventures. He's led Redpoint's investments in companies like Abridge, Physical Intelligence & Legora. Follow Jacob on Twitter (@jacobeffron).

On Unsupervised Learning we probe the sharpest minds in AI in search for the truth about what's real today, what will be real in the future and what it all means for businesses and the world. If you're a builder, researcher or investor navigating the AI world, this podcast will help you deconstruct and understand the most important breakthroughs and see a clearer picture of reality. Subscribe to this show to stay up to date on our latest episodes.

Sarah Buchner is the Founder and CEO of Trunk Tools (https://trunktools.com/). She is a one-of-a-kind founder, having spent her young life as a carpenter in Austria and then working her way up the ranks of the construction industry. She’s also earned several graduate degrees including an MS in Civil Eng, a PHD in Data Science and an MBA from Stanford.

Trunk Tools is an AI Tool for the $13T construction industry aimed at enhancing project management and addressing the skilled Labor shortage.

We wanted to have Sarah on to share her experience in building a vertical AI tool - specifically: where AI will have the biggest impact on construction and behind the scenes of how they’ve built tools like TrunkText and TrunkScheduler to massively reduce the amount of rework in construction.

With your co-hosts: 

@jacobeffron 

- Partner at Redpoint, Former PM Flatiron Health 

@patrickachase 

- Partner at Redpoint, Former ML Engineer LinkedIn 

@ericabrescia 

- Former COO Github, Founder Bitnami (acq’d by VMWare) 

@jordan_segall 

- Partner at Redpoint

Jacob is an AI investor at Redpoint Ventures. He's led Redpoint's investments in companies like Abridge, Physical Intelligence & Legora. Follow Jacob on Twitter (@jacobeffron).

On Unsupervised Learning we probe the sharpest minds in AI in search for the truth about what's real today, what will be real in the future and what it all means for businesses and the world. If you're a builder, researcher or investor navigating the AI world, this podcast will help you deconstruct and understand the most important breakthroughs and see a clearer picture of reality. Subscribe to this show to stay up to date on our latest episodes.