Svicpodcast.com: For Questions, Book recs, Videos, Newsletters, etc.

patreon.com/svic

Follow us:

Threads: www.threads.net/@svicpodcast

Twitter: https://x.com/svicpodcast

LinkedIn: https://www.linkedin.com/company/svicpodcast

Tiktok: https://www.tiktok.com/@svicpodcast

Instagram: https://www.instagram.com/svicpodcast

Spotify: https://podcasters.spotify.com/pod/show/svicpodcast

Show hosts Joe (Eng VP) and Jordan (M&A Deal Lead) worked at various companies from Google, Apple, Facebook, Microsoft, Salesforce, Slack, Carta, Splunk, Wealthfront, Adobe, and more.

#ai #openai

Chapters

00:00 -Yolo Training Run

00:51 - OpenAI’s Model Scaling Process Explained

03:17 - Mixture of Experts and AI Efficiency Gains

05:34 - DeepSeek’s Hardware Innovations and GPU Optimization

07:50 - The Role of Low-Level GPU Engineering in AI Advancements

09:23 - AI Cost Performance Trends and Expectations

11:10 - The Shift from Open Research to Closed AI Development

12:47 - Meta's Open-Source AI Strategy vs. OpenAI’s Approach

14:45 - OpenAI’s Incremental Releases and Competitive Strategy

17:12 - DeepSeek Disrupting OpenAI’s Market Dominance

19:51 - Meta’s AI Strategy and the Open-Source Advantage

22:33 - The Business Model of AI Labs and Open-Source Challenges

24:59 - The Shift to Self-Supervised Learning and Human Preference Data

28:59 - Synthetic Data and the Evolution of AI Training

31:45 - Search vs. Reward Models in AI Reasoning

34:12 - The Bitter Lesson and How AI Models Improve Over Time

39:10 - Loss Functions, YOLO Runs, and Training Failures

46:17 - DeepSeek’s Leadership and China’s AI Push

48:50 - AI as a National Security Issue and Global Competition

51:47 - AI Safety Concerns and the Impact on Model Development

56:28 - AI Regulation, Conspiracies, and Industry Dynamics

57:46 - Closing Thoughts and Upcoming Episodes