Is deep learning hitting a wall? It’s 2025 and Mixture of Experts is back and better than ever. In episode 36, host Tim Hwang is joined by Chris Hay, Kate Soule and Kush Varshney to debrief one of the biggest releases of 2024, OpenAI o3. Next, DeepSeek-V3 is here! Finally, will AI exist in 2027? The experts dissect the AI bet between Miles Brundage and Gary Marcus. All that and more on the first Mixture of Experts of 2025.

The opinions expressed in this podcast are solely those of the participants and do not necessarily reflect the views of IBM or any other organization or entity.

00:00 — Intro

00:49 — OpenAI o3

14:40 — DeepSeek-V3

28:00 — The Brundage/Marcus bet

Will 2025 be the year of AI agents? In Episode 35 of Mixture of Experts, host Tim Hwang is joined by some show veterans to debrief 2024 in AI. This week, we review AI models, agents, hardware and product releases with some of the top industry experts. What was the best model of 2024? Is NVIDIA king? What are some of the AI trends in 2025? All that and more on this special edition of Mixture of Experts.

The opinions expressed in this podcast are solely those of the participants and do not necessarily reflect the views of IBM or any other organization or entity.

Is pre-training a thing of the past? In Episode 34 of Mixture of Experts, host Tim Hwang is joined by Abraham Daniels, Vagner Santana and Volkmar Uhlig to debrief this week in AI. First, OpenAI cofounder Ilya Sutskever said that “peak data” was achieved, does this mean there is no longer a need to model pre-training? Next, IBM released Granite 3.1 with a slew of features, we cover them all. Then, there is a new way to steal AI models, how do we protect against model exfiltration. Finally, can NVIDIA Jetson for AI developers really increase hardware accessibility? Tune-in for more!

The opinions expressed in this podcast are solely those of the participants and do not necessarily reflect the views of IBM or any other organization or entity.

00:01 — Intro

00:49— Is pre-training over?

10:25 — Granite 3.1

22:23 — AI model stealing

33:38—NVIDIA Jetson

Is o1 Pro worth the cost? In Episode 33 of Mixture of Experts, host Tim Hwang is joined by Marina Danilevsky, Kate Soule and Vyoma Gajjar. First, the experts debrief the 12 Days of OpenAI. Next, we review some of the top papers in NeurIPS, how are the experts keeping up with all these research papers? Then, we are back with another benchmark, can ARC Prize make AGI more tractable? Finally, Meta announced the launch of Llama 3.3 70B with the promise of 405B performance, can we have our cake and eat it too? Find out more on today’s Mixture of Experts!

The opinions expressed in this podcast are solely those of the participants and do not necessarily reflect the views of IBM or any other organization or entity.

What’s the mystery behind the name ChatGPT refuses to discuss? In Episode 32 of Mixture of Experts host Tim Hwang dives into the hottest topics shaping the AI landscape with an all-star panel: Aaron Baughman, Vagner Figueredo de Santana, and Shobhit Varshney. First, they disect the biggest announcements and takeaways from AWS re:Invent 2024, Amazon’s premier AI event. Next, they talk about overcoming architectural vulnerabilities in AI systems, and finally, they uncover the curious case of a name ChatGPT won’t discuss—and the questions this raises about privacy and transparency in AI. Get ready for an episode packed with insights, debates, and forward-thinking perspectives!

The opinions expressed in this podcast are solely those of the participants and do not necessarily reflect the views of IBM or any other organization or entity.

How much future learning will be done with an AI assistant? In Episode 31 of Mixture of Experts, host Tim Hwang is joined by Phaedra Boinodiris, Marina Danilevsky and Skyler Speakman for the AI in education special episode. First, the experts give an update on the state of AI within education. Next, we cover concerns around AI safety and literacy, what do students and teachers need to be aware of? Finally, the panel gives their predictions on what the future of education holds as it relates to AI. Tune-in to this special episode for an in-depth analysis!

The opinions expressed in this podcast are solely those of the participants and do not necessarily reflect the views of IBM or any other organization or entity.

Should your AI assistant remember everything about you? In Episode 30 of Mixture of Experts, host Tim Hwang is joined by Vagner Santana, Vyoma Gajjar and Shobhit Varshney. First, the experts breakdown claims of “near-infinite memory” within AI models. Next, Shobhit is fresh off the plane from Microsoft Ignite, he shares some of the exciting new announcements following the event. Then, a new benchmark has entered the chat, what do we know about FrontierMath? Finally, AlphaFold3 is now more open, why does this matter? Find out more on today’s episode!

The opinions expressed in this podcast are solely those of the participants and do not necessarily reflect the views of IBM or any other organization or entity.

Is 2024 the year scaling AI officially breaks? In Episode 29 of Mixture of Experts, host Tim Hwang is joined by Anthony Annunziata, Kate Soule and Naveen Rao. First, the experts discuss whether we are living in a post scale world. Next, we can’t have an episode without chatting AI agents, but what does the future hold for this technology? Finally, is AGI here to stay? Tune-in to this week’s Mixture of Experts to find out.

The opinions expressed in this podcast are solely those of the participants and do not necessarily reflect the views of IBM or any other organization or entity.

Could AI wipe out software engineers? In Episode 28 of Mixture of Experts, host Tim Hwang is joined by Chris Hay, Kaoutar El Maghraoui, and Shobhit Varshney. First, the experts discuss GitHub reporting a rise of developers driven by AI code assistant tools. Next, Big Sleep finds a vulnerability in SQLite, what is the future for these kinds of AI agents? Finally, OpenAI released SearchGPT, what is the future of AI search? Tune-in today to find out!

The opinions expressed in this podcast are solely those of the participants and do not necessarily reflect the views of IBM or any other organization or entity.

Agents, agents, and more agents! In Episode 27 of Mixture of Experts, host Tim Hwang is joined by Volkmar Uhlig and Vyoma Gajjar. First, the experts chat about Mark Benioff’s spicy tweet, and what this means for the future of AI agents. Next, how much energy is needed to power AI models, and should we be concerned? Then, the experts debrief Anthropic’s release of computer use. Finally, Google is integrating SynthID-Text into Gemini to help watermark AI-generated text, do we need this feature? Learn more on today’s Mixture of Experts.

The opinions expressed in this podcast are solely those of the participants and do not necessarily reflect the views of IBM or any other organization or entity.