Jean-Francois Puget is a Director and Distinguished Engineer at NVIDIA, where he leads the Kaggle Grandmasters team, and he's ranked third on Kaggle's all-time list. We caught him on the day NVIDIA announced Nemotron Ultra and its new agent skills repo. We talk about what skills actually are, why they beat MCP tools on context cost, and how NVIDIA built an evaluation pipeline to separate skills that help from skills that don't.

From there we talk about the thing JFP cares about most: evaluation. He explains why most LLM benchmarks reward overfitting, how his team discovered O3 could pick the right files to fix SWE-bench issues without reading them, and why the only benchmarks he trusts are the ones where you commit before you see the score, which is exactly how Kaggle works. He predicts a "bloodbath" for the wave of competitors letting coding agents chase leaderboard scores with no notion of validation.

We also get into what coding agents are actually good for ("a mix of a genius and a dumb person"), the multi-agent system at NVIDIA that built a working PyTorch clone that runs 10x slower than the real thing, his unfiltered take on frontier lab PR and the Mythos release, whether AI is a bubble, and the story of how his team won ARC-AGI with a 4-billion-parameter model at 20 cents a task, including jumping from third to first in the final hours of a seven-month competition.

Timeline

• 00:00 — Intro
• 01:05 — NVIDIA's announcements: Nemotron Ultra and the agent skills repo
• 07:21 — Skills vs MCP tools, and progressive disclosure
• 10:24 — Agents that write their own skills: a new form of learning
• 13:33 — When overfitting is fine (and when it isn't)
• 15:47 — Why most LLM benchmarks reward overfitting
• 17:06 — The SWE-bench contamination story: O3 picks files without reading them
• 19:45 — How LLMs changed Kaggle, and the coming "bloodbath"
• 25:40 — What makes a good data scientist: evaluation and one-bit experiments
• 28:56 — Running Codex at scale: the top token consumers at NVIDIA
• 29:37 — Did coding agents kill AutoML?
• 30:16 — Genius and dumb at once: the limits of coding agents
• 35:21 — Humans in the loop, sandboxing, and the teenage hacker who never wrote code
• 37:42 — Mythos, frontier lab PR, and open source
• 40:08 — Why NVIDIA builds open models, and where it's already frontier
• 43:48 — World models, robots, and the coffee test
• 49:20 — Why agents still can't play Dota
• 50:24 — Is AI a bubble?
• 53:14 — Winning ARC-AGI with a 4B model at 20 cents a task
• 57:39 — Kaggle is a legal drug

  Music:

  • "Kid Kodi" - Blue Dot Sessions - via Free Music Archive - CC BY-NC 4.0.

  About: The Information Bottleneck is hosted by Ravid Shwartz-Ziv and Allen Roush, featuring in-depth conversations with leading AI researchers about the ideas shaping the future of machine learning.

In this episode, we talked with Dimitris Papailiopoulos, researcher at Microsoft Research's AI Frontiers lab and professor at the University of Wisconsin, about doing research in the age of agents. Dimitris told us about the Sunday morning that changed how he works: he handed Claude Code and Codex a question he'd been sitting on for years, went about his day, and came back to an answer. After a few days of dread about what's left for humans, he landed somewhere more optimistic, calling this the golden age of asking questions.

We talked about his "smallest transformer that can add" leaderboard, a symbolic GSM8K solver built from if-else statements, and what happened when he put two Claude Code instances in the same file system and told them to do something cool (one pair invented a communication protocol, the other played Battleship). We also got into diversity and slop in agent-generated ideas, why agents get stubborn after a million tokens, harness overfitting on Terminal-Bench, continual learning and world models, whether agents need vision, and where information theory actually helps in AI and where it's a katana used to make coffee.

Timeline

00:00 Intro
01:45 How agents changed the way Dimitris does research
04:30 A Sunday morning with Claude Code, Codex, and GSM8K
07:15 The dread, then the golden age of asking questions
08:20 Taste and verification, and how we train students now
09:53 Will models make human verification obsolete?
11:30 The smallest transformer that can add 10-digit numbers
13:40 Humans as initializers for gradient descent in idea space
15:32 Allen on diversity, slop profiles, and high temperature research
21:44 When Claudes meet: Battleship, invented protocols, and a grokking paper
25:53 Single agent vs multi-agent under fixed compute
30:28 Auto-research benchmarks and what agents actually accelerate
35:14 Inside the symbolic GSM8K solver (with a live progress check)
40:04 Idea overfitting and why agents refuse to change course
44:00 Learning from failure traces and harness overfitting
48:04 Continual learning, memory files, and world models
51:30 Why don't labs personalize models on your own history?
57:52 Agent-to-agent communication: is Jira the right tool?
1:01:25 Multimodality: vision as a tool vs one unified model
1:05:40 Information theory and AI, or making coffee with a katana
1:11:23 Closing thoughts: ask bigger questions

Music:

• "Kid Kodi" - Blue Dot Sessions - via Free Music Archive - CC BY-NC 4.0.

About: The Information Bottleneck is hosted by Ravid Shwartz-Ziv and Allen Roush, featuring in-depth conversations with leading AI researchers about the ideas shaping the future of machine learning.

Phillip Isola, professor at MIT, joins us to talk about representation learning: what makes a representation good, why different models seem to converge on similar representations, and whether pre-training is really over.

We discuss the platonic representation hypothesis and its limits, why clustering structure matters more than global geometry, and Phillip's new neural thickets paper arguing that post-training is easier than people think because pre-trained weights already sit near solutions to downstream tasks. Phillip also explains why he thinks LLMs are already world models, why he's betting on RNNs making a comeback, and why his most exciting current direction is artificial life: putting LLM agents in open environments with no fixed task and studying them like new organisms.

Timeline:

00:00 Intro song
00:13 Intro
01:05 What is representation learning and why it matters
04:09 What makes a representation good: minimality and sufficiency
10:03 How cross entropy and contrastive learning shape representations
14:35 Dimensionality reduction and why dimension isn't the right complexity measure
16:35 Compression and geometric clustering during training
19:27 The platonic representation hypothesis and what actually converges
22:53 Local neighborhoods vs global structure: the Aristotelian follow-up
24:33 When convergence is strong: truth vs the space of possibility
28:09 Is there true similarity in the world? The Bouba-Kiki effect
30:56 World models vs autoregressive LLMs
32:14 Diffusion LLMs as a special case of autoregressive models
33:42 What architectures win in five years: the case for RNNs
36:11 Grad student descent, or do we actually have principles?
40:51 Feathers and wings: what to take from biology
43:17 How close are we to brain-like models? Marr's three levels
47:01 Are better models becoming less human-like?
49:38 Is pre-training all you need? The neural thickets paper
54:18 LoRA, low rank fine-tuning, and why post-training is easier than we thought
56:01 RL environments and what our benchmarks actually test
1:01:11 Artificial life: LLM agents as new organisms
1:07:20 What's overlooked in AI research right now
1:08:36 Why stay in academia, and doing science in the age of Opus

Music:

• "Kid Kodi" - Blue Dot Sessions - via Free Music Archive - CC BY-NC 4.0.

About: The Information Bottleneck is hosted by Ravid Shwartz-Ziv and Allen Roush, featuring in-depth conversations with leading AI researchers about the ideas shaping the future of machine learning.

Most AI-for-science companies are selling shovels. Qichao Hu wants the gold.

In this episode, we talk with Qichao, the founder and CEO of Molecular Universe, the AI-for-science platform that grew out of SES AI, a high-energy-density battery developer he's run for fourteen years. His core distinction is that companies from the AI world build tools, such as foundation models that predict properties, while companies from the science world care about the final product, such as the new battery or material that actually ships. Molecular Universe sits firmly on the science side, and the difference shows up everywhere from what they publish to what they refuse to.

We get into the actual workflow of materials discovery and where AI compresses it. A single trial in a traditional lab can take a year with maybe a 40% success rate; the goal is to run a thousand candidates in parallel and turn that year into a week. Qichao walks through improving low-temperature fast-charging for EV batteries:  from hypothesis generation through molecule-, material-, and device-level property prediction, down to autonomous labs that synthesize and test the top candidates without a human touching a pipette.

The hardest problem, it turns out, isn't predicting molecular properties or measuring device performance, but it's the black box connecting the two. In batteries, that's the solid-electrolyte interface, which the field has been hand-waving about since the seventies. And the thing standing in the way of cracking it isn't a clever training trick but data: companies sitting on twenty years of records are finding it too messy, incomplete, and poorly labeled to train on, and are having to start collecting from scratch with new protocols and robots.

Timeline

• 00:13 — Intro and welcome;
• 01:19 — Shovel vs. gold
• 05:18 — Why the world's smartest scientist doesn't automatically give you a better battery
• 07:25 — The discovery workflow
• 09:37 — Exploration vs. exploitation
• 11:54 — Safety and filtering: screening novel molecules against banned and toxic-substance lists
• 17:55 — How hypotheses get generated, and where frontier LLMs help
• 20:29 — From hypothesis to ~400 formulations: property prediction, ranking, and handing off to autonomous labs
• 26:37 — "A foundation model for everything" — and the black box between molecular properties and device performance
• 30:01 — World models and physics
• 33:09 — The great unknown in batteries
• 37:08 — Simulation vs. reality: calibrating massive simulated datasets with a sliver of experimental data
• 41:47 — Lab robotics: how fast the hardware has caught up, and what a floor of autonomous labs looks like
• 43:50 — The real bottlenecks
• 50:21 — Pre-training from scratch vs. post-training LLMs, and why training tricks haven't reduced the need for good data
• 52:42 — Evaluation
• 55:42 — Publish the B+ model, keep the A model
• 58:05 — Five years out
• 1:00:37 — Closing thoughts and wrap

Music:

• "Kid Kodi" - Blue Dot Sessions - via Free Music Archive - CC BY-NC 4.0.

About: The Information Bottleneck is hosted by Ravid Shwartz-Ziv and Allen Roush, featuring in-depth conversations with leading AI researchers about the ideas shaping the future of machine learning.

We talk a lot on this show about RL, agents, and the move between pre-training and post-training, but not enough about the layer everything actually runs on. Benny Chen, co-founder of Fireworks AI, one of the largest inference platforms around, walks us through what it takes to serve models at scale: sourcing GPUs, writing the kernels, the runtime, and the routing layer that lets a customer hit one endpoint and forget the rest.

We talk why the real bottleneck is power, not chips, and why that favors Nvidia and Google. Why MoE keeps winning even when dense models look better on paper and why he'd rather run fungible capacity at 95% than specialized chips at 60%. We also talk about quantization limits, where RL efficiency has to go next, and his case that AI is still under-hyped. We also get into cross-region training, sparse autoencoders and why interpretability hasn't taken off in open source, whether open models can close the gap, and a frank read on Anthropic's go-to-market.

Timeline

• 00:00 — Intro: the part of AI nobody talks about
• 01:20 — What "infrastructure for AI" actually means: the layers, from GPUs up to routing
• 02:59 — Why not just buy your own GPUs and do it yourself?
• 05:17 — The scale Fireworks runs at
• 06:35 — Hardware inflation, GPU costs, and the real risk hiding in commit duration
• 10:14 — Nvidia vs AMD vs TPUs, and why power is the bottleneck
• 11:57 — Mixing GPU types and generations; fungibility vs. specialization
• 14:22 — Once you have the GPUs, what's the next layer to build?
• 17:04 — Dense vs. MoE, and why the hardware picks the winner
• 21:07 — Quantization: is FP4 the floor? TurboQuant and INT vs. FP
• 24:28 — How tied are the algorithms to the hardware?
• 25:12 — DeepSeek, DeepGEMM, and next-token prediction as reconstruction loss
• 28:50 — Why RL is still wildly inefficient compared to pre-training
• 30:08 — Speculative decoding, AI-generated kernels, and auto-research
• 34:00 — The AGI question: why text gets automated but vision may stay expensive
• 37:07 — Hype check: why Benny thinks AI is still under-hyped
• 41:28 — Training vs. inference at the infrastructure level
• 44:12 — Scaling across data centers: cross-region training with Cursor
• 45:40 — Sparse autoencoders, interpretability, and why open source is human-constrained
• 49:04 — Will open models catch up — on quality and on compute?
• 51:41 — Are we plateauing? Opus 4.7 vs. 4.6 and the coming data wars
• 54:41 — Physical limits, HBM, and whether chips keep getting faster
• 58:17 — The belief about inference everyone gets wrong
• 59:31 — Anthropic, mythos, and a frank take on go-to-market
• 1:04:41 — Wrap-up

Music:

• "Kid Kodi" - Blue Dot Sessions - via Free Music Archive - CC BY-NC 4.0.

About: The Information Bottleneck is hosted by Ravid Shwartz-Ziv and Allen Roush, featuring in-depth conversations with leading AI researchers about the ideas shaping the future of machine learning.

Oded Rechavi is a biologist at Tel Aviv University and the co-founder of QED, a company building AI to review scientific work. He's also spent years studying worms.

We start with what's wrong with peer review and grant funding: why it takes years to publish, why reviewers are often your own competitors, and why the whole thing is locked to an economic model that rewards publishing more papers, not better ones. Oded explains why he doesn't call QED "peer review" at all, and what it would take to actually validate science instead of just stamping it.

Then we get into the biology. C. elegans has exactly 959 cells, every one of them named, and a fully mapped brain. Oded's lab studies how a worm's experiences get passed to its offspring through RNA rather than DNA — meaning what happens to a worm in its lifetime can change its descendants. We also talk about using ancient DNA to reassemble the Dead Sea Scrolls, what AI can and can't do for biology, and why he wants to build an "Ironman suit" for researchers rather than replace them.

00:00 Intro

01:35 Why scientific publishing is broken

04:02 Years to publish, and what it costs science

07:20 Bad reviewers, conflicts of interest, and the money

10:47 Why preprints don't fix it

15:37 How AI conferences handle review

22:07 Conferences vs. journals — does slow review help?

25:22 Building QED: review, not peer review

30:02 Tracking a paper from idea to submission

33:11 What writing a grant actually involves

35:00 The ERC reviewer crisis

37:06 Tailoring feedback to your field

41:48 Switching to biology

44:30 Every cell has a name: inside C. elegans

46:28 Inheritance without DNA

48:16 What the worm "thinks" changes its offspring

51:58 Reassembling the Dead Sea Scrolls with ancient DNA

56:07 Psychedelics and worms

58:36 Can AI run the research itself?

1:04:49 Automation vs. validation

1:07:12 The origin of life

1:08:49 Why people reject AI-written work

1:16:18 Will humans still have a role?

1:17:39 Wrap-up

Music:

• "Kid Kodi" - Blue Dot Sessions - via Free Music Archive - CC BY-NC 4.0.

About: The Information Bottleneck is hosted by Ravid Shwartz-Ziv and Allen Roush, featuring in-depth conversations with leading AI researchers about the ideas shaping the future of machine learning.

Will AI take our jobs? We put the question to Alex Imas, the new Director of AGI Economics at Google DeepMind and a professor at Chicago Booth, whose entire job now is studying how frontier AI reshapes the economy. His short answer: probably some of them, but the popular story is mostly wrong about which jobs and how fast.

Alex makes the case that a job is a bundle of tasks, not a single thing AI either does or doesn't do, and that the number of people who should actually care about is how much consumer demand responds to falling prices. Get that wrong and you predict mass layoffs. Get it right and you sometimes predict more hiring. We get into why the automation panic is two centuries old, why he thinks blue-collar work is in more danger than white-collar, and why the people already winning are the ones adopting AI fastest.

We also cover the AGI versus ASI distinction and why it changes everything for the economy, what happens when there's no moat and open models stay six to eight months behind, the three-tier pricing future he sees coming after the 2026 compute crunch, and what any of this means if you're deciding whether to send your kids to college.

• The episode was recorded before Alex joined Google

Timestamps

00:00 Meeting Alex Imas

00:44 Will AI take our jobs?

03:35 Is this an AI question or an economics question?

06:18 The economy is already behind the AI we have

07:43 Why AI adoption is K-shaped

12:51 Was Andrew Yang right?

13:45 The automation panic is 200 years old

16:46 Dario's six-month claim, and why we don't see it yet

17:22 A job is not a task

22:38 The three numbers that actually predict the labor market

22:42 The chess engine analogy and the centaur phase

25:45 Recursive self-improvement and the hamburger problem

30:06 Should AI labs be the ones answering alignment questions?

31:17 The "invisible hand wave" and why nobody wants fully autonomous AI

33:27 AGI vs ASI, and why the difference is everything

35:28 Commodities vs relational goods

41:14 Star Trek, replicators, and predicting with sci-fi

45:20 Inequality and the Upper West Side VCs

46:21 Your money manager was automated in the 1960s

50:47 Are OpenAI and Anthropic overvalued? The moat problem

54:29 What has to be true for the losses to make sense

55:43 Cognitive atrophy and monopoly fears

57:00 The 2026 compute crunch and the three-tier pricing future

1:01:52 The Apple vs Android analogy

1:03:54 A rich-country perspective

1:04:16 Protecting the skills that actually matter

1:07:02 Will not using AI become a status symbol?

1:08:53 Does capitalism even survive?

1:13:44 Redistribution becomes the political battleground

1:18:16 Blue collar vs white collar: who's really at risk

1:21:18 Advice for parents in an AI world

1:22:43 Saving for retirement when the Valley says don't

1:25:06 Will non-elite colleges survive?

Music:

• "Kid Kodi" - Blue Dot Sessions - via Free Music Archive - CC BY-NC 4.0.

About: The Information Bottleneck is hosted by Ravid Shwartz-Ziv and Allen Roush, featuring in-depth conversations with leading AI researchers about the ideas shaping the future of machine learning.

In this episode, we sit down with Wenhu Chen, research scientist at Meta MSL, assistant professor at the University of Waterloo, and the person behind MMLU-Pro and MMMU. If you've read a frontier model release in the last two years, you've seen his benchmarks. That makes him one of the best people to answer the question everyone dances around: when a model jumps from 40% to 90% on your benchmark, how much of that is real? In this episode, we dig into why benchmarks have become the loss function of the entire field - design a bad one, and thousands of brilliant researchers will spend months hill-climbing in the wrong direction. Wenhu is surprisingly candid about the limits of his own creations: contamination is everywhere, saturation turns frontier benchmarks into unit tests, and popular alternatives, such as LM Arena, mostly measure tone and length rather than capability. His answer is to evaluate models where they've never been: private codebases, hospital data, and the messy, live internet.

We also talk about ClawBench, his new benchmark that deploys agents to over 140 real production websites to do things people actually want done, such, such as ordering food, booking tickets, and applying for jobs. The best model in the world completes about a third of these tasks. We unpack why: bot detection, models that refuse to click "pay," agents that give up the moment an environment doesn't match their training, and harnesses that can swing results by 20% without changing the model at all.

Along the way, we cover the overlooked science of evaluating pre-training, data flywheels, and synthetic environments for agent training, and whether RL teaches models to reason or just surfaces what's already there. We close with Wenhu's predictions: exploration and adaptability will improve rapidly, but security will become the field's hardest problem as agents gain real permissions in the real world.

Timestamps

00:00 – Intro
00:55 – What good evaluation means, and how it's changed since the early GPT days
03:35 – Benchmarks as the field's loss function
05:50 – Contamination: the problem nobody fully solves
08:08 – MMLU-Pro scores: real progress or training on the test set?
11:05 – Can you measure creativity?
12:34 – Why human judges and arenas are unreliable — and what to use instead
19:22 – What a good benchmark actually looks like
22:34 – Chain of thought: signal or scratchpad?
26:01 – Auto-research and hill-climbing agents
28:52 – Harnesses: 20% swings without touching the model
32:28 – Safety, model release, and an "FDA for models"
36:53 – The overlooked science of pre-training evaluation
43:49 – Designing pre-training benchmarks when one run costs a billion dollars
49:45 – ClawBench: agents on 140+ live websites, and why the best model gets 33%
54:42 – How MMLU-Pro and MMMU-Pro were born from public complaints
59:16 – Pixel agents vs. APIs: will MCP kill computer use?
1:02:11 – Training agents: data flywheels and synthetic environments
1:05:43 – SFT vs. RL, and does RL teach reasoning or reveal it?
1:09:21 – What gets solved next year — and what doesn't
1:14:32 – Undervalued ideas, and what's next for ClawBench

Music:

• "Kid Kodi" - Blue Dot Sessions - via Free Music Archive - CC BY-NC 4.0.

About: The Information Bottleneck is hosted by Ravid Shwartz-Ziv and Allen Roush, featuring in-depth conversations with leading AI researchers about the ideas shaping the future of machine learning.

In this episode, we sit down with Wenhu Chen, research scientist at Meta MSL, assistant professor at the University of Waterloo, and the person behind MMLU-Pro and MMMU. If you've read a frontier model release in the last two years, you've seen his benchmarks. That makes him one of the best people to answer the question everyone dances around: when a model jumps from 40% to 90% on your benchmark, how much of that is real? In this episode, we dig into why benchmarks have become the loss function of the entire field - design a bad one, and thousands of brilliant researchers will spend months hill-climbing in the wrong direction. Wenhu is surprisingly candid about the limits of his own creations: contamination is everywhere, saturation turns frontier benchmarks into unit tests, and popular alternatives, such as LM Arena, mostly measure tone and length rather than capability. His answer is to evaluate models where they've never been: private codebases, hospital data, and the messy, live internet.

We also talk about ClawBench, his new benchmark that deploys agents to over 140 real production websites to do things people actually want done, such, such as ordering food, booking tickets, and applying for jobs. The best model in the world completes about a third of these tasks. We unpack why: bot detection, models that refuse to click "pay," agents that give up the moment an environment doesn't match their training, and harnesses that can swing results by 20% without changing the model at all.

Along the way, we cover the overlooked science of evaluating pre-training, data flywheels, and synthetic environments for agent training, and whether RL teaches models to reason or just surfaces what's already there. We close with Wenhu's predictions: exploration and adaptability will improve rapidly, but security will become the field's hardest problem as agents gain real permissions in the real world.

Timestamps

00:00 – Intro
00:55 – What good evaluation means, and how it's changed since the early GPT days
03:35 – Benchmarks as the field's loss function
05:50 – Contamination: the problem nobody fully solves
08:08 – MMLU-Pro scores: real progress or training on the test set?
11:05 – Can you measure creativity?
12:34 – Why human judges and arenas are unreliable — and what to use instead
19:22 – What a good benchmark actually looks like
22:34 – Chain of thought: signal or scratchpad?
26:01 – Auto-research and hill-climbing agents
28:52 – Harnesses: 20% swings without touching the model
32:28 – Safety, model release, and an "FDA for models"
36:53 – The overlooked science of pre-training evaluation
43:49 – Designing pre-training benchmarks when one run costs a billion dollars
49:45 – ClawBench: agents on 140+ live websites, and why the best model gets 33%
54:42 – How MMLU-Pro and MMMU-Pro were born from public complaints
59:16 – Pixel agents vs. APIs: will MCP kill computer use?
1:02:11 – Training agents: data flywheels and synthetic environments
1:05:43 – SFT vs. RL, and does RL teach reasoning or reveal it?
1:09:21 – What gets solved next year — and what doesn't
1:14:32 – Undervalued ideas, and what's next for ClawBench

Music:

• "Kid Kodi" - Blue Dot Sessions - via Free Music Archive - CC BY-NC 4.0.

About: The Information Bottleneck is hosted by Ravid Shwartz-Ziv and Allen Roush, featuring in-depth conversations with leading AI researchers about the ideas shaping the future of machine learning.

In the second half of our conversation with Jürgen Schmidhuber, we focus on the key ideas he's pursued since the early 1990s and discuss why he believes these concepts are only now being rediscovered.

We start with JEPA. Jürgen argues that the method LeCun named in 2022 is the same family he published in 1992 as Predictability Maximization. From there he traces the adversarial lineage back further still, to his 1990 world-model paper and 1991 Predictability Minimization  -  the curiosity-driven minimax games he sees as the real origins of GANs.

We also talk about why these ideas took thirty years to land, why today's trillion-dollar data-center buildout is driven by AGI fear, and why he thinks Apple may come out ahead.

The back half turns to what he sees as the real frontier: physical AI. Today's systems are superhuman behind the screen but helpless at a leaky pipe, and until a robot can use human tools, there's no AGI. He discusses self-replicating, self-improving machines as "a new kind of life," reframes continual learning and test-time training as ideas from his 1991 fast-weight work, and detours through Solomonoff's universal prior, Hutter's AIXI, and the Gödel machine.

We close on the subject Jürgen is famous for: scientific credit. He makes his case for rigorous attribution, casts himself as a "speaker for the dead" championing forgotten pioneers like Ivakhnenko, and reflects candidly on whether the fights are personal.

Timeline

00:30 — What JEPA is, and the 1992 Predictability Maximization story

04:54 — Implementing PMAX: autoencoders, Siamese networks, Infomax

09:10 — Predictability Minimization, factorial codes, and the roots of GANs

16:00 — Why it took 30 years: the economics of compute

20:52 — Data, the web, and 1990 as the origin point

23:09 — Hardware inflation, the trillion-dollar buildout, and the coming crash

34:05 — Physical AI: the plumber problem and self-replicating machines

41:14 — Which 90s ideas are being scaled right now

45:26 — Continual learning and test-time training as "old hats"

55:19 — Measuring intelligence: Solomonoff, AIXI, and the Gödel machine

1:05:26 — Self-replication and von Neumann

1:09:51 — Will he see AGI in his lifetime?

1:10:42 — Credit, integrity, and being a "speaker for the dead"

Music:

• "Kid Kodi" - Blue Dot Sessions - via Free Music Archive - CC BY-NC 4.0.
• "Palms Down" - Blue Dot Sessions - via Free Music Archive - CC BY-NC 4.0.
• Changes: trimmed

About: The Information Bottleneck is hosted by Ravid Shwartz-Ziv and Allen Roush, featuring in-depth conversations with leading AI researchers about the ideas shaping the future of machine learning.