(0:00) Introduction — Marc Jacobstein opens the episode and introduces Edward Hu, world-renowned AI researcher (LoRA, μTransfer) and now entrepreneur
(2:05) LoRA origins — Edward describes the practical need that led to the creation of LoRA at Microsoft in 2021 while collaborating with OpenAI
(6:50) Why LoRA worked — Low-rank updates to large models made fine-tuning efficient and widely accessible
(9:45) Biological analogy — LoRA as the equivalent of learning a new instrument with shared prior knowledge
(11:35) Industry adoption — Edward reflects on companies thanking him for saving compute costs and resource usage
(13:30) Intelligence as compression — Discussion of whether brains and AI share a compression-based structure for learning
(17:15) Risks of over-biological inspiration — Why not everything in AI needs to imitate the brain
(18:45) MuTransfer explained — Training stability via infinite-width theoretical insights and their practical payoff
(23:00) Real-world impact of MuTransfer — Especially valuable for massive models at top labs like OpenAI, Anthropic
(26:30) The joy of theory with impact — The rare but powerful moment when theory turns into applied breakthrough
(28:05) Training trillion-parameter models — How MuTransfer enables precise hyperparameter prediction at scale
(30:30) Edward’s path — From Microsoft to PhD with Yoshua Bengio to OpenAI and now entrepreneurship
(34:00) Why leave OpenAI? — Personal itch for autonomy and new challenges, despite leaving pre-vesting
(37:20) From researcher to founder — Edward’s shift to launching multiple ventures with strong partners
(41:00) “Tech alone isn’t enough” — Why proprietary data + distribution is the real edge for AI startups
(44:00) Vertical vs horizontal plays — Studio model allows broad application of AI in many industries
(46:40) LoRA-as-a-metaphor — Marc proposes a meta analogy: Edward as the foundation, fine-tuned by domain experts
(49:25) The fall of engineering costs — Why software is easier to build than ever, and what that unlocks
(52:00) The new venture model — VCs must adapt to a world with lower costs and smaller teams
(55:45) Edward’s self-awareness — On knowing his strengths and staying in his zone of genius
(57:10) The Compute Exchange — Edward unveils a new project: a dynamic GPU auction platform for buyers and sellers
(1:00:20) Compute is the new oil — Making GPU access as transparent and efficient as financial markets
(1:02:10) On IP and creator rights — Edward’s take on respecting creativity, compensating artists, and not undermining the creative industry
(1:06:00) The soul in human creation — Machines may be good imitators, but creativity may still be uniquely human
(1:08:00) Recommended Reading — Edward recommends The Creative Act: A Way of Being by Rick Rubin as a meditation on creativity across disciplines
(1:10:15) Outro — Marc reflects on Edward’s polymath career arc and cross-industry potential
