Chain of Thought | AI Agents, Infrastructure & Engineering

How Superhuman Built AI Into a 100ms Product | Loïc Houssier


Listen Later

Loïc Houssier leads engineering at Superhuman, the email client Grammarly acquired for ~$825 million in July 2025. Before Superhuman he was CTO of OpenTrust (acquired by DocuSign), ran engineering at ProductBoard, and started his career in applied cryptography for France's defense industry, including work on nuclear submarine systems. Loïc joined Superhuman in early 2024 and within 30 days was leading a six-week sprint to ship AI Inbox.

Superhuman's brand is built on speed: every interaction under 100 milliseconds. LLMs do not run in 100 milliseconds. So Loïc walks Conor through how his team retrofitted AI into a product that was already winning without it: pre-caching context for the mobile voice feature, starting every feature on the smartest available model and only then fine-tuning down to cheap dedicated infrastructure, treating "look foolish" as a P0 bug class, and refusing to auto-send any email even when their agents could.

This is a practitioner's tour of what it actually takes to put AI on top of a product that has to stay fast, stay quiet, and never embarrass the user.

We cover:

  • The model-routing strategy: Opus and frontier models to prove a feature, then fine-tuned BERT classifiers on dedicated inference
  • Pre-caching voice and tone context separately from dictation to keep the mobile voice feature feeling fast
  • Why eval engineering at Superhuman is owned by PMs, and how a single "how much time did I spend in Waymo last month" query exposes the eigenvectors a feature has to cover
  • Why "look foolish" is a P0 bug class, and where the boundary between agent agency and agent laziness actually sits
  • How Superhuman's pod structure (PM, tech lead, designer) and a central AI platform team support aligned autonomy
  • Hiring for AI fluency: how interview questions are changing and what self-augmenting engineers look like
  • Pattern detection as the leadership skill that transfers from nuclear submarines to AI email

Chapters:

(00:00) Cold open: pattern detection beats new tools
 (00:18) Loïc's path: cryptography, OpenTrust, ProductBoard, Superhuman
 (02:13) Retrofitting AI into a 100ms product
 (04:08) Voice on mobile: pre-caching LLM context to keep the feel fast
 (07:46) Frontier first, then fine-tune: model strategy across features
 (11:04) The "double-dipping" trick that worked on GPT-4 and stopped working
 (12:25) Cognitive load and staying current as a leader
 (16:59) Balancing YC founder urgency with peer CTO grounding
 (19:28) Pods, AI Guild, and aligned autonomy
 (23:15) Managing models vs. managing people: delegation in reverse
 (28:27) The Waymo example: eigenvectors of evaluation
 (32:15) Day 30 onboarding: leading the AI Inbox sprint
 (35:04) Why email is the killer agent use case
 (38:51) Auto-draft, never auto-send
 (39:57) Agent agency vs. agent laziness
 (43:07) Hiring for AI fluency
 (45:55) Pattern detection is the leadership skill
 (47:21) Nuclear submarines as engineering reference points
 (48:37) Closing thoughts
 (49:38) Superhuman is hiring

Connect with Loïc:

  • LinkedIn: https://www.linkedin.com/in/houssier/
  • Superhuman careers: https://superhuman.com/careers
  • Superhuman: https://superhuman.com

Connect with Conor:

  • Newsletter: https://newsletter.chainofthought.show/
  • Twitter/X: https://x.com/ConorBronsdon
  • LinkedIn: https://www.linkedin.com/in/conorbronsdon/
  • YouTube: https://www.youtube.com/@ConorBronsdon

More episodes: https://chainofthought.show

Thanks to Galileo — download their free 165-page guide to mastering multi-agent systems at galileo.ai/mastering-multi-agent-systems

...more
View all episodesView all episodes
Download on the App Store

Chain of Thought | AI Agents, Infrastructure & EngineeringBy Conor Bronsdon

  • 5
  • 5
  • 5
  • 5
  • 5

5

27 ratings


More shows like Chain of Thought | AI Agents, Infrastructure & Engineering

View all
The Daily by The New York Times

The Daily

112,194 Listeners