AI shopping agents can already complete a purchase through a checkout page — but until recently, they did it the hard way: parsing raw HTML, guessing which element was a button, and burning enormous amounts of time and tokens in the process. This episode digs into how Stripe rebuilt its checkout flow around WebMCP, a new browser standard that lets a webpage hand an AI agent a clean set of callable actions instead of a wall of HTML to interpret. We cover why the old approach was so expensive (nearly two million tokens and two and a half minutes for one purchase), how Stripe's design reveals only the actions that make sense at each step of checkout so the agent can't get confused or go off the rails, and why reusing the exact same backend that serves human shoppers turned out to matter as much as the new interface itself. Then we get into the numbers: a controlled test across six different AI models found the new approach cut token usage by more than 40%, shaved roughly a minute off every transaction, and still completed every single purchase successfully. Source article: "How Stripe Is Designing Checkout for AI Agents" — Stripe Dot Dev Blog, https://stripe.dev/blog/how-stripe-is-designing-checkout-for-ai-agents (published 2026-09).

Most AI agent benchmarks report one number: average accuracy across several runs of a task. But average accuracy hides something that matters a lot more in practice — whether the agent behaves the same way every time you ask it to do the exact same thing. An agent that's 80% accurate on average could be one that nails some tasks every single time and flips a coin on others, and you'd never know which from the headline number. This episode digs into a fix IBM Research built for that blind spot: instead of expensively re-running whole tasks over and over to check for flakiness, their system pinpoints the specific moments inside a single run where the agent's decision was genuinely up for grabs — moments that would tip a different way if you asked again — and turns those moments into explicit rules the agent follows the next time around. We cover how that detection works without needing any human-labeled "right answers," why the fix didn't cost the agent any of its average accuracy to gain reliability, and the concrete numbers from testing on a benchmark called AppWorld: nearly doubling the fraction of tasks the agent could nail on every single attempt, while the rules it learned also carried over to help on tasks it hadn't seen before. Source article: "Your Agent Aced the Task. Will It Do It Again?" — IBM Research / Hugging Face Blog, https://huggingface.co/blog/ibm-research/altk-evolve-consistency (published 2026-09).

Ask an LLM to write the single best headline for a product or a feed item, and it will — but "best on average" quietly means "generic," and the users with more specific tastes get left out. This episode digs into GESE, a system Baidu built to fix that: instead of asking the model for one polished headline, it trains the model to write a whole spread of different headlines, each aimed at a different way someone might be interested in the item, and then hands the job of picking the right one per user to a fast, lightweight selector running in real time. We cover why "mode collapse" is a sneaky failure mode for AI-generated personalization, how reinforcement learning is used to keep the model's outputs diverse instead of narrowing them down, and the production results from a platform with over 100 million daily users — a measurable lift in both clicks and how long people actually stuck around afterward. Source article: "Generate to Explore, Select to Exploit: Aligning LLM-based Headline Generation with Personalized Recommendation" — arXiv (Baidu Inc.), https://arxiv.org/abs/2609.15094 (published 2026-09).

A deep dive into AGAS, a new attack that uses a coordinated swarm of AI agents to manipulate what a recommendation system shows you. Instead of one bot spamming fake reviews, a central "Coordinator" directs a team of role-switching worker agents — some probing how the system reacts, some quietly building a believable rating history, and one, only once fully trusted, delivering the actual push for the target product. We cover the business problem (fake-profile "shilling" attacks are getting cheaper and more humanlike thanks to AI agents), the technical approach (the Coordinator's adaptive strategies, the workers' four rotating roles, and why going dormant on purpose is itself a tactic), and the results: the attack was dramatically more effective at promoting a target item, far better at evading existing fraud detectors, and vastly cheaper to run than prior AI-driven attacks — while barely disturbing the experience for everyone else on the platform. Source article: "An Efficient and Effective Agentic Group Shilling Attack on Recommender Systems" — arXiv (academic), https://arxiv.org/abs/2609.09551 (published 2026-09).

A deep dive into KernelEvolve, the system Meta built to close a widening gap: the number of GPU and accelerator kernels a company needs optimized grows multiplicatively across hardware generations, model architectures, and operator types, while expert kernel engineers — the people who could tune them by hand — don't scale nearly as fast. We cover the business problem (thousands of configurations across NVIDIA and AMD GPUs, Meta's own MTIA chips, and CPUs, each needing its own hand-tuned kernel), the technical approach (an LLM synthesizer that writes candidate kernels across multiple languages, a Monte Carlo tree search engine that explores hundreds of variants per kernel instead of generating just one, a knowledge base that compounds what it learns across sessions, and a final stage that turns successful optimization runs into training data for smaller specialist models), and the realized payoff: over 60% inference throughput improvement on NVIDIA GPUs for a production ads model, over 25% training throughput improvement on Meta's own chips, and kernel optimization work that used to take weeks of expert time now running in hours. Source article: "KernelEvolve: How Meta's Ranking Engineer Agent Optimizes AI Infrastructure" — Engineering at Meta, https://engineering.fb.com/2026/04/02/developer-tools/kernelevolve-how-metas-ranking-engineer-agent-optimizes-ai-infrastructure/ (published 2026-04).

Airbnb needed to understand the rare, risky edge cases a new AI customer-service assistant might run into — the kind of thing that used to take a data scientist months of manually reading through support transcripts, one judgment call at a time. So Airbnb built Insight Miner: an agent harness that doesn't just run an LLM over the data, but wraps it in an explicit scientific-methodology layer — extract, embed, cluster, then classify — so the process of investigation, not just the answer, is reproducible and auditable by someone other than the original analyst. We walk through how the harness is built, why "methodology as infrastructure" is the real idea here rather than any single new technique, and how investigations that used to take months now take days — with usage spreading from a handful of data scientists to dozens of non-technical teams across the company. Source article: "Beyond the Model: Engineering AI Infra with Scientific Judgement" — Airbnb Tech Blog, https://airbnb.tech/ai-ml/beyond-the-model-engineering-ai-infra-with-scientific-judgement/ (published 2026-09).

Bayesian A/B testing gets pitched as more intuitive, safer to peek at early, and automatically smarter about multiple metrics — so why didn't Spotify adopt it? This episode walks through Spotify's own math: under the flat priors most platforms actually ship, Bayesian and frequentist testing produce numerically identical results, and the "advantages" people cite either require infrastructure most teams don't have (calibrated priors, Bayes-factor stopping, hundreds of historical experiments) or quietly reduce to standard frequentist practice anyway. We cover the business context (why experimentation trustworthiness matters more than framework fashion), the statistical reasoning (posterior equivalence, the winner's curse, decision theory), and what it means for teams facing the same "should we go Bayesian?" question. Source article: "Why Spotify Is Not Using Bayesian A/B Testing" — Spotify Engineering, https://engineering.atspotify.com/2026/9/why-spotify-is-not-using-bayesian-a-b-testing (published 2026-09).

Instacart's machine learning engineers ran an experiment most teams are quietly wondering about: what happens when you let a coding agent loose not just on your pipeline code, but on the actual modeling work — picking features, trying architectures, tuning hyperparameters — for models that have already been optimized by humans for months? The results were a genuine surprise in places: on some of Instacart's most mature, hardest-to-improve production models, agent-driven runs delivered 3-5% offline error reductions among their most promising attempts, holding up against a battery of randomization checks before anyone trusted them. But the team is just as candid about the other side — agents with broad permissions in a modeling environment are a real safety surface, not a hypothetical one, and they're recommending sandboxing and compliance awareness even after their own trial runs went mostly smoothly. Source article: "Agentic Machine Learning Modeling at Instacart" — Instacart Tech Blog, https://tech.instacart.com/agentic-machine-learning-modeling-at-instacart-fb3ecd295ee7 (published 2026-09).

Every show on Netflix lives or dies a little by its synopsis — the couple of sentences that decide whether you press play. With a catalog too vast for humans to proofread line by line, Netflix built a team of AI "judges" to score synopses on tone, clarity, precision, and factual accuracy, each specializing in its own dimension rather than trying to do it all at once. To make sure those judges could actually be trusted, the team calibrated them against 600 hand-labeled examples through eight rounds of work with real creative writers — and didn't stop until the AI's agreement with expert writers matched how much the writers agreed with each other. The payoff: judged quality scores turn out to predict real viewer behavior, like whether people actually click play and how often they abandon a title early, letting Netflix catch a weak synopsis months before a show even launches. Source article: "Evaluating Netflix Show Synopses with LLM-as-a-Judge" — Netflix Technology Blog, https://netflixtechblog.com/evaluating-netflix-show-synopses-with-llm-as-a-judge-6269251e6f28 (published 2026-04).

A deep dive into how Google Research flipped the standard recipe for training AI models to use tools and APIs. Instead of writing a user question first and then struggling to find a matching chain of tool calls that answers it — which fails constantly once you're picking from thousands of real APIs — their ToolGrad system builds a verified, working tool-call workflow first, using an iterative loop that treats an AI critic's plain-language feedback as a steering signal (borrowed from a technique called "textual gradients"), and only then writes the question that workflow answers. We cover the business problem (query-first data generation wastes enormous effort on unanswerable or unverifiable examples), the technical approach (a four-step propose-execute-select-update loop building tool-call chains from a catalog of 16,000+ real APIs), and the standout result: a fine-tuned 12-billion-parameter model trained on this data scored competitively with Gemini 2.5 Pro and Claude 4.5 Opus on a leading function-calling benchmark — and notably outperformed the very model that generated its own training data. Source article: "ToolGrad: Efficient Tool-Use Dataset Generation with Textual 'Gradients'" — Google Research, https://research.google/blog/toolgrad-efficient-tool-use-dataset-generation-with-textual-gradients/ (published 2026-09).