Data Science in the Wild

Google's ToolGrad flips tool-use data generation: run real API calls first, then write the question they answer. A 12B model trained on just 500 examples scored 83.1 on the Berkeley Function Calling Leaderboard, beating GPT-5's 74.4.

Key takeaways:

• Answer first. Writing a query first and hunting for an API chain that solves it often fails. ToolGrad builds a valid chain of executed calls and writes the user query afterward, reaching a 99.8% annotation pass rate.
• Textual “gradients.” An API proposer suggests next calls, they run in parallel, and an LLM critic picks a branch with plain-language feedback that steers the next step.
• Small data, strong model. ToolGrad-500, built from ToolBench's 16k+ APIs, fine-tuned Gemma-3 into ToolGrad-12B. It scored 83.1, close to Gemini 2.5 Pro (83.2) and Claude 4.5 Opus (82.8).
• The student beats the teacher. The data came from Gemini 2.5 Flash-Lite, yet the fine-tuned 12B model outperformed it.
• Our take. Generation is grounded in real execution, so the data is clean by construction.

Chapters:

00:00 Shooting arrows blindfolded

00:47 From query-first to answer-first

02:19 Why query-first fails

03:50 Execution as ground truth

04:36 The API proposer

05:22 Parallel execution

06:07 Textual gradients and the critic

08:24 ToolGrad-500 and ToolGrad-12B

09:55 Benchmark results

11:28 Beating the teacher

13:01 Related work

14:34 Our take

16:07 Recap

Sources:

• ToolGrad: Efficient tool-use dataset generation with textual “gradients” (Google Research blog, 2026)

Related episode: E8: Airbnb — The LLM Eval Loop That Went From Weeks to a Day

Enjoyed it? Follow Data Science in the Wild (one real production ML/AI system or paper per episode, hosted by Peter Liu) and rate it on Apple Podcasts or Spotify.

Also from Peter: The GenAI Evolution Atlas, Wednesday AI deep dives and Friday news roundups.

Made with AI-generated voices; curated by Peter Liu from the sources above.

When an AI agent calls another agent that changes production, who's responsible? Uber gives every agent hop its own short-lived token that records the whole chain back to a human, with P99 latency under 40 milliseconds.

Key takeaways:

• Old identity sees only the last caller. In agent-to-agent chains, a policy engine can't tell which human or agent started a request.
• Two layers of proof. SPIFFE/SPIRE verifies the workload. A separate agent identity ties the specific agent to it, so an agent can't pretend to be one it isn't allowed to host.
• Exchange tokens at every hop. Based on OAuth 2.0 Token Exchange (RFC 8693), each hop trades its token at a security token service for a new, short-lived JWT carrying the full actor chain. There's no master token to steal.
• Enforced at the gateway. An MCP gateway checks the chain before an agent touches tools or services, and the chain is traceable in observability.
• Fast and adopted. Token exchange P99 latency stays under 40 ms, and thousands of internal agents use the system.

Chapters:

00:00 A 2 a.m. outage caused by an agent

01:33 Why agents break identity

03:52 The telephone-game problem

06:11 Workload identity with SPIFFE/SPIRE

07:42 Carrying the actor chain

09:12 Token exchange at every hop

10:45 The MCP gateway

13:01 Latency and adoption

14:34 What's novel

16:05 Meta's REA and other approaches

17:37 Revocation mid-task

19:08 Limitations and next steps

21:26 Closing question

Sources:

• Solving the Identity Crisis for AI Agents (Uber Engineering, May 2026)
• RFC 8693: OAuth 2.0 Token Exchange (IETF)

Related episode: E6: Shopify — An AI Agent That Verifies Its Own Work

Enjoyed it? Follow Data Science in the Wild (one real production ML/AI system or paper per episode, hosted by Peter Liu) and rate it on Apple Podcasts or Spotify.

Also from Peter: The GenAI Evolution Atlas, Wednesday AI deep dives and Friday news roundups.

Made with AI-generated voices; curated by Peter Liu from the sources above.

Netflix's GenPage builds your entire homepage, rows and titles, with one transformer in one pass, replacing separate retrieval, ranking and layout systems. In A/B tests it lifted engagement and cut serving latency by 20%.

Key takeaways:

• The homepage as a generation task. Your history and context form the prompt, and the page is generated as a structured sequence from a restricted vocabulary of rows and titles.
• Pretrain, then post-train. GenPage first learns from production pages, then is tuned with weighted binary classification or reinforcement learning, so pages people actually engaged with count more.
• Diversity for free. RL post-training made homepages more diverse even though diversity was never part of the objective.
• Richer prompts beat bigger models. Offline, enriching the prompt helped more than scaling model capacity, at least in Netflix's current regime.
• Faster, not slower. Against a mature production recommender, GenPage improved the core engagement metric and cut end-to-end serving latency by 20% by removing hops between services.

Chapters:

00:00 A homepage written just for you

01:30 The legacy multi-stage stack

03:02 Why stage-by-stage is slow

04:34 The homepage as a prompt

06:06 Restricted vocabulary, structured output

06:51 Pretraining and post-training

09:55 Diversity nobody asked for

11:27 Prompts vs. model size

13:00 Online results

15:18 How novel is it?

16:49 Companion work

18:20 The end of hard-coded layouts

20:38 Closing question

Sources:

• GenPage: Towards End-to-End Generative Homepage Construction at Netflix (Wang, Pan, Baltrunas et al., Netflix; arXiv, Jun 2026 (RecSys 2026))

Related episode: E10: Tencent — Teaching a Recommender to Reason About Cold Starts

Enjoyed it? Follow Data Science in the Wild (one real production ML/AI system or paper per episode, hosted by Peter Liu) and rate it on Apple Podcasts or Spotify.

Also from Peter: The GenAI Evolution Atlas, Wednesday AI deep dives and Friday news roundups.

Made with AI-generated voices; curated by Peter Liu from the sources above.

AWS and NVIDIA's “physical AI model factory” trains robot and self-driving models in a continuous loop on one cluster. One Cosmos 3 checkpoint can imagine the future, label actions in raw video, and drive the robot.

Key takeaways:

• One cluster, closed loop. Synthetic-data generation, post-training and evaluation share one SageMaker HyperPod cluster and storage instead of separate clusters that sit idle.
• One token stream. Cosmos 3 treats video, images, actions and sound as a single sequence, with a reasoner and a generator joined by attention at every layer.
• Three jobs, one checkpoint. Which tokens start as noise decides the mode: world model (predicts the next frames), action labeler (infers the action behind a change), or policy (controls the robot).
• Sizes. Super is 64B on a 32B Qwen3-VL backbone, adapted with rank-16 LoRA. Nano is 16B, and Edge is a 4B model for on-device use.
• Efficiency. Super stayed near-linear (about 0.97 of linear or better) from one to four 8×H200 nodes. MFU was about 0.50 for Super and 0.24 for Nano.

Chapters:

00:00 Robots that stop learning

00:45 AWS and NVIDIA's model factory

01:32 The closed loop

03:04 Why separate clusters waste GPUs

04:35 Cosmos 3: one token stream

06:08 Reasoner and generator

08:26 Model tiers and LoRA

09:59 Three modes, one checkpoint

12:16 Running on SageMaker HyperPod

14:33 Scaling and goodput

16:03 Checkpoint timing (Young/Daly)

16:50 What's new

17:36 Industry comparisons

19:53 Recap

Sources:

• Build a Physical AI model factory with NVIDIA Cosmos 3 on SageMaker HyperPod (AWS Machine Learning Blog, 2026)

Related episode: E4: Google — Ending the Six-Hour Forecast Gap

Enjoyed it? Follow Data Science in the Wild (one real production ML/AI system or paper per episode, hosted by Peter Liu) and rate it on Apple Podcasts or Spotify.

Also from Peter: The GenAI Evolution Atlas, Wednesday AI deep dives and Friday news roundups.

Made with AI-generated voices; curated by Peter Liu from the sources above.

Tencent's TGR pushes its production recommenders, which serve hundreds of millions of users, from a fragmented four-stage cascade toward one generative model that can reason. On cold starts, new-user Hit@1 rose 477.8%.

Key takeaways:

• Why cascades hurt. Retrieval, pre-ranking, ranking and reranking are optimized separately. That fragments decisions and caps scaling.
• CCFormer, a transformer ranker. It keeps each feature's meaning and compresses long histories. Results: +3.57% CTR and +1.71% ad revenue, fully launched in two of five tested scenarios.
• Generating with semantic IDs. BARGE keeps generated item IDs valid. HiGR generates a whole slate at once, with a 5x inference speedup and up to +1.22% watch time.
• Reasoning, computed offline. TGR-Reason produces reasoning tokens offline and injects them at serving time. Cold-start new-user Hit@1 rose 477.8% and new-user conversion rose 13.09%.
• Our take. Few of the parts are new; the novelty is offline reasoning and making it all run at Tencent scale.

Chapters:

00:00 The hiring-funnel analogy

00:46 Tencent's TGR

03:03 Why cascades fragment decisions

04:35 CCFormer: a new ranking engine

08:21 Semantic IDs

09:53 BARGE: valid IDs

10:38 HiGR: generating whole slates

12:11 TGR-Reason: offline reasoning

13:45 Results

14:31 Solving cold starts

15:16 How novel is it?

18:18 Netflix and related work

19:51 Recap

Sources:

• TGR: Advancing Industrial Recommendation from Generative-Paradigm Ranking toward Unified Generation and Reasoning (TGR Team, Tencent; arXiv, Sep 2026)

Related episode: E9: NetEase — The Agent That Audits Its Own Recommender

Enjoyed it? Follow Data Science in the Wild (one real production ML/AI system or paper per episode, hosted by Peter Liu) and rate it on Apple Podcasts or Spotify.

Also from Peter: The GenAI Evolution Atlas, Wednesday AI deep dives and Friday news roundups.

Made with AI-generated voices; curated by Peter Liu from the sources above.

NetEase's AutoLR uses AI agents to take recommender improvements from idea to Launch Review, at RMB 3–4 in LLM API cost per iteration. It also caught its own failure mode: noisy wins that quietly reset the baseline.

Key takeaways:

• Agents propose, a harness decides. LLM agents generate and analyze ideas. A deterministic harness runs code, evaluations and the ledger, so every decision can be audited at Launch Review.
• Specialists, then a roundtable. Separate code-constraint and domain agents critique each proposal before a synthesis step merges overlapping ideas.
• Cheap enough to run routinely. Moving routine iterations from Claude Opus-class models to a DeepSeek-V4-Pro/Flash stack brought LLM API cost to RMB 3–4 per iteration, excluding training compute.
• Results, read carefully. The audit covers 1,586 completed evaluations. Nine Launch Reviews in NetEase's Dashen app showed positive lifts summing to +5.75% consumption penetration, +10.83% consumption time and +5.55% valid views. These are descriptive sums, not one causal effect.
• The KEEP ratchet. Noisy offline wins become the new baseline and fake steady progress. The fix is protocol-matched repeats.

Chapters:

00:00 The factory behind your feed

01:31 Why recommender upgrades are slow

03:49 Rule of separation

05:23 Specialist agents vs. context dilution

06:54 The roundtable synthesis

09:11 Choosing what to test

11:29 Cutting LLM costs

12:14 Results across nine launches

13:45 The KEEP ratchet

15:17 Fix: protocol-matched repeats

17:34 Compared with freer agent systems

19:06 Closing thoughts

Sources:

• AutoLR: Automating the Path from Research to Launch Review in Industrial Recommender Systems (Zhang et al., NetEase; arXiv, Sep 2026)

Related episode: E10: Tencent — Teaching a Recommender to Reason About Cold Starts

Enjoyed it? Follow Data Science in the Wild (one real production ML/AI system or paper per episode, hosted by Peter Liu) and rate it on Apple Podcasts or Spotify.

Also from Peter: The GenAI Evolution Atlas, Wednesday AI deep dives and Friday news roundups.

Made with AI-generated voices; curated by Peter Liu from the sources above.

Airbnb cut LLM evaluation from weeks to about a day. Four layers: name the noise, make evaluation deterministic, patch bugs with tiny LoRA micro-adapters, and test the seams between components.

Key takeaways:

• Name the noise first. About three-quarters of LLM-generated reference answers differed across labeling runs, and the judge drifted about 1% run to run. Separating fixable (epistemic) from inherent (aleatoric) noise tells you what to fix.
• Make evaluation deterministic. Caching results by sample and configuration means reruns stop regenerating noise and stay comparable.
• Patch, don't retrain. Micro-adapters are small LoRA patches (rank under 50) on a shared adapter, trained in under an hour on one GPU, shipped like hotfixes behind two gates and retired under lifecycle rules.
• Test the seams. End-to-end checks catch failures that pass every component test, like a correct retrieval misread by the LLM.
• Our take. It's classic software discipline applied to LLMs, and the judge must be calibrated before it's trusted.

Chapters:

00:00 A fix that takes weeks to verify

00:46 Airbnb's evaluation rebuild

02:17 The slow, noisy feedback loop

04:34 Layer 1: name the noise

06:51 Drifting references and judges

07:36 Layer 2: deterministic caching

09:55 Layer 3: LoRA micro-adapters

12:15 Governing the patches

13:46 Layer 4: testing the seams

14:31 Impact: weeks to a day

15:16 Our analysis

18:18 Recap and closing question

Sources:

• From weeks to a day: how we made LLM evaluation fast enough to iterate on (Baharak Saberidokht, Airbnb Engineering, Jul 2026)

Related episode: E15: Netflix — Grading the Graders: Can an AI Judge Your Synopsis?

Enjoyed it? Follow Data Science in the Wild (one real production ML/AI system or paper per episode, hosted by Peter Liu) and rate it on Apple Podcasts or Spotify.

Also from Peter: The GenAI Evolution Atlas, Wednesday AI deep dives and Friday news roundups.

Made with AI-generated voices; curated by Peter Liu from the sources above.

Airbnb's JourneyFormer replaces hundreds of hand-crafted guest statistics with a transformer that reads each guest's raw event history. In a three-week A/B test it added 0.55% more bookers in search and 5.04% more email views.

Key takeaways:

• Aggregates erase the story. Hand-built “what this guest likes” features lose order, recency and co-occurrence. A sequence model keeps them.
• Two timelines. A long-term sequence of milestones like bookings and a short-term sequence of recent activity (capped at 200 events), with location encoded through 14 levels of S2 cells.
• Built for serving cost. 128-dimension embeddings are shrunk to 32 before the guest profile is stored, and sparse computation skips empty sequence slots.
• Results. Offline NDCG rose 1.48%. Online, search saw +0.55% bookers, +0.82% booked nights and +0.90% views, and email ranking saw +5.04% views.
• Our take. The tools aren't new; the work is in taming long, noisy, real travel histories.

Chapters:

00:00 Guessing a friend's favorite movie

00:45 Airbnb's JourneyFormer

02:17 What aggregate features lose

03:49 Years of history at scale

05:20 S2 cells: encoding place

06:50 The transformer encoder

09:54 Shrinking embeddings for serving

11:25 Sparse computation

12:11 Results offline and online

14:30 What's genuinely new

17:33 Related work

18:19 Commentary and closing question

Sources:

• JourneyFormer: Encoding Airbnb Guest Journey with Sequence Modeling (Zha et al., Airbnb; arXiv, Jun 2026 (KDD '26))

Related episode: E1: LinkedIn — A Transformer That Ranks Your Feed

Enjoyed it? Follow Data Science in the Wild (one real production ML/AI system or paper per episode, hosted by Peter Liu) and rate it on Apple Podcasts or Spotify.

Also from Peter: The GenAI Evolution Atlas, Wednesday AI deep dives and Friday news roundups.

Made with AI-generated voices; curated by Peter Liu from the sources above.

Shopify's River is an AI agent that closes the gap between “a security patch was written” and “the vulnerability is gone in production.” It re-verifies fixes against the live repo, and security merges rose from about 10% to 80%.

Key takeaways:

• A patch isn't a fix. A PR that passed CI can go stale before merge. River distrusts tracker records and re-checks the live code before anything is marked resolved.
• It does the tedious work. River rebases, retriggers CI and repairs lock files. On one run, it opened four Slack threads covering 35 findings within a minute.
• It knows when to hand off. Judgment calls go to engineers with the evidence attached.
• Results. In the first 11 days of the dependency workflow, the open-issue backlog fell about 70%. Security merges through the freshness-gated queue went from about 10% to 80%.
• What's new. Not that an AI writes code, but the discipline of tying evidence to commits before closing anything.

Chapters:

00:00 When “done” isn't done

00:47 Shopify's River

01:32 The gap between a patch and a fix

03:48 Stale PRs and a growing backlog

05:19 River rechecks the live code

07:41 Rebasing, CI and lock files

09:59 Many fixes at once

10:46 Handing off to humans

12:17 Evidence before “resolved”

13:03 Results: backlog down 70%

15:21 Beyond security patching

16:06 What's actually new

18:23 Recap

Sources:

• How River takes security work from a fix to merge (Erin Son and Kaiyi Li, Shopify Engineering, Sep 2026)

Related episode: E2: Shopify — An AI Agent That Trains Itself

Enjoyed it? Follow Data Science in the Wild (one real production ML/AI system or paper per episode, hosted by Peter Liu) and rate it on Apple Podcasts or Spotify.

Also from Peter: The GenAI Evolution Atlas, Wednesday AI deep dives and Friday news roundups.

Made with AI-generated voices; curated by Peter Liu from the sources above.

Meta's SilverTorch turns recommendation retrieval into one GPU-native PyTorch model: ANN search, filtering and scoring that used to be separate CPU services. “Index as Model” reports up to 23.7× higher throughput.

Key takeaways:

• Collapse the stack. Retrieval normally chains an ANN index, feature filters and scoring models as separate services; every hop adds network and serialization cost.
• Index as model. SilverTorch expresses the index, filters and scorers as tensors and operators inside one trainable, servable model on GPUs.
• Two key kernels. A GPU Bloom index handles candidate filtering, and Int8-quantized embeddings with a fused ANN kernel make search fit in GPU memory with little accuracy loss.
• The payoff. Up to 23.7× more requests per second than a multi-service baseline, and a 13.35× cost-per-request advantage over the CPU-based approach with neural reranking.
• Not for everyone. The hosts' take: a GPU monolith fits Meta's scale; smaller teams may not need to tear out their microservices.

Chapters:

00:00 The plumbing behind your feed

00:46 Meta's “Index as Model” idea

02:20 The classic stack: ANN, filters, scoring

03:54 Why service hops hurt

05:26 What “index as model” means

07:45 A GPU Bloom filter

10:02 Int8 and the fused ANN kernel

12:17 Unified scoring layers

13:49 Results: 23.7× throughput

16:04 How novel is it?

17:39 Related work at Meta and Pinterest

19:56 Should everyone drop microservices?

22:13 Recap and closing question

Sources:

• SilverTorch: Index as Model — A New Retrieval Paradigm for Recommendation Systems (Engineering at Meta, May 2026)

Related episode: E1: LinkedIn — A Transformer That Ranks Your Feed

Enjoyed it? Follow Data Science in the Wild (one real production ML/AI system or paper per episode, hosted by Peter Liu) and rate it on Apple Podcasts or Spotify.

Also from Peter: The GenAI Evolution Atlas, Wednesday AI deep dives and Friday news roundups.

Made with AI-generated voices; curated by Peter Liu from the sources above.