A deep dive into how Shopify rebuilt the improvement loop for Sidekick, its AI agent, as a four-stage flywheel that keeps getting better without perpetually scaling frontier-model spend. We cover the business context (quality that gets expensive fast when you only buy it by upgrading models), and the technique: a human-calibrated quality rubric (validated with Cohen's kappa), an LLM-as-judge tuned with DSPy/GEPA/ACE and — crucially — validated against real production A/B outcomes before it's trusted, automated prompt/tool research, and continual learning via SFT plus GRPO fine-tuning that uses the calibrated judge itself as the reward signal. The payoff: serving cost for a GraphQL agent handling 2,000 requests/min fell roughly 96%, from ~$27M to ~$1M/year. The real lesson is the discipline of calibrating the judge before using it as a reward.

Source article: "Sidekick's Continual Learning Loop" — Shopify Engineering, https://shopify.engineering/sidekicks-continual-learning-loop (published 2026-08).

A deep dive into how LinkedIn replaced its DCNv2 feed ranker with Feed SR — a decoder-only transformer that reads up to 1,000 of a member's past impressions as an ordered sequence, served on a disaggregated CPU/GPU stack with custom kernels. We cover the business context (a 1.2B-member feed whose legacy ranker couldn't model history as a sequence), the technical approach (RoPE positional embeddings, a parallel-DCNv2 scoring head, shared-context batching, and a custom flash-attention CUDA kernel), the honest negative results (why a fine-tuned LLM-ranker and a TransAct-style encoder were tried and rejected), and the realized impact: +2.10% time spent and +3.52% social actions in a production A/B test — at roughly 0.7× the inference energy of the CPU-served incumbent despite being a larger model.

Source article: "An Industrial-Scale Sequential Recommender for LinkedIn Feed Ranking (Feed SR)" — LinkedIn / arXiv, https://arxiv.org/abs/2602.12354 (published 2026-02).