Snacks Weekly on Data Science

In this episode, we discuss how Pinterest moved from three independently evolving ads engagement models toward a unified modeling framework. The team started with a simple merged model, then progressively refined it by combining complementary modeling components, adding surface-specific calibration and multi-task learning, preserving surface-specific model exports, and optimizing both model computation and serving infrastructure. The result was a system that could share learning across surfaces while still supporting the differences that mattered for each one, with the final approach delivering significant improvements in both offline and online metrics.

For more details, you can refer to their published tech blog, linked here for your reference: https://medium.com/pinterest-engineering/unifying-ads-engagement-modeling-across-pinterest-surfaces-4b5cd3d99e67

In this episode, we talk about Lyft’s need to predict whether riders will convert across a wide range of highly specific situations, where the available data becomes increasingly sparse as the situations become more specific. Instead of relying entirely on a global model or trusting noisy local estimates, Lyft built a hierarchical Bayesian framework where specific models can borrow information from broader models. This allows the system to capture local behavior when enough evidence exists while relying more on stable, broader patterns when local data is limited.

For more details, you can refer to their published tech blog, linked here for your reference: https://eng.lyft.com/predicting-rider-conversion-in-sparse-data-environments-with-bayesian-trees-07227ff92789

In this episode, we discuss the budget allocation problem faced by Atlassian when trying to optimize its marketing investment decisions across many channels, products, and geographic markets. The company solved this with a two-stage system: a global payout model that predicts how business outcomes change with spend, followed by a constrained optimizer that finds the best allocation across the entire portfolio. They then added an independent optimizer as a reliability check and connected the system to a continuous offline-online feedback loop. The result was a double-digit percentage reduction in cost at equivalent spending levels, showing that significant gains can come simply from allocating the same money more intelligently.

For more details, you can refer to their published tech blog, linked here for your reference: https://www.atlassian.com/blog/how-we-build/budget-allocation-at-scale-building-ml-optimization-system

In this episode, we discuss how LinkedIn moves beyond keyword matching to find candidates based on actual qualification fit. The team first created a scalable relevance definition using LLM-based supervision, distilled that knowledge into customized embeddings, and then designed the retrieval, ranking, infrastructure, and evaluation system around that signal. The result was a search system that improved both candidate relevance and recruiter engagement while actually returning fewer candidates.

For more details, you can refer to their published tech blog, linked here for your reference: https://www.linkedin.com/blog/engineering/ai/semantic-search-for-ai-agents-at-scale-retrieval-and-ranking-for-linkedins-hiring-assistant

In this episode, we discuss how Meta measures the business value of shared ML infrastructure when that value appears indirectly through downstream model launches. Instead of relying on adoption, usage, or traditional infrastructure metrics, the team built an observational causal framework around individual model launches. By comparing feasible adopters with comparable non-adopters and controlling for major confounding factors, the team estimated the incremental revenue associated with infrastructure adoption. This produced a very different picture from simply looking at which platforms were used the most.

For more details, you can refer to their published tech blog, linked here for your reference: 

https://medium.com/@AnalyticsAtMeta/measuring-what-matters-how-causal-inference-turned-ai-infrastructure-into-a-quantifiable-business-bbc49bddaa2b

In this episode, we discuss two LLM-assisted pipelines for turning unstructured text into a structured taxonomy with minimal manual annotation. The bottom-up approach lets semantic representations and clustering discover categories from the data, making it useful for open-ended exploration and relatively efficient at scale. The top-down approach starts with LLM-generated categories and recursively classifies and subdivides the corpus, making it more controllable and easier to align with business concepts, but at a higher computational cost. What makes both approaches effective is not simply the use of an LLM, but the way the LLM is combined with conventional techniques to create a systematic workflow for a problem that was previously highly manual.

For more details, you can refer to their published tech blog, linked here for your reference: https://medium.com/data-science-at-microsoft/from-chaos-to-clarity-building-taxonomies-from-unstructured-text-using-large-language-models-c1303db3adb1

In this episode, we talk about how Instacart identified a limitation of a standard taxonomy: it could classify products, but it couldn't connect them the way real shoppers do. The team developed a solution around Semantic IDs, which compress product embeddings into hierarchical discrete codes, while contrastive learning and taxonomy-based supervision make those codes more semantically meaningful. The result is a representation that can support better product understanding, catalog quality, and related business needs.

For more details, you can refer to their published tech blog, linked here for your reference: https://tech.instacart.com/semantic-ids-product-understanding-at-scale-5283e0288f5a 

In this episode, we discuss how Faire addresses a fundamental bottleneck in machine learning development: there are more potential experiments than human scientists can realistically run. They developed Autoscience, giving an AI agent a structured environment where it can plan, execute, evaluate, remember, and continuously refine experiments. What makes the approach effective is not the agent alone, but the combination of a reliable experimentation harness, integration with the existing ML stack, and the context needed to explore like an experienced Applied Scientist.

For more details, you can refer to their published tech blog, linked here for your reference: https://craft.faire.com/autoscience-how-we-built-agent-driven-ml-exploration-at-faire-f3183e7deb29 

In this episode, we discuss how DoorDash built a reliable metadata generation platform that could operate across millions of constantly changing menu items. The solution combined multimodal generation, automated evaluation through LLM juries, optimization loops for improving model behavior, and distributed infrastructure for cost-efficient scaling. By transforming unstructured menu information into precise, structured attributes, DoorDash unlocked many new product capabilities that ultimately helped the company create a more seamless food discovery experience for customers.

For more details, you can refer to their published tech blog, linked here for your reference: https://careersatdoordash.com/blog/building-food-metadata-with-llm-juries-context-optimization-multimodal-ai/

In this episode, we explore how The New York Times built a real-time decision system that estimates the causal impact of different paywall actions, optimizes those decisions based on business objectives, and continuously measures performance using long-term business outcomes. The result is a more flexible and personalized subscription experience that balances immediate conversions with long-term growth.

For more details, you can refer to their published tech blog, linked here for your reference: https://open.nytimes.com/scaling-subscriptions-at-the-new-york-times-with-real-time-causal-machine-learning-5f23a7b24ff4