In this episode, we will explore why evaluating LLM-based chatbots is critical for businesses, the limitations of traditional evaluation methods, and what could be a good robust evaluation framework covering both search performance and LLM-specific metrics. 
For more details, you can refer to their published tech blog, linked here for your reference: https://medium.com/data-science-at-microsoft/evaluating-llm-based-chatbots-a-comprehensive-guide-to-performance-metrics-9c2388556d3e

In this episode, we will explore how The New York Times balances algorithmic recommendations with editorial judgment. We discuss their business challenge, examine their hybrid content recommendation system, and look at refinements designed to improve the reader experience.

For more details, you can refer to their published tech blog, linked here for your reference: https://open.nytimes.com/how-the-new-york-times-incorporates-editorial-judgement-in-algorithms-to-curate-home-screen-content-85f48209fdad

In this episode, we will explore how Airbnb upgraded its conversational AI system, leveraging LLMs in a controlled and predictable way. We will first examine their business needs, highlighting why traditional chatbot-based workflows were no longer sufficient. Then, we will break down their technical solution, which combines structured workflows with AI-powered reasoning, context management, and a guardrail framework. This ensures that AI enhances automation while staying reliable, context-aware, and well-guarded against risks.
For more details, you can refer to their published tech blog, linked here for your reference: https://medium.com/airbnb-engineering/personal-data-classification-2d816d8ea516

In this episode, we will explore the importance of the Large Language Model (LLM) and the forces shaping the LLM economy: competition among AI giants, GPU scarcity, and tokens as the new currency. These dynamics drive innovation and challenge businesses to optimize resources and costs strategically.

For more details, you can refer to their published tech blog, linked here for your reference: https://medium.com/wix-engineering/the-emerging-economy-of-llms-883f2ab13067

In this episode, we will explore why Klaviyo developed its global holdout group feature and how its engineering team overcame the technical challenges. This feature helps Klaviyo’s customers run fair and unbiased experiments across multiple marketing channels, ultimately enhancing the accuracy of their marketing performance insights.

For more details, you can refer to their published tech blog, linked here for your reference: https://klaviyo.tech/creating-global-holdout-groups-8d48c8cf7266

In this episode, we will explore why code reviews are critical for a fast-growing marketplace like Faire and the challenges that come with scaling them manually as the engineering team expands. We’ll dive into how Large Language Models (LLMs) offer a game-changing solution—automating code reviews by providing instant, context-aware feedback, enforcing coding best practices, and integrating seamlessly into existing development workflows.

For more details, you can refer to their published tech blog, linked here for your reference: https://craft.faire.com/automated-code-reviews-with-llms-cf2cc51bb6d3 

In this episode, we will explore how Instacart uses data science to optimize its incentive promotions. We will discuss the business challenge, introduce the concept of surrogate indices, and walk through the step-by-step process of building and applying one.

For more details, you can refer to their published tech blog, linked here for your reference: https://tech.instacart.com/instacarts-economics-team-using-surrogate-indices-to-estimate-long-run-heterogeneous-treatment-0bf7bc96c6e6

In this episode, we will explore Meta’s AI scaling challenges and how the company leverages productivity and efficiency to optimize its computing power. We also discuss how analytics insights help identify active levers to improve AI development.

For more details, you can refer to their published tech blog, linked here for your reference: https://medium.com/@AnalyticsAtMeta/innovation-demands-compute-how-to-enable-ml-productivity-and-efficiency-38f56cf520f7

In this episode, we will explore how Coupang integrates Large Language Models (LLMs) to enhance its machine learning ecosystem. We'll break down Coupang’s business model, key machine learning categories, and the role of Foundation Models in improving efficiency and accuracy. Additionally, we'll walk through the LLM development lifecycle, discuss critical infrastructure decisions, and examine strategies for overcoming challenges such as GPU scarcity and scaling AI operations.

For more details, you can refer to their published tech blog, linked here for your reference:https://medium.com/coupang-engineering/accelerating-coupangs-ai-journey-with-llms-2817d55004d3 

In this episode, we will explore how Expedia ranks lodging options to optimize both customer experience and business objectives. We will discuss the business problem—how ranking impacts Expedia’s success and the challenges of hotel recommendations. Then, we will break down the data science solution, covering the objective functions, features/signals, machine learning architectures, and the evaluation of the system.
For more details, you can refer to their published tech blog, linked here for your reference:https://medium.com/expedia-group-tech/choosing-the-right-candidates-for-lodging-ranking-d0841bf40c0e