MLOps.community

Feature Stores at Shopify and Skyscanner // Matt Delacour and Mike Moran // Reading Group #4


Listen Later

MLOps Reading Group meeting on February 11, 2022  

Join the Community: ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠https://go.mlops.community/YTJoinIn⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠

Get the newsletter: ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠https://go.mlops.community/YTNewsletter⁠⁠⁠⁠


Reading Group Session about Feature Stores with Matt Delacour and Mike Moran  

--------------- ✌️Connect With Us ✌️ -------------
Join our Slack community: https://go.mlops.community/slack
Follow us on Twitter: @mlopscommunity
Connect with us on LinkedIn: https://www.linkedin.com/company/mlopscommunity/
Sign up for the next meetup: https://go.mlops.community/register
Catch all episodes, Feature Store, Machine Learning Monitoring, and Blogs: https://mlops.community/

Timestamps:
[00:05] Matt's intro
[00:26] Mike's intro
[01:09] Matt’s talk: Feature store system at Shopify
[01:45] What is Shopify?
[02:05] Shopify Use Case
[02:38] Choosing a solution
[03:19] Managed service vs In-house vs Open-source (Feast)
[06:01] Why did we choose Feast?
[11:25] Implementation Strategy (multi-repo vs mono-repo approaches)
[13:01] Mono-repo approach breakdown
[14:30] Internal SDK
[17:01] Q&A: Does Feast satisfy scalability for online inference of Shopify's latency requirements?
[19:05] Q&A: Do you rely on Feast to serialize data to the online store?
[20:13] Q&A: Is your mono-repo library a subset of Feast?
[21:18] Q&A: Did you consider using git submodules for a multi-repo?
[23:02] Q&A: Are you storing embeddings with Feast?
[24:30] Q&A: Regarding the mono-repo, which modules are responsible for feature engineering? How do you guarantee that different feature engineering can be used across many DS?
[27:58] Mike’s talk (Feature store at Skyscanner)
[28:08] Kaleidoscope System
[28:25] Background and context of the Feature store
[29:30] Initial state of the feature store
[30:13] How does the marketing team also leverage the feature store
[31:04] Current state of the feature store (marketing & machine learning)
[31:44] SDK approach of creating schemas with dataframes (easy access)
[32:16] Reusability across teams among the marketing and DS team
[33:06] GDPR constraints
[33:34] Data updates at the feature store
[36:09] Q&A: When a DS updates a feature, how are you communicating that across teams?
[38:25] Q&A: Are you applying different levels of feature engineering to increase the likelihood of a DS going back to a previous checkpoint of processing?
[40:55] Q&A: In what languages are you implementing the feature store?
[44:28] Q&A: Regarding performance-wise, how do you decide what code remains in Apache Spark vs SQL?
[49:00] Wrap-up

...more
View all episodesView all episodes
Download on the App Store

MLOps.communityBy Demetrios

  • 4.6
  • 4.6
  • 4.6
  • 4.6
  • 4.6

4.6

23 ratings


More shows like MLOps.community

View all
The a16z Show by Andreessen Horowitz

The a16z Show

1,093 Listeners

Software Engineering Daily by Software Engineering Daily

Software Engineering Daily

624 Listeners

Super Data Science: ML & AI Podcast with Jon Krohn by Jon Krohn

Super Data Science: ML & AI Podcast with Jon Krohn

302 Listeners

NVIDIA AI Podcast by NVIDIA

NVIDIA AI Podcast

343 Listeners

Data Engineering Podcast by Tobias Macey

Data Engineering Podcast

146 Listeners

Y Combinator Startup Podcast by Y Combinator

Y Combinator Startup Podcast

227 Listeners

Practical AI by Practical AI LLC

Practical AI

205 Listeners

Machine Learning Street Talk (MLST) by Machine Learning Street Talk (MLST)

Machine Learning Street Talk (MLST)

95 Listeners

Dwarkesh Podcast by Dwarkesh Patel

Dwarkesh Podcast

519 Listeners

No Priors: Artificial Intelligence | Technology | Startups by Conviction

No Priors: Artificial Intelligence | Technology | Startups

129 Listeners

This Day in AI Podcast by Michael Sharkey, Chris Sharkey

This Day in AI Podcast

228 Listeners

AI + a16z by a16z

AI + a16z

36 Listeners

Lightcone Podcast by Y Combinator

Lightcone Podcast

22 Listeners

Training Data by Sequoia Capital

Training Data

42 Listeners

The Pragmatic Engineer by Gergely Orosz

The Pragmatic Engineer

71 Listeners