The Reasoning Show

Etsy's Big Data Cloud Migration


Listen Later

SHOW: 410

DESCRIPTION: Aaron and Brian talk with Kevin McHale (Senior Staff Engineer at Etsy) about the migration of their Big Data / Data Science platform from on-premises to Google Cloud, the business drivers for the migration, and the lessons the team has learned throughout the multi-year process.

SHOW SPONSOR LINKS:

  • Digital Ocean Homepage
  • Get Started Now and Get a free $50 Credit on Digital Ocean
  • Datadog Homepage - Modern Monitoring and Analytics
  • Try Datadog yourself by starting a free, 14-day trial today. Listeners of this podcast will also receive a free Datadog T-shirt
  • [FREE] Try an IT Pro Challenge
  • Get 20% off VelocityConf passes using discount code CLOUD

CLOUD NEWS OF THE WEEK:

  • GitHub Actions now includes CI/CD

SHOW INTERVIEW LINKS:

  • Etsy Homepage
  • Etsy’s Engineering Blog (“Code as Craft”)
  • Migrating Etsy to Google Cloud

SHOW NOTES:

Topic 1 - Welcome to the show. Before we dive into your work at Etsy, tell us a little bit about your background prior to Etsy - you’re pretty good at math. 

Topic 2 -  You’ve been working on two very interesting projects at Etsy - both building/evolving the data platform, and helping to manage the migration to Google Cloud Platform (GCP). Let’s start by talking about the Etsy data platform.

Topic 3 - What have been some of the business drivers that are pushing the data platform to collect more information, to become more cloud-native, and to better enable data pipelines? 

Topic 4 - At some point in 2017-18, Etsy decided to migrate some of the platforms to Google Cloud. Tell us about that decision-process, and how the migration has been going. What have been some of the lessons learned? 

Topic 5 -  How does working with Google Cloud (or just being in the public cloud) help accelerate the work that you’re doing on evolving the Etsy data platform? 

FEEDBACK?

  • Email: show at thecloudcast dot net
  • Twitter: @thecloudcastnet and @ServerlessCast

FEEDBACK?

  • Email: show @ reasoning dot show
  • Bluesky: @reasoningshow.bsky.social
  • Twitter/X: @ReasoningShow
  • Instagram: @reasoningshow
  • TikTok: @reasoningshow
...more
View all episodesView all episodes
Download on the App Store

The Reasoning ShowBy Massive Studios

  • 4.6
  • 4.6
  • 4.6
  • 4.6
  • 4.6

4.6

147 ratings


More shows like The Reasoning Show

View all
The Changelog: Software Development, Open Source by Changelog Media

The Changelog: Software Development, Open Source

288 Listeners

The a16z Show by Andreessen Horowitz

The a16z Show

1,105 Listeners

Software Engineering Daily by Software Engineering Daily

Software Engineering Daily

626 Listeners

Talk Python To Me by Michael Kennedy

Talk Python To Me

583 Listeners

Soft Skills Engineering by Jamison Dance and Dave Smith

Soft Skills Engineering

287 Listeners

Super Data Science: ML & AI Podcast with Jon Krohn by Jon Krohn

Super Data Science: ML & AI Podcast with Jon Krohn

306 Listeners

NVIDIA AI Podcast by NVIDIA

NVIDIA AI Podcast

343 Listeners

Tech Brew Ride Home by Morning Brew

Tech Brew Ride Home

964 Listeners

Practical AI by Practical AI LLC

Practical AI

212 Listeners

AWS Podcast by Amazon Web Services

AWS Podcast

204 Listeners

The Real Python Podcast by Real Python

The Real Python Podcast

140 Listeners

Big Technology Podcast by Alex Kantrowitz

Big Technology Podcast

512 Listeners

This Day in AI Podcast by Michael Sharkey, Chris Sharkey

This Day in AI Podcast

228 Listeners

AI + a16z by a16z

AI + a16z

34 Listeners

The Pragmatic Engineer by Gergely Orosz

The Pragmatic Engineer

77 Listeners