Building the Backend: Data Solutions that Power Leading Organizations

Optimizing Spark in the Cloud - with Jean-Yves Stephan

Listen Later

This episode features Jean-Yves Stephan Co-Founder & CEO @ Data Mechanics (recently Acq. by Spot by NetApp), during our discussion we talk about optimizing Spark to run in the cloud at a low cost.

Top 3 Value Bombs:

  • Running Spark CAN be expensive but there are ways to reduce your current operating costs by 50-75% by smart automations (i.e.  tune for node type, memory and CPU). 
  • Spot instances can lower your costs by utilizing unused instances. 
  • Creating serverless architectures and using containers will allow for more flexibility with deployment models and scalability. 



...more
View all episodesView all episodes
Download on the App Store
Building the Backend: Data Solutions that Power Leading OrganizationsBy Travis Lawrence
  • 4.7
  • 4.7
  • 4.7
  • 4.7
  • 4.7

4.7

13 ratings

More shows like Building the Backend: Data Solutions that Power Leading Organizations

View all
Data Engineering Podcast by Tobias Macey

Data Engineering Podcast

139 Listeners