Spark of The Future with DataBricks

Spark is one of the most popular platforms for Data Science and AI development today and the team at DataBricks is driving massive innovation in this community, both open-source and commercially.

Cory (https://twitter.com/Cory_Minton?ref_src=twsrc%5Egoogle%7Ctwcamp%5Eserp%7Ctwgr%5Eauthor) sat down with Reynold Xin, Co-Founder and Chief Architect, and Michael Armburst (https://www.linkedin.com/in/michaelarmbrust/), Principal Architect, at DataBricks to understand the history of Spark development and uncover the massive developments in the platform available now and in the near future making Spark more accessible and easier for Data Scientists and Engineers alike.

This podcast was recorded at Spark+AI Summit in San Francisco (https://databricks.com/sparkaisummit/north-america), California in April 2019.

Links

DataBricks (https://databricks.com/)

Reynold (https://databricks.com/speaker/reynold-xin) on LinkedIn (https://www.linkedin.com/in/reynoldxin/) & Twitter (https://twitter.com/rxin)

Michael (https://databricks.com/speaker/michael-armbrust) on LinkedIn (https://www.linkedin.com/in/michaelarmbrust/) & Twitter (https://twitter.com/michaelarmbrust)

DeltaLake (https://delta.io/) and Announcement (https://databricks.com/session/announcing-databricks-delta)

Apple Presentation on DeltaLake (https://databricks.com/session/keynote-from-apple)

MLFlow (https://mlflow.org/)

Videos from Spark+AI Summit 2019 (https://databricks.com/sparkaisummit/north-america/sessions)

Reynold Keynote Recording (https://databricks.com/session/official-announcement-of-koalas-open-source-project) on Koalas

Michael Keynote Recording (https://databricks.com/session/announcing-databricks-delta) on Delta

Did you know we are on Spotify?  Tune in to The Big Data Beard! (https://open.spotify.com/show/62li7mm5ADkeXUS3tuOZ24)

Don’t forget that we are going to Domino Data Labs’ upcoming REV2 Conference in NYC May 23-24.  Visit the REV Conference Website (http://REV.dominodatalab.com) and use promo code BDB_REV100 to save $100 on your pass!

Music from this episode is by Andrew Belle (https://itunes.apple.com/us/artist/andrew-belle/37712246).  Please go check him out...you'll thank us!