Hello, welcome to The New Stack Context, a podcast where we review the week’s hottest news in cloud-native technologies/at-scale application development and look ahead to topics we expect will gain more attention in coming weeks.

For this Context, we talk with two data scientists from Pivotal, who write this week about the pros and cons of using Apache Spark for running data science workloads at scale. Apache Spark is an in-memory data analytics engine that is wildly popular with data scientists because of its speed, scalability and ease-of-use. Plus, it happens to be an ideal workload to run on Kubernetes, the Pivotal team writes.

We spoke with them about the pros and cons of Apache Spark, what data science workloads they’ve been experimenting with lately at Pivotal, and how application architectures are evolving, in general, to better support data storage and processing at scale.

Then, later in the show, we'll go over the highlights from his day at QCon developer conference in New York, including microservices debugging and how the Envoy service mesh is preparing to work with Kafka.

TNS editor-in-chief Alex Williams and TNS managing editor Joab Jackson co-hosted this episode of Context.