The Data Stack Show

158: The Orchestration Layer as the Data Platform Control Plane With Nick Schrock of Dagster Labs


Listen Later

Highlights from this week’s conversation include:

  • Nick’s background and journey in data (2:28)
  • Founding Dagster Labs (7:50)
  • The evolution of data engineering (12:32)
  • Fragmentation in data infrastructure (15:04)
  • The role of orchestration in data platforms (19:53)
  • The importance of operational tools for data pipelines (25:01)
  • Lessons learned from working with GraphQL (26:19)
  • The role of the orchestrator in data engineering (34:51)
  • The boundaries between data infrastructure and product engineering (37:33)
  • Different orchestrators in the data infrastructure landscape(42:03)
  • The role of MLOps in data engineering (46:04)
  • Data Quality and Orchestration (51:04)
  • Future of Data Teams and Orchestration (54:27)
  • Final thoughts and takeaways from (58:01)

The Data Stack Show is a weekly podcast powered by RudderStack, the CDP for developers. Each week we’ll talk to data engineers, analysts, and data scientists about their experience around building and maintaining data infrastructure, delivering data and data products, and driving better outcomes across their businesses with data.

RudderStack helps businesses make the most out of their customer data while ensuring data privacy and security. To learn more about RudderStack visit rudderstack.com.

...more
View all episodesView all episodes
Download on the App Store

The Data Stack ShowBy Rudderstack

  • 5
  • 5
  • 5
  • 5
  • 5

5

12 ratings


More shows like The Data Stack Show

View all
a16z Podcast by Andreessen Horowitz

a16z Podcast

1,001 Listeners

The Twenty Minute VC (20VC): Venture Capital | Startup Funding | The Pitch by Harry Stebbings

The Twenty Minute VC (20VC): Venture Capital | Startup Funding | The Pitch

510 Listeners

Talk Python To Me by Michael Kennedy

Talk Python To Me

586 Listeners

Software Engineering Daily by Software Engineering Daily

Software Engineering Daily

628 Listeners

Odd Lots by Bloomberg

Odd Lots

1,773 Listeners

Super Data Science: ML & AI Podcast with Jon Krohn by Jon Krohn

Super Data Science: ML & AI Podcast with Jon Krohn

296 Listeners

NVIDIA AI Podcast by NVIDIA

NVIDIA AI Podcast

324 Listeners

Data Engineering Podcast by Tobias Macey

Data Engineering Podcast

140 Listeners

DataFramed by DataCamp

DataFramed

269 Listeners

Practical AI by Practical AI LLC

Practical AI

190 Listeners

Switched On by Bloomberg

Switched On

98 Listeners

Dwarkesh Podcast by Dwarkesh Patel

Dwarkesh Podcast

354 Listeners

No Priors: Artificial Intelligence | Technology | Startups by Conviction

No Priors: Artificial Intelligence | Technology | Startups

125 Listeners

BG2Pod with Brad Gerstner and Bill Gurley by BG2Pod

BG2Pod with Brad Gerstner and Bill Gurley

433 Listeners

Money Stuff: The Podcast by Bloomberg

Money Stuff: The Podcast

373 Listeners