The Digital Executive

Revolutionizing Data Engineering: Dagster’s Journey and Open-Source Impact with CTO Nick Schrock | Ep 789


Listen Later

Send us a text

In this enlightening episode of The Digital Executive Podcast, host Brian Thomas welcomes Nick Schrock, the innovative mind behind Dagster Labs. Nick delves into his impressive journey from a principal engineer and director at Facebook to founding Dagster Labs, driven by his desire to solve engineering pain points. He explains how his past experiences, including co-creating GraphQL, shaped his approach to addressing complex data infrastructure challenges.


Dagster, a prominent open-source data orchestration platform, is at the heart of their discussion. Nick elaborates on how Dagster redefines data engineering by prioritizing asset-oriented workflows over task-focused ones, ensuring more robust and intuitive data management. He emphasizes the platform's unique approach to consolidating tools, improving lineage tracking, and enhancing operational context, all contributing to a more streamlined and effective data engineering process.

Furthermore, Nick highlights the significant role of open-source projects in shaping the tech industry, particularly in legacy companies. He shares insights into how these projects foster community building and empower engineers to adopt new technologies more freely.

The podcast concludes with Nick's perspective on the future of data and ML engineering. He anticipates a shift towards greater accuracy and productivity, with Dagster strategically positioned to address these evolving needs by integrating software engineering principles into data engineering.

Overall, the episode offers a deep dive into the transformative impact of Dagster in the data engineering landscape and the broader implications for the tech industry.

...more
View all episodesView all episodes
Download on the App Store

The Digital ExecutiveBy Coruzant Technologies

  • 5
  • 5
  • 5
  • 5
  • 5

5

14 ratings


More shows like The Digital Executive

View all
Mad Money w/ Jim Cramer by CNBC

Mad Money w/ Jim Cramer

4,074 Listeners

Freakonomics Radio by Freakonomics Radio + Stitcher

Freakonomics Radio

32,112 Listeners

Pivot by New York Magazine

Pivot

9,248 Listeners

The Daily by The New York Times

The Daily

110,617 Listeners

All-In with Chamath, Jason, Sacks & Friedberg by All-In Podcast, LLC

All-In with Chamath, Jason, Sacks & Friedberg

9,263 Listeners

Hard Fork by The New York Times

Hard Fork

5,463 Listeners

The Ezra Klein Show by New York Times Opinion

The Ezra Klein Show

15,405 Listeners

Search Engine by PJ Vogt

Search Engine

4,314 Listeners

The AI Daily Brief (Formerly The AI Breakdown): Artificial Intelligence News and Analysis by Nathaniel Whittemore

The AI Daily Brief (Formerly The AI Breakdown): Artificial Intelligence News and Analysis

508 Listeners

Impact with Eddie Wilson by Eddie Wilson

Impact with Eddie Wilson

16 Listeners

Devil in the Desert by ABC News

Devil in the Desert

1,823 Listeners