Contributor

Prefect with Jeremiah Lowin

Listen Later

Eric Anderson (@ericmander) and Jeremiah Lowin (@jlowin) discuss Prefect, a workflow management system and data orchestration tool under development as an open-source project. Jeremiah initially created Prefect to solve a technical challenge specific to his own work, but soon realized that it was appealing to a very wide range of different clients. Listen to today’s episode to learn why Jeremiah believes most attempts to build a unified framework for solving data orchestration fail.

In this episode we discuss:

  • Solving the “negative engineering problem”
  • Learning from the complaints of data engineers at Apache Airflow
  • The difficulty of having a product that serves two masters
  • How COVID changed the direction of Prefect

    • Links:

    • Prefect
    • Apache Airflow
    • Why Not Airflow?

      • People mentioned:

      • Jim O'Shaughnessy (@jposhaughnessy)
      • Patrick O’Shaughnessy (@patrick_oshag)
        • ...more
        View all episodesView all episodes
        Download on the App Store
        ContributorBy Eric Anderson
        • 5
        • 5
        • 5
        • 5
        • 5

        5

        11 ratings

        More shows like Contributor

        View all
        Planet Money by NPR

        Planet Money

        30,823 Listeners

        The Daily by The New York Times

        The Daily

        111,761 Listeners

        This Week in NoCode + AI by jj englert

        This Week in NoCode + AI

        4 Listeners