Contributor

Prefect with Jeremiah Lowin

Listen Later

Eric Anderson (@ericmander) and Jeremiah Lowin (@jlowin) discuss Prefect, a workflow management system and data orchestration tool under development as an open-source project. Jeremiah initially created Prefect to solve a technical challenge specific to his own work, but soon realized that it was appealing to a very wide range of different clients. Listen to today’s episode to learn why Jeremiah believes most attempts to build a unified framework for solving data orchestration fail.

In this episode we discuss:

  • Solving the “negative engineering problem”
  • Learning from the complaints of data engineers at Apache Airflow
  • The difficulty of having a product that serves two masters
  • How COVID changed the direction of Prefect

    • Links:

    • Prefect
    • Apache Airflow
    • Why Not Airflow?

      • People mentioned:

      • Jim O'Shaughnessy (@jposhaughnessy)
      • Patrick O’Shaughnessy (@patrick_oshag)
        • ...more
        View all episodesView all episodes
        Download on the App Store
        ContributorBy Eric Anderson
        • 5
        • 5
        • 5
        • 5
        • 5

        5

        11 ratings

        More shows like Contributor

        View all
        Planet Money by NPR

        Planet Money

        30,680 Listeners

        The Daily by The New York Times

        The Daily

        112,277 Listeners

        This Week in AI by JJ Englert

        This Week in AI

        4 Listeners