Fabric Architecture Podcast

Apache Airflow in Fabric: When Code-First Orchestration Earns Its Keep

Listen Later

Apache Airflow in Fabric: When Code-First Orchestration Earns Its Keep

Episode 13 • 2026-03-27

Duration: 8:19

When should you reach for Apache Airflow Job instead of Data Pipelines in Microsoft Fabric? Matthias and Fabia break down pool types, cost traps, dbt fragility, and the architectural threshold where visual orchestration breaks down.

What we discuss

  • A real-world mistake from a pre-Fabric era
  • The one question that reframes the architectural debate
  • How we got here — predecessor products and evolution
  • Why the "obvious" answer is often wrong
  • A real Reddit/Microsoft Q&A question unpacked
  • The concrete recommended architecture
  • F-SKU realism — what this actually costs
  • When the rejected approach is actually right
  • Risks of the recommended path
  • What Microsoft is shipping that changes the calculus
  • The architectural principle to take home

    • Key takeaways

    • I'm stealing that. But yeah — match the orchestrator to the orchestration complexity. If your workflow fits on a visual canvas, keep it there. Airflow is for when Python is genuinely the clearest way to express your pipeline logic.
    • For maybe... sixty, seventy percent of Fabric workloads, Data Pipeline is the right answer. Airflow earns its spot when you need cross-cloud orchestration, dbt models, complex dependency graphs, or your team already thinks in Python DAGs....
    • You just paid for a permanent orchestra conductor to wave a baton at six musicians who already know the song.

      • Resources

      • What is Apache Airflow Job?
      • Run a Fabric item using Apache Airflow DAGs
      • Apache Airflow compute in Fabric
      • Apache Airflow job pricing
      • Transform data using dbt
      • Migrate to Apache Airflow job in Microsoft Fabric
      • CI/CD for Apache Airflow in Fabric
      • Apache Airflow Job region availability
      • Quickstart: Create an Apache Airflow Job
      • Apache Airflow Job workspace settings
      • Access Apache Airflow Job Logs
      • Sync a GitHub repository in Apache Airflow Job
      • Run Hello-world DAG in Apache Airflow Job
      • Fabric region availability
      • MS Learn

        • About the show

        Built on ElevenLabs voice synthesis. Matthias — cloned voice. Fabia — designed AI co-host. See Matthias live on YouTube (Fabric Friday), at his meetups, and at conferences like FabCon.

        Hosted by Matthias Falland — Microsoft Data Platform MVP and community architect behind the Fabric Periodic Table. New episodes every Friday.

        Submit your case

        Have an architecture decision you are wrestling with? DM Matthias on LinkedIn — find him as Matthias Falland. Three to five sentences about the decision, your team size, and your current stack. We anonymize before airing.

        Built on ElevenLabs voice synthesis. Brand design based on fabricperiodictable.com.

        ...more
        View all episodesView all episodes
        Download on the App Store
        Fabric Architecture PodcastBy Matthias Falland