The Data Flowcast: Mastering Apache Airflow ® for Data Engineering and AI

Uphold’s Approach to Orchestrating Modern Data Workflows with Jaime Oliveira


Listen Later

A strong data-driven mindset underpins how fintech teams scale analytics, infrastructure and decision-making across the business.


In this episode, Jaime Oliveira, Lead Data Engineer at Uphold, joins us to discuss how Uphold structures its data organization and orchestration strategy. Jaime shares how the team uses Airflow and dbt to support analytics, reporting and data activation while evolving their approach as the stack grows.


Key Takeaways:


00:00 Introduction.

01:23 A data-driven mindset supports product development and business decisions.

02:55 Diverse ingestion pipelines enable scalable analytics.

04:18 A single orchestration platform simplifies analytics workflows.

05:17 Early experience with orchestration tools shapes engineering practices.

08:16 Analytics orchestration works best when aligned with transformation workflows.

09:25 Infrastructure choices involve tradeoffs in testing, visibility and overhead.

16:39 More collaborative workflow tools could improve accessibility and autonomy.


Resources Mentioned:


Jaime Oliveira

https://www.linkedin.com/in/jaime-oliveira-b075855a/


Uphold | LinkedIn

https://www.linkedin.com/company/upholdinc/


Uphold | Website

https://uphold.com


Apache Airflow

https://airflow.apache.org


dbt

https://www.getdbt.com


Snowflake

https://www.snowflake.com


Kubernetes

https://kubernetes.io


Astronomer Cosmos

https://astronomer.github.io/astronomer-cosmos


Cosmos e-book

https://www.astronomer.io/ebooks/orchestrating-dbt-with-airflow-using-cosmos/




Thanks for listening to “The Data Flowcast: Mastering Apache Airflow® for Data Engineering and AI.” If you enjoyed this episode, please leave a 5-star review to help get the word out about the show. And be sure to subscribe so you never miss any of the insightful conversations.



#AI #Automation #Airflow

...more
View all episodesView all episodes
Download on the App Store

The Data Flowcast: Mastering Apache Airflow ® for Data Engineering and AIBy Astronomer

  • 5
  • 5
  • 5
  • 5
  • 5

5

20 ratings


More shows like The Data Flowcast: Mastering Apache Airflow ® for Data Engineering and AI

View all
Freakonomics Radio by Freakonomics Radio + Stitcher

Freakonomics Radio

32,258 Listeners

The Joe Rogan Experience by Joe Rogan

The Joe Rogan Experience

229,660 Listeners

The Twenty Minute VC (20VC): Venture Capital | Startup Funding | The Pitch by Harry Stebbings

The Twenty Minute VC (20VC): Venture Capital | Startup Funding | The Pitch

539 Listeners

Software Engineering Daily by Software Engineering Daily

Software Engineering Daily

627 Listeners

Data Engineering Podcast by Tobias Macey

Data Engineering Podcast

146 Listeners

Masters of Scale by WaitWhat

Masters of Scale

3,987 Listeners

DevOps Paradox by Darin Pope & Viktor Farcic

DevOps Paradox

25 Listeners

The Real Python Podcast by Real Python

The Real Python Podcast

140 Listeners

All-In with Chamath, Jason, Sacks & Friedberg by All-In Podcast, LLC

All-In with Chamath, Jason, Sacks & Friedberg

10,208 Listeners

SmartLess by Jason Bateman, Sean Hayes, Will Arnett

SmartLess

58,521 Listeners

Hard Fork by The New York Times

Hard Fork

5,556 Listeners

The Data Stack Show by Rudderstack

The Data Stack Show

13 Listeners

The Data Engineering Show by The Firebolt Data Bros

The Data Engineering Show

8 Listeners

The AWS Developers Podcast by Amazon Web Services

The AWS Developers Podcast

26 Listeners

The Economics Show by Financial Times

The Economics Show

149 Listeners