Data With Direction

Send us a text

Is your team spending time repairing broken data pipelines?

In this live session, I’ll walk you through 5 of the most common—and costly—mistakes I’ve seen in 25+ years of data engineering. From schema drift disasters to overengineered tech stacks, I’ll break down real-world examples and teach you how to build smarter, cleaner, and more reliable data systems.

✅ What you’ll learn:

How to future-proof against schema changes

Why simple tools often outperform complex ones

Where data quality goes to die—and how to fix it

How to avoid secrets and paths hardcoded in shame

What maintainable pipelines actually look like

Whether you’re a junior engineer or a team lead, these lessons will save you time, money, and frustration.

💬 Have a mistake to share? Drop it in the chat... I’ll respond personally!

🔗 Need help fixing your stack? 
Reach out: https://www.gambilldataengineering.com/consultation-request

#DataEngineering #ETL #DataPipelines #SQL #CloudData #Snowflake #Databricks #DataOps

Chris Gambill is a data engineering consultant and educator with 25+ years of experience helping organizations modernize their data stacks. As founder of Gambill Data, he specializes in data strategy, cloud migration, and building resilient analytics platforms for mid-market and enterprise clients. He’s passionate about making real-world data engineering accessible.

Connect with Chris on LinkedIn or learn more at gambilldata.com.

Send us a text

Futureproof your career and master the idea of T-Skills! As Data Engineers we have a choice to make right now, given the new AI world:

Go deeper into their Engineering skills and become the 5% who still apply their craft to solve the very complex data use cases.

Go broad and increase their other skills, because (per Shane) 95% of the Data Engineering work will be democratized for the common data use cases.

Plus a few bonus insights, and revisiting old data lore as well.

Support the show

Chris Gambill is a data engineering consultant and educator with 25+ years of experience helping organizations modernize their data stacks. As founder of Gambill Data, he specializes in data strategy, cloud migration, and building resilient analytics platforms for mid-market and enterprise clients. He’s passionate about making real-world data engineering accessible.

Connect with Chris on LinkedIn or learn more at gambilldata.com.

Send us a text

Data engineering in 2025 isn't about cool tools but about survival, as most current data engineers risk becoming replaceable commodities if they can't demonstrate real-world problem-solving skills and business understanding. Successful data engineers need to prove they've overcome significant challenges and understand the business impact of their work, not just technical implementation.

• Basic skills like Python, cloud certifications and ETL pipeline building are becoming commoditized
• AI and offshore contractors will replace engineers who only move data from point A to B
• Real data engineers can discuss their failures and what they learned from them
• Employers value candidates who can handle crises with business users breathing down their necks
• Understanding the cost implications of pipeline failures is crucial but rarely discussed
• The best engineers aim to automate themselves out of repetitive tasks
• Building something that breaks on purpose can help develop critical troubleshooting skills
• Documenting mistakes through detailed post-mortems demonstrates accountability and growth

Share your worst data blunder in the comments - what it cost you and your company. I'll highlight the most honest stories in a future episode, and might invite those with the most cringeworthy failures to join me for coffee on my podcast.

Support the show

Chris Gambill is a data engineering consultant and educator with 25+ years of experience helping organizations modernize their data stacks. As founder of Gambill Data, he specializes in data strategy, cloud migration, and building resilient analytics platforms for mid-market and enterprise clients. He’s passionate about making real-world data engineering accessible.

Connect with Chris on LinkedIn or learn more at gambilldata.com.

Send us a text

Are you thinking about jumping into DataOps Or just curious what all the hype is about

In this episode, Chris dives deep into why DataOps is not all sunshine and rainbows. From the real challenges of the job to the toolbox every DataOps engineer relies on, you’ll get the honest truth... and a few laughs along the way.

Timestamps
00:00 – Why NOT to Be a DataOps Engineer!
01:00 – What is DataOps (Really)
02:00 – The Hard Truths It’s Not All Automation
03:30 – The Tools Behind DataOps
05:00 – Why Some People Love It Anyway
06:00 – Q&A Your DataOps Stories and Questions

Mentioned tools
Airflow, dbt, Great Expectations, Orchestra, Terraform, Prefect, Monte Carlo, DataKitchen, Dagster, YAML, Git, Bash, Python, and more.

Did this video help Have a DataOps war story or a burning question Drop it in the comments! If you want a checklist to see if you’re ready for DataOps, grab the download below!

#DataOps #DataEngineering #DevOps #Airflow #dbt #Orchestra #Terraform #CareerAdvice #RealityCheck #DataKitchen #GreatExpectations #MonteCarlo #Prefect

Support the show

Chris Gambill is a data engineering consultant and educator with 25+ years of experience helping organizations modernize their data stacks. As founder of Gambill Data, he specializes in data strategy, cloud migration, and building resilient analytics platforms for mid-market and enterprise clients. He’s passionate about making real-world data engineering accessible.

Connect with Chris on LinkedIn or learn more at gambilldata.com.

Send us a text

What is the modern data stack—and how do tools like Spark, dbt, and Airflow actually work together? In this lesson, I’ll break it down step by step with real-world insights.

In Lesson 2 of the Data Engineering 101 series, we take a deep dive into the Modern Data Stack—what it is, how the components work together, and why data modeling is still the backbone of every great pipeline.

💡 You’ll learn how tools like Spark, dbt, Pandas, and Airflow function like machines in a factory—transforming raw logs and cloud-stored data into meaningful business insight.

We’ll also talk about:

Databases vs. data warehouses

Cloud storage as digital basements

Why orchestration matters

When Excel is actually OK

And how to start mapping your own data stack

📦 This episode includes visuals, analogies, and action steps you can apply right now—whether you're building from scratch or modernizing legacy systems.

📚 Watch the full Data Engineering 101 Playlist:
👉 https://www.youtube.com/playlist?list=PLewT1HTMY0WZ4tpoBw-w_CcewwrEDNmsU

🚨 Don’t miss Lesson 3: SQL Fundamentals (coming next week!)

—
Looking to get more technical practice? Join Codecademy while supporting the channel! 
If you are a high school or college student you get 35% off with this link:
https://www.pntrs.com/t/2-591852-361892-213588
Not a student, but want a special discount, get 15% off the normal prices with this special link from Gambill Data: https://www.pntrac.com/t/2-523372-361892-213588

🔔 Subscribe for more real-world data engineering strategies, tools, and career advice.
➕ Follow me on LinkedIn: https://www.linkedin.com/in/databasemanagement/
💻 Check out consulting & mentoring resources:  https://www.gambilldataengineering.com
—
Chapters: 
00:00 Intro – Why the Cloud Changed Everything  
00:39 What We’ll Cover in This Lesson  
00:58 What Is the Modern Data Stack?  
01:34 Databases vs. Data Warehouses  
02:18 Cloud Storage = Digital Basements  
03:02 Data Processing Tools (Spark, dbt, Pandas)  
03:24 Airflow: The Data Factory Supervisor  
05:10 Cloud Platforms: Azure, AWS, GCP   
06:03 Modeling Is Like Organizing a Library  
07:06 Relational, Dimensional, and NoSQL Models  
09:14 When Excel Is OK!  
10:10 Key Takeaways  
11:20 Coming Up Next: SQL Fundamentals  

#DataEngineering #ModernDataStack #ApacheSpark #dbt #Airflow #DataModeling #ETL #CloudData #SQL #Azure #AWS #GCP

Support the show

Chris Gambill is a data engineering consultant and educator with 25+ years of experience helping organizations modernize their data stacks. As founder of Gambill Data, he specializes in data strategy, cloud migration, and building resilient analytics platforms for mid-market and enterprise clients. He’s passionate about making real-world data engineering accessible.

Connect with Chris on LinkedIn or learn more at gambilldata.com.

Send us a text

🚨 Major News in Data Engineering! 🚨

Today, two industry-shaking headlines dropped:
1️⃣ Salesforce announces its $8B acquisition of Informatica.
2️⃣ DuckDB releases DuckLake, their open-source lakehouse format!

Join me LIVE as I break down what these moves mean for data engineers, the enterprise stack, and the future of open-source analytics. Bring your questions and hot takes!

🔗 **News Links:**
- DuckLake Announcement: https://duckdb.org/2025/05/27/ducklake.html
- DuckLake Website: https://ducklake.select/
- Salesforce x Informatica: https://techcrunch.com/2025/05/27/salesforce-acquires-informatica-for-8-billion/

👇 Drop your thoughts in the chat: Are you excited about DuckLake? Concerned about consolidation in the data space? Let’s talk!

Join us live next Friday (6/6) at 10pm ET when we talk to  @the-data-coach !
Then on June 13th at 10pm ET when we chat with  @agiledata 's 

#DataEngineering #BreakingNews #DuckDB #Salesforce #Informatica #Lakehouse #CloudData #AI

👋 Subscribe for more real-world data engineering news, tutorials, and live breakdowns!

🔗 Connect on LinkedIn: https://www.linkedin.com/in/databasemanagement/

#Salesforce x #informatica  Acquisition
#techcrunch  – Salesforce acquires Informatica for $8B
https://techcrunch.com/2025/05/27/salesforce-acquires-informatica-for-8-billion/

Salesforce Investor Relations Press Release
https://investor.salesforce.com/news/news-details/2025/Salesforce-Signs-Definitive-Agreement-to-Acquire-Informatica/default.aspx

Informatica: #1 in Five Markets (IDC market share blog)
https://www.informatica.com/blogs/informatica-ranked-1-in-five-markets-in-new-idc-data-integration-and-intelligence-software-market-shares-report.html

PeerSpot Informatica PowerCenter Reviews
https://www.peerspot.com/products/informatica-powercenter-reviews

LinkedIn – Informatica Salesforce acquisition comments
https://www.linkedin.com/search/results/content/?keywords=Salesforce%20Informatica%20acquisition&origin=GLOBAL_SEARCH_HEADER

#DuckDB & #DuckLake
DuckLake Official Announcement
https://duckdb.org/2025/05/27/ducklake.html

DuckLake Interactive Demo
https://ducklake.select/

DuckDB Wikipedia
https://en.wikipedia.org/wiki/DuckDB

#dbt Labs Blog – DuckDB’s Rise
https://www.getdbt.com/blog/duckdb-modern-analytics/

PeerSpot DuckDB Community Reviews
https://www.peerspot.com/products/duckdb-reviews

Support the show

Chris Gambill is a data engineering consultant and educator with 25+ years of experience helping organizations modernize their data stacks. As founder of Gambill Data, he specializes in data strategy, cloud migration, and building resilient analytics platforms for mid-market and enterprise clients. He’s passionate about making real-world data engineering accessible.

Connect with Chris on LinkedIn or learn more at gambilldata.com.

Send us a text

Behind the Scenes: How I Work Full-Time & Still Help Clients

Support the show

Chris Gambill is a data engineering consultant and educator with 25+ years of experience helping organizations modernize their data stacks. As founder of Gambill Data, he specializes in data strategy, cloud migration, and building resilient analytics platforms for mid-market and enterprise clients. He’s passionate about making real-world data engineering accessible.

Connect with Chris on LinkedIn or learn more at gambilldata.com.

Send us a text

Are hidden problems lurking in your data pipelines?

In both medicine and data engineering, “silent killers” can do major damage... quietly, with no warning, until it’s almost too late. In this live session, Chris Gambill breaks down the three hidden threats every data engineering team needs to catch before disaster strikes.

You’ll learn:

How technical debt is like high blood pressure (and how to lower it before a meltdown)

How invisible data drift is the “carbon monoxide” of analytics

Why organizational issues can quietly destroy projects like the most notorious silent cancers

Real-world stories, prevention tactics, and easy-to-remember frameworks for protecting your data health

🔴 Join live, share your war stories, and learn the habits that separate resilient teams from those heading for disaster.

#DataEngineering #DataQuality #LiveStream #TechTalk #SilentKillers

Support the show

Chris Gambill is a data engineering consultant and educator with 25+ years of experience helping organizations modernize their data stacks. As founder of Gambill Data, he specializes in data strategy, cloud migration, and building resilient analytics platforms for mid-market and enterprise clients. He’s passionate about making real-world data engineering accessible.

Connect with Chris on LinkedIn or learn more at gambilldata.com.