Hello, my fair-weathered friends and readers! I am gone on vacation this week with my family, probably at this moment lying in the sand on a beach (Lord willing the creek don’t rise), not thinking of you all.

Anywho, be that as it may, I didn’t want you to miss my pretty face, so here is a video of me ranting about Apache Iceberg, something I’ve had a lot of practice doing and enjoy quite thoroughly.

For all you free-loaders out there, you can get 20% off to celebrate Memorial Day.

https://dataengineeringcentral.substack.com/Merica

This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit dataengineeringcentral.substack.com/subscribe

This is a free preview of a paid episode. To hear more, visit dataengineeringcentral.substack.com

It’s time for another episode of the Data Engineering Central Podcast. In this episode, we cover …

* Rust-based tool called UV to replace pip and poetry etc

* Apache X-Table and the Future of the Lake House

* How is AI going to affect you?

Thanks for being a consumer of Data Engineering Central; your support means a lot. Please share this podcast with your friend…

It’s time for another episode of the Data Engineering Central Podcast. In this episode, we cover …

* AWS Lambda + DuckDB and Delta Lake (Polars, Daft, etc).

* IAC - Long Live Terraform.

* Databricks Data Quality with DQX.

* Unity Catalog releases for DuckDB and Polars

* Bespoke vs Managed Data Platforms

* Delta Lake vs. Iceberg and UinFORM for a single table.

Thanks for b…

This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit dataengineeringcentral.substack.com/subscribe

In todays episode of Data Engineering Central Podcast we talk about a few hot topics, AWS S3 Tables, Databricks raising money, are Data Contracts Dead, and the Lake House Storage Format battle!

It's a good one, buckle up!

This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit dataengineeringcentral.substack.com/subscribe

It’s time for another episode of the Data Engineering Central Podcast. In this episode we cover …

* Apache Airflow vs Databricks Workflows

* End-of-Year Engineering Planning for 2025

* 10 Billion Row Challenge with DuckDB vs Daft vs Polars

* Raw Data Ingestion.

As usual, the full episode is available to paid subscribers, and a shortened version to you free loaders out there, don’t worry, I still love you though.

This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit dataengineeringcentral.substack.com/subscribe

It’s time for another episode of Data Engineering Central Podcast, our third one! Topics in this episode …

* Should you use DuckDB or Polars?

* Small Engineering Changes (PR Reviews)

* Daft vs Spark on Databricks with Unity Catalog (Delta Lake)

* Primary and Foreign keys in the Lake House

Enjoy!

This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit dataengineeringcentral.substack.com/subscribe

Welcome to the Data Engineering Central Podcast —— a no-holds-barred discussion on the Data Landscape.

Welcome to Episode 02

In today’s episode, we will talk about the following topics from the Data Engineering perspective …

* Using OpenAI’s o1 Model to do Data Engineering work

* Lord Save us from more ETL tools

* Rust for the small things

* Hosted (SaaS) vs Build

This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit dataengineeringcentral.substack.com/subscribe

Welcome to the Data Engineering Central Podcast —— a no-holds-barred discussion on the Data Landscape.

Welcome to Episode 01

In today’s episode we will talk about the following topics from the Data Engineering perspective …

* Snowflake vs Databricks.

* Is Apache Spark being replaced??

* Notebooks in Production. Bad.

This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit dataengineeringcentral.substack.com/subscribe