When it comes to data management, have we come a long way since the early 2000s? Or has it simply taken us 20 years to finally realize that you can’t scale properly without data modeling. With over 20 years of experience in the data space, leading engineering teams at Cisco, Oracle, Greenplum, and now as Sr. Director of Engineering at BlackRock, Krishnan Viswanathan talks about the data engineering challenges that existed two decades ago and still exist today.

• Zach Wilson on What Makes a Great Data Engineer
• Joe Reis and Matt Housley on The Fundamentals of Data Engineering
• Bill Inmon, The Godfather of Data Warehousing

How good you are at Spark or Flink ≠ how good you are at data engineering. After years of data engineering experience at Airbnb, Netflix, and Facebook, Zach Wilson is now focused on spreading the knowledge in EcZachly and all over social media. He met Benjamin Wagner to explain why data modeling and storytelling are more important than the actual tech, why data engineering is going to see more job growth than data science, and what brought him to start creating content, reaching over 250K followers on LinkedIn.

• Zach Wilson on What Makes a Great Data Engineer
• Joe Reis and Matt Housley on The Fundamentals of Data Engineering
• Bill Inmon, The Godfather of Data Warehousing

Data engineers are not paid to do support. Liran Yogev, Director of Engineering at ZipRecruiter, and Doron Porat, Director of Infrastructure at Yotpo talk about building resilient self-service products that keep customers happy and engineers calm. 

They walked the bros through their data stacks and explained how ZipRecruiter is completely rebuilding its data layer from scratch.

• Zach Wilson on What Makes a Great Data Engineer
• Joe Reis and Matt Housley on The Fundamentals of Data Engineering
• Bill Inmon, The Godfather of Data Warehousing

Barr Moses, CEO of Monte Carlo explains the difference between data quality and data observability, and how to make sure your data is accurate in a world where so many different teams are accessing it.

• Zach Wilson on What Makes a Great Data Engineer
• Joe Reis and Matt Housley on The Fundamentals of Data Engineering
• Bill Inmon, The Godfather of Data Warehousing

Weichen Wang, Senior Engineering Manager at Amplitude, came to meet the bros to talk about Amplitude's cutting-edge data stack and how it processes 5 Trillion real-time events while dealing with mutable data and massive scale.

• Zach Wilson on What Makes a Great Data Engineer
• Joe Reis and Matt Housley on The Fundamentals of Data Engineering
• Bill Inmon, The Godfather of Data Warehousing

• Zach Wilson on What Makes a Great Data Engineer
• Joe Reis and Matt Housley on The Fundamentals of Data Engineering
• Bill Inmon, The Godfather of Data Warehousing

Sudeep Kumar, Principal Engineer at Salesforce is a ClickHouse fan. He considers the shift to Clickhouse as one of his biggest accomplishments during his eBay days and walks Boaz through his experience with the platform. How on one hand it handled 2B events per minute, but also how it required rollups which compromised granularity when extending time windows. 

Besides a ClickHouse review from a practitioner’s point of view, Sudeep tells us about interesting use-cases he’s working on at Salesforce. 

• Zach Wilson on What Makes a Great Data Engineer
• Joe Reis and Matt Housley on The Fundamentals of Data Engineering
• Bill Inmon, The Godfather of Data Warehousing

According to Maxime Beauchemin, CEO & Founder at Preset and Creator of Apache Superset and Apache Airflow, it's not so straight-forward to understand what you're really getting into and the vastness of the skills that are required in order to build a thriving company.

Picking the right system and services is key for a successful start, and can help you avoid the chaos of having too many tools spread across multiple teams.

Plus, Max walks the bros through the genesis of Airflow, Superset & Presto, and Airflow's old school marketing approach that won the hearts of developers across the world. And just like the terminator, once the machine takes over, you can't stop.

• Zach Wilson on What Makes a Great Data Engineer
• Joe Reis and Matt Housley on The Fundamentals of Data Engineering
• Bill Inmon, The Godfather of Data Warehousing

According to Yoav Shmaria, VP R&D Platform at Similarweb, the best way to manage data warehouse costs is to tag every table, database or ETL running to have good granularity over every feature.  

Besides handy cost management tips, Yoav walks the bros through the tech stack he implemented to analyze 100s of TBs of web data to serve fast customer-facing analytics.

Full disclosure, Similarweb is a Firebolt customer, but the bros kept it objective, and there’s no Firebolt talk in this episode.

According to Yoav Shmaria, VP R&D Platform at Similarweb, the best way to manage data warehouse costs is to tag every table, database or ETL running to have good granularity over every feature.  

Besides handy cost management tips, Yoav walks the bros through the tech stack he implemented to analyze 100s of TBs of web data to serve fast customer-facing analytics.

Full disclosure, Similarweb is a Firebolt customer, but the bros kept it objective, and there’s no Firebolt talk in this episode.

• Zach Wilson on What Makes a Great Data Engineer
• Joe Reis and Matt Housley on The Fundamentals of Data Engineering
• Bill Inmon, The Godfather of Data Warehousing