There's no shortage of technical content for data engineers, but a massive gap exists when it comes to the non-technical skills required to advance beyond a senior role. I sit down with Yordan Ivanov, Head of Data Engineering and writer of "Data Gibberish," to talk about this disconnect.

We dive into his personal journey of failing as a manager the first time, learning the crucial "people" skills, and his current mission to help data engineers learn how to speak the language of business.

Key areas we explore:

• The Senior-Level Content Gap: Yordan explains why his non-technical content on career strategy and stakeholder communication gets "terrible" engagement compared to technical posts, even though it's what's needed to advance.
• The Managerial Trap: Yordan's candid story about his first attempt at management, where he failed because he cared only about code and wasn't equipped for the people-centric aspects and politics of the role.
• The Danger of AI Over-reliance: A deep discussion on how leaning too heavily on AI can prevent the development of fundamental thinking and problem-solving skills, both in coding and in life.
• The Maturing Data Landscape: We reflect on the end of the "modern data stack euphoria" and what the wave of acquisitions means for innovation and the future of data tooling.
• AI Adoption in Europe vs. the US: A look at how AI adoption is perceived as massive and mandatory in Europe, while US census data shows surprisingly low enterprise adoption rates

The world of data is being reset by AI, and the infrastructure needs to evolve with it. I sit down with streaming legend Tyler Akidau to discuss how the principles of stream processing are forming the foundation for the next generation of "agentic AI" systems.

Tyler, who was an AI cynic until recently, explains why he's now convinced that AI agents will fundamentally change how businesses operate and what problems we need to solve to deploy them safely.

Key topics we explore:

• From Human Analytics to Agentic Systems: How data architectures built for human analysis must be re-imagined for a world with thousands of AI agents operating at machine speed.

• Auditing Everything: Why managing AI requires a new level of governance where we must record all data an agent touches, not just metadata, to diagnose its complex and opaque behavior

• The End of Windowing's Dominance: Tyler reflects on the influential Dataflow paper he co-authored and explains why he now sees a table-based abstraction as a more powerful and user-friendly model than focusing on windowing.
• The D&D Alignment of AI: Tyler's brilliant analogy for why enterprises are struggling to adopt AI: we're trying to integrate "chaotic" agents into systems built for "lawful good" employees.

• A Reset for the Industry: Why the rise of AI feels like the early 2010s of streaming, where the problems are unsolved and everyone is trying to figure out the answers.

I still see some companies acting sheepish with AI, too scared to even try it out. That's a massive mistake. Now is the time to play offense with incorporating AI into your company and reimagining what it can become.

Are dashboards dead? For complex enterprise use cases, the answer might be yes. In this episode, I'm joined by Irina Malkova (VP Data & AI at Salesforce), to discuss her team's transformational journey from building complex dashboards to deploying AI-powered conversational agents.

We dive deep into how this shift is not just a change in tooling, but a fundamental change in how users access insights and how data teams measure their impact.

Join us as we cover:

• 
  
• The Shift from Dashboards to Agents: We discuss why dashboards can create a high cognitive load and fail users in complex scenarios , and how conversational agents in the flow of work (like Slack) provide targeted, actionable insights and boost adoption.
• 
  
• What is Product Telemetry?: Irina explains how telemetry is evolving from a simple engineering observability use case to a critical data source for AI, machine learning, and recommendation systems.
• 
  
• Why Standard RAG Fails in the Enterprise: Irina shares why typical RAG approaches break down on dense, entity-rich corporate data (like Salesforce's help docs) where semantic similarity isn't enough, leading to the rise of Graph RAG.
• 
  
• The New, Measurable ROI of Data: How moving from BI to agents allows data teams to precisely measure impact, track downstream actions, and finally have a concrete answer to the ROI question that was previously impossible to justify.
• 
  
• Data Teams as Enterprise Leaders: Why data teams are uniquely positioned to lead AI transformation, as they hold the enterprise "ontology" and have experience building products under uncertainty.

It's all about acquisitions, acquisitions, acquisitions! Matt Housley joins me to tackle the biggest rumor in the data world this week: the potential acquisition of dbt Labs by Fivetran. This news sparks a wide-ranging discussion on the inevitable consolidation of the Modern Data Stack, a trend we predicted as the era of zero-interest-rate policy ended.

We also talk about financial pressures, vendor exposure to the rise of AI, the future of data tooling, and more.

In this episode, I sit down with Saket Saurabh (CEO of Nexla) to discuss the fundamental shift happening in the AI landscape. The conversation is moving beyond the race to build the biggest foundational models and towards a new battleground: context. We explore what it means to be a "model company" versus a "context company" and how this changes everything for data strategy and enterprise AI.

Join us as we cover:

• Model vs. Context Companies: The emerging divide between companies building models (like OpenAI) and those whose advantage lies in their unique data and integrations.

• The Limits of Current Models: Why we might be hitting an asymptote with the current transformer architecture for solving complex, reliable business processes.

• "Context Engineering": What this term really means, from RAG to stitching together tools, data, and memory to feed AI systems.

• The Resurgence of Knowledge Graphs: Why graph databases are becoming critical for providing deterministic, reliable information to probabilistic AI models, moving beyond simple vector similarity.

  AI's Impact on Tooling: How tools like Lovable and Cursor are changing workflows for prototyping and coding, and the risk of creating the "-10x engineer."

• The Future of Data Engineering: How the field is expanding as AI becomes the primary consumer of data, requiring a new focus on architecture, semantics, and managing complexity at scale.

In this episode, I sit down with Ole to discuss his new book, "Fundamentals of Metadata Management." We move past the simple definition of "data about data" to a more nuanced view of metadata as something that exists in two places at once , serving as a pointer to find information elsewhere.

Ole introduces his core concept of the "MetaGrid"—the interconnected, yet siloed, web of metadata repositories that already exists within every large organization across various teams and technologies. He argues that the key to better metadata management is not to build a new monolithic system but to recognize, document, and integrate the MetaGrid that's already there, hiding in plain sight.

The conversation also covers the impact of the AI hype cycle , the lessons learned from the Data Mesh movement , the sociological incentives that help or hinder metadata projects , and the cultural clash between the worlds of data engineering and library science.

The way we work is changing right in front of us. In this rant, I talk about how I'm seeing AI reshape how technical and non-technical people do their work. The bottom line - there's a lot of room to innovate and evolve your job.

In this discussion, I sit down with data veterans Remco Broekmans and Marco Wobben to explore why so many data projects fail. They argue that the problem isn't the technology, but a fundamental misunderstanding of communication, culture, and long-term strategy.The conversation goes deep into the critical shift from being a "hardcore techie" to focusing on translating business needs into data models. They use the classic "involved party" data modeling pattern as a prime example of how abstract IT jargon creates a massive disconnect with the business.Marco shares a fascinating (and surprising) case study of the Dutch Railroad organization, which has been engaged in an 18-year information modeling "program" - not a project - to manage its immense complexity. This sparks a deep dive into the cultural and work-ethic differences between the US and Europe, contrasting the American short-term, ROI-driven "project" mindset with the European capacity for long-term, foundational "programs".Finally, they tackle the role of AI. Is it a silver bullet or just the latest shiny object? They conclude that AI's best use is as an "intern" or "assistant", a tool to brainstorm, ask questions, and handle initial prototyping, but never as a replacement for the deep, human-centric work of understanding a business.Timestamps:00:00 - Introduction01:09 - Marco Wobben introduces his 25-year journey in information modeling.01:56 - Remco Broekmans reintroduces himself and his focus on the communication aspect of data.03:22 - The progression from hardcore techie to focusing on communication over technology.08:16 - Why is communication in data and IT projects so difficult? 09:49 - The "Involved Party" Problem: A perfect example of where IT communication goes wrong with the business.13:35 - The essence of IT is automating the communication that happens on the business side.18:39 - Discussing a client with 20,000 distinct business terms in their information model.21:55 - The story of the Dutch Railroad's 18-year information modeling program that reduced incident response from 4 hours to 2 seconds.27:25 - Project vs. Program: A key mindset difference between the US and Europe.34:18 - The danger of chasing shiny new tools like AI without getting the fundamentals right first.39:55 - Where does AI fit into the world of data modeling? 43:34 - Why you can't trust AI to be the expert, especially with specialized business jargon.47:18 - The role of risk in trusting AI, using a self-driving car analogy.53:27 - Cultural differences in work pressure and ethics between the US and the Netherlands.59:29 - Why personality and communication skills are more important than a PhD for data modelers.01:03:38 - What is the purpose of an AI-run company with no human benefit? 01:11:21 - Using AI as an instructive tool to improve your own skills, not just to get an answer.01:14:12 - How AI can be used as a "sidekick" to ask dumb questions and help you think.01:18:00 - Where to find Marco and Remco online

Are you a giver or a taker?

It seems like every few months, I have to put out a PSA about how I get annoyed at unsolicited pitches from people who are one-sided and transactional. These people are takers. I've got no time for takers.

Instead, pay it forward. Give away your ideas in free articles and videos. Mentor people. Create an open source project. Be a friend and a human without asking anything in return. Be a giver.