Software Engineering Radio - the podcast for professional software developers

SE Radio 717: Eric Tschetter on Decoupling Observability


Listen Later

In this episode, host Amey Ambade sits with Eric Tschetter, co-founder of Apache Druid and Chief Architect at Imply, to dissect the critical move toward Decoupling Observability. To begin, they define three pillars—logs, metrics, and traces—and consider why the rise of microservices has made traditional, tightly coupled stacks a major source of pain. Such coupled systems can lead to issues such as vendor lock-in, prohibitive scaling costs, and operational complexity.

Drawing parallels to the Business Intelligence world's separation, Tschetter presents an architectural solution with four distinct layers: Ingest/Route, Data Storage, Query/Compute, and Visualization. This framework aims to provide flexibility to combat the limitations of monolithic observability tools. The conversation moves into the practical challenges and significant benefits of this decoupled model, focusing heavily on data portability and the role of technologies such as OpenTelemetry in standardizing schemas so that data can flow freely between multiple back-ends. A significant portion of the discussion is dedicated to the Query/Compute layer, specifically how Apache Druid addresses the unique demands of real-time analytics on observability data, including indexing strategies and unifying results across hot and cold storage. They also delve into operational survival, covering critical topics like smart sampling to preserve high-value signals, best practices for buffering and backpressure, and the governance models required for multiple teams to safely access the same data lake.

The episode concludes with an honest look at the complexity trade-offs and a roadmap for organizations considering a migration from a coupled vendor stack.

...more
View all episodesView all episodes
Download on the App Store

Software Engineering Radio - the podcast for professional software developersBy [email protected] (SE-Radio Team)

  • 4.4
  • 4.4
  • 4.4
  • 4.4
  • 4.4

4.4

270 ratings


More shows like Software Engineering Radio - the podcast for professional software developers

View all
The Changelog: Software Development, Open Source by Changelog Media

The Changelog: Software Development, Open Source

288 Listeners

The Vergecast by The Verge

The Vergecast

3,722 Listeners

Software Engineering Daily by Software Engineering Daily

Software Engineering Daily

626 Listeners

Talk Python To Me by Michael Kennedy

Talk Python To Me

583 Listeners

Thoughtworks Technology Podcast by Thoughtworks

Thoughtworks Technology Podcast

44 Listeners

Syntax - Tasty Web Development Treats by Wes Bos & Scott Tolinski - Full Stack JavaScript Web Developers

Syntax - Tasty Web Development Treats

985 Listeners

Darknet Diaries by Jack Rhysider

Darknet Diaries

8,077 Listeners

CoRecursive: Coding Stories by Adam Gordon Bell - Software Developer

CoRecursive: Coding Stories

189 Listeners

Practical AI by Practical AI LLC

Practical AI

212 Listeners

The Stack Overflow Podcast by The Stack Overflow Podcast

The Stack Overflow Podcast

63 Listeners

The Real Python Podcast by Real Python

The Real Python Podcast

140 Listeners

Last Week in AI by Skynet Today

Last Week in AI

313 Listeners

Machine Learning Street Talk (MLST) by Machine Learning Street Talk (MLST)

Machine Learning Street Talk (MLST)

101 Listeners

Big Technology Podcast by Alex Kantrowitz

Big Technology Podcast

512 Listeners

Latent Space: The AI Engineer Podcast by Latent.Space

Latent Space: The AI Engineer Podcast

101 Listeners