OpenObservability Talks

Building web-scale observability at Slack, Pinterest & Twitter - OpenObservability Talks S2E09


Listen Later

What does it take to build observability in a web-scale company such as Slack, Pinterest and Twitter?

On this episode of OpenObsevability Talks I'll host Suman Karumuri to hear how he built these systems from the ground up on these #BigTech co's, about his recent research papers and more.

Suman Karumuri is a Sr. Staff Software Engineer and the tech lead for Observability at Slack. Suman Karumuri is an expert in distributed tracing and was a tech lead of Zipkin and a co-author of OpenTracing standard, a Linux Foundation project via the CNCF. Previously, Suman Karumuri has spent several years building and operating petabyte scale log search, distributed tracing and metrics systems at Pinterest, Twitter and Amazon.  In his spare time, he enjoys board games, hiking and playing with his kids.

The episode was live-streamed on 16 February 2022 and the video is available at https://youtu.be/IvidkV3TfYg 

OpenObservability Talks episodes are released monthly, on the last Thursday of each month and are available for listening on your favorite podcast app and on YouTube.

We live-stream the episodes on Twitch and YouTube Live - tune in to see us live, and pitch in with your comments and questions on the live chat.
https://www.twitch.tv/openobservability
https://www.youtube.com/channel/UCLKOtaBdQAJVRJqhJDuOlPg


Show Notes:

* Who owns observability in large organizations?

* The gaps in current way of handling metrics 

* MACH research paper for metrics storage engine

* The gaps in current way of handling logs Slack KalDB

* SlackTrace - Slack in house tracing system 

Resources:

  • Research paper: building Observability Data Management Systems
  • CIDR paper: Video
  • SlackTrace blog post, talk.
  • Logging at Twitter
  • Pintrace: A Distributed Tracing Pipeline talk by Suman at LISA
  • Observability Engineering book
  • Observability Trends for 2022
  • Yelp engineering with Elasticsearch and Lucene
  • Socials:

    • Twitter: https://twitter.com/OpenObserv
    • Twitch: https://www.twitch.tv/openobservability
    • YouTube: https://www.youtube.com/channel/UCLKOtaBdQAJVRJqhJDuOlPg

    • ...more
      View all episodesView all episodes
      Download on the App Store

      OpenObservability TalksBy Dotan Horovits

      • 5
      • 5
      • 5
      • 5
      • 5

      5

      2 ratings


      More shows like OpenObservability Talks

      View all
      Software Engineering Radio - the podcast for professional software developers by se-radio@computer.org

      Software Engineering Radio - the podcast for professional software developers

      266 Listeners

      Wait Wait... Don't Tell Me! by NPR

      Wait Wait... Don't Tell Me!

      38,660 Listeners

      SANS Internet Stormcenter Daily Cyber Security Podcast (Stormcast) by Johannes B. Ullrich

      SANS Internet Stormcenter Daily Cyber Security Podcast (Stormcast)

      628 Listeners

      The Changelog: Software Development, Open Source by Changelog Media

      The Changelog: Software Development, Open Source

      285 Listeners

      The Cloudcast by Massive Studios

      The Cloudcast

      153 Listeners

      Thoughtworks Technology Podcast by Thoughtworks

      Thoughtworks Technology Podcast

      42 Listeners

      Conversations with Tyler by Mercatus Center at George Mason University

      Conversations with Tyler

      2,397 Listeners

      Data Engineering Podcast by Tobias Macey

      Data Engineering Podcast

      139 Listeners

      The Daily by The New York Times

      The Daily

      111,160 Listeners

      Kubernetes Podcast from Google by Abdel Sghiouar, Kaslin Fields

      Kubernetes Podcast from Google

      180 Listeners

      Hard Fork by The New York Times

      Hard Fork

      5,370 Listeners

      System Design by Wes and Kevin

      System Design

      93 Listeners