OpenObservability Talks

OpenObservability Talks

By Dotan HorovitsTechnology
Download on the App Store

OpenObservability Talks episodes

  • Is Kubernetes Monitoring Flawed? - OpenObservability Talks S3E08

    A 3-node Kubernetes cluster with Prometheus will ship around 40k active series by default! Do we really need all that data? The current state of Kubernetes open source monitoring is in need of improvement. High churn rate of pod metrics, proliferation of metrics with low usage, and configuration complexity are some of the issues that need to be addressed.

    I discussed this topic with Aliaksandr Valialkin, CTO at VictoriaMetrics and creator of the open source project. We discussed the common problems, as well as directions and best practices to overcome some of these complexities as individuals and as a community. We also discussed VictoriaMetrics open source project and how it addresses some of these challenges.

    Aliaksandr a Golang engineer, who likes writing simple and performant code and creating easy-to-use programs. Sometimes these hard-to-match requirements work together, like in the VictoriaMetrics case.

    The episode was live-streamed on 24 January 2023 and the video is available at https://www.youtube.com/live/Z-58C8HFGb8

    OpenObservability Talks episodes are released monthly, on the last Thursday of each month and are available for listening on your favorite podcast app and on YouTube.

    We live-stream the episodes on Twitch and YouTube Live - tune in to see us live, and chime in with your comments and questions on the live chat.
    https://www.twitch.tv/openobservability
    https://www.youtube.com/@openobservabilitytalks

    Have you got an interesting topic you'd like to share in an episode? Reach out to us and submit your proposal at https://openobservability.io/

    Show Notes:

    • monitoring microservice system, app and communications
    • high churn rate for pod metrics
    • Kubernetes produces too many metrics by defaults, most of which are unused
    • recommended listing of metrics
    • removing unused metric labels to reduce cardinality
    • Prometheus native (exponential buckets) historgrams
    • Configuration complexity with multiple deployments
    • OpenTelemetry and OpenMetrics open specifications
    • collecting system metrics and application metrics uniformly
    • VictoriaMetrics essentials
    • VictoriaMetrics extensions beyond Prometheus
    • a full stack monitoring collection, analysis and alerting
    • how to join the VictoriaMetrics community
    • industry update: 2023 cloud native predictions post by CNCF CTO

      • Resources:

      • Why Prometheus cannot query remote storage in an expected way via remote_read protocol - https://github.com/prometheus/prometheus/issues/4456 
      • VictoriaMetrics scaling to 100 million metrics per second https://www.youtube.com/watch?v=xfed9_Q0_qU
      • https://victoriametrics.com/
      • https://github.com/VictoriaMetrics/VictoriaMetrics
      • https://docs.victoriametrics.com/#community-and-contributions

        • Socials:

        • Twitter: https://twitter.com/OpenObserv
        • Twitch: https://www.twitch.tv/openobservability
        • YouTube: https://www.youtube.com/@openobservabilitytalks

          • Dotan Horovits 

          Twitter: https://twitter.com/horovits  
          LinkedIn: https://www.linkedin.com/in/horovits/  


          Aliaksandr Valialkin

          Twitter: https://twitter.com/valyala

          LinkedIn: https://www.linkedin.com/in/valyala/

          1 hr 1 min
        • What's new in the Prometheus ecosystem? - OpenObservability Talks S3E07

          So much has been going on with the Prometheus project and its ecosystem, that it’s time to have a proper catch up. And there’s no better person to walk us through it than Julien Pivotto, who debriefed the community last month at PromCon.

          Julien Pivotto is a maintainer of Prometheus, the open source monitoring and alerting solution. He is the co-founder of the company O11y, that provides premium support for open source observability tools such as Prometheus, Thanos and Grafana.

          The episode was live-streamed on 19 December 2022 and the video is available at https://www.youtube.com/watch?v=Vui4EgveUxg

          OpenObservability Talks episodes are released monthly, on the last Thursday of each month and are available for listening on your favorite podcast app and on YouTube.

          We live-stream the episodes on Twitch and YouTube Live - tune in to see us live, and chime in with your comments and questions on the live chat.
          https://www.twitch.tv/openobservability
          https://www.youtube.com/@openobservabilitytalks

          Have you got an interesting topic you'd like to share in an episode? Reach out to us and submit your proposal at https://openobservability.io/

          Show Notes:

          • Prometheus is 10 years old
          • Prometheus mission statement and directions
          • New Agent Mode for work with external backends
          • Service Discovery ecosystem, plugins and updates
          • Time series database basics and updates
          • New support for native histograms
          • Examplars in Prometheus to correlate metrics to traces
          • PromQL query language updates
          • PromLens contribution to Prometheus
          • Prometheus UI updates
          • Visualization options: Grafana and Perses 
          • Alertmanager updates
          • Windows exporter, MySQL and other new exporters
          • Long term support for Prometheus project
          • Thanos, Cortex, Mimir - ecosystem update
          • Prometheus community  

            • Resources:

            • PromCon EU 2022 talk: https://www.youtube.com/watch?v=2Wlza5jrS-U
            • Prometheus GitHub: https://github.com/prometheus/prometheus
            • Why Your Monitoring Dashboard May Be Lying to You: https://horovits.medium.com/ca477e80589e

              • Socials:

              • Twitter: https://twitter.com/OpenObserv
              • Twitch: https://www.twitch.tv/openobservability
              • YouTube: https://www.youtube.com/channel/UCLKOtaBdQAJVRJqhJDuOlPg
                • Host: https://twitter.com/horovits 
                  • 1 hr 5 min
                • Meta’s data driven approach to observability - OpenObservability Talks S3E06

                  At Meta (Facebook, Instragram et al) everything is data, and data driven approach is the rule, from product to engineering, from HR to finance. This is also how the team at Meta treats observability. Let’s see how we treat observability as a data analytics problem, and what you can implement, even if you’re not a hyperscaler.

                  On this episode I’ll host David Ostrovsky from Meta. David is a software developer with over 20 years of industry experience, speaker, trainer, blogger and co-author of “Pro Couchbase Server”. He specializes in large-scale distributed system architecture.

                  The episode was live-streamed on 7 November 2022 and the video is available at https://youtu.be/1l0HKUDoX4Q

                  OpenObservability Talks episodes are released monthly, on the last Thursday of each month and are available for listening on your favorite podcast app and on YouTube.

                  We live-stream the episodes on Twitch and YouTube Live - tune in to see us live, and chime in with your comments and questions on the live chat.
                  https://www.twitch.tv/openobservability
                  https://www.youtube.com/channel/UCLKOtaBdQAJVRJqhJDuOlPg

                  Have you got an interesting topic you'd like to share in an episode? Reach out to us and submit your proposal at https://openobservability.io/

                  Show Notes:

                  • Addressing business observability
                  • Machine learning and predictions in observability
                  • From the business inwards, using SLOs
                  • Accelerate engineering quality with developer observability
                  • Organizational and communications aspects of high scale observability
                  • Actionable observability
                  • How small-medium size orgs can achieve a similar effect
                  • OpenTelemetry demo is GA
                  • PromLens is open sourced and contributed to Prometheus

                    • Resources:

                    • TEMPLE signals for observability: https://medium.com/@YuriShkuro/temple-six-pillars-of-observability-4ac3e3deb402
                    • PromLens open sourced: https://promlabs.com/blog/2022/10/25/promlabs-and-chronosphere-open-source-the-promlens-query-builder

                      • Socials:

                      • Twitter: https://twitter.com/OpenObserv
                      • Twitch: https://www.twitch.tv/openobservability
                      • YouTube: https://www.youtube.com/channel/UCLKOtaBdQAJVRJqhJDuOlPg
                        • 1 hr 1 min
                      • Platform Engineering: DevOps evolution or a fancy rename? - OpenObservability Talks S3E05

                        Everyone’s talking about Platform Engineering these days. Even Gartner featured it in its Hype Cycle for Software Engineering 2022. But what is Platform Engineering really about? Is it the next stage in the evolution of DevOps? Is it just a fancy rebrand for DevOps or SRE? And how does observability pertain to platform engineering?

                        On this episode of OpenObservability Talks Horovits hosted George Hantzaras, Director of Cloud Platform Engineering at Citrix. George is a distributed systems expert and a hands-on engineering leader with focus on delivering B2B cloud services at scale. Coming from a DevOps background, he focuses on implementing SRE at enterprise scale working with cloud native technologies. He has been organizing the Athens Cloud Computing Meetup since 2016 and the Athens Hashicorp User Group. Most recently, he has been a speaker at global events like Hashiconf, DeveloperWeek, Voxxed Days, DevNexus and more, focusing on reliability engineering, agile leadership, scaling engineering teams, and entrepreneurship.

                        The episode was live-streamed on 6 October 2022 and the video is available at https://youtu.be/zJGMUVY6fDM

                        OpenObservability Talks episodes are released monthly, on the last Thursday of each month and are available for listening on your favorite podcast app and on YouTube.

                        We live-stream the episodes on Twitch and YouTube Live - tune in to see us live, and chime in with your comments and questions on the live chat.
                        https://www.twitch.tv/openobservability
                        https://www.youtube.com/channel/UCLKOtaBdQAJVRJqhJDuOlPg

                        Have you got an interesting topic you'd like to share in an episode? Reach out to us and submit your proposal at https://openobservability.io/

                        Show Notes:

                        • Why Platform Engineering?
                        • Different teams employ different stacks. How does Platform Engineering fits in?
                        • Platform Engineering at Citrix
                        • How does it map to the common DevOps maturity models?
                        • Is Platform Engineering different from PaaS?
                        • Platform as a product
                        • Culture element is central in Platform Engineering
                        • Generating buy-in within the product’s engineering
                        • Metrics and quantifying the benefits of Platform Engineering
                        • Balancing flexibility and simplicity, and striking the right level of abstraction
                        • Delimitation between what’s developed by Platform vs. Product teams.
                        • Generating Golden Paths for engineering
                        • Observability and Platform Engineering

                          • Resources:

                          • George Hantzaras presentations
                          • Designing Golden Paths
                          • platformengineering.org
                          • Platform Engineering KPIs

                            • Socials:

                            • Twitter: https://twitter.com/OpenObserv
                            • Twitch: https://www.twitch.tv/openobservability
                            • YouTube: https://www.youtube.com/channel/UCLKOtaBdQAJVRJqhJDuOlPg
                              • 59 min
                            • Where Are My App’s Traces?? Instrumentation in Practice - OpenObservability Talks S3E04

                              Instrumentation is that black magic that makes our application emit traces, logs, metrics or other telemetry. How does it work? What options are available in different programming languages, such as Java, Python and Go? What does OpenTelemetry offer in this domain?  

                              On this episode of OpenObservability Talks I hosted Eden Federman, Co-Founder & CTO at keyval, a company focused on making observability simpler. Eden is the creator of two open source projects: Odigos and Go automatic instrumentation (now part of OpenTelemetry). Eden is passionate about everything related to observability and performance monitoring. He also created kubectl-flame, a profiler for Kubernetes.

                              The episode was live-streamed on 22 September 2022 and the video is available at https://youtu.be/VFykWV1mLAI

                              OpenObservability Talks episodes are released monthly, on the last Thursday of each month and are available for listening on your favorite podcast app and on YouTube.

                              We live-stream the episodes on Twitch and YouTube Live - tune in to see us live, and chime in with your comments and questions on the live chat.
                              https://www.twitch.tv/openobservability
                              https://www.youtube.com/channel/UCLKOtaBdQAJVRJqhJDuOlPg

                              Have you got an interesting topic you'd like to share in an episode? Reach out to us and submit your proposal at https://openobservability.io/

                              Show Notes:

                              • What is instrumentation
                              • Manual and Automatic instrumentation
                              • Different languages offer different options
                              • Java instrumentation capabilities
                              • Go instrumentation capabilities
                              • Instrumentation when using programming frameworks
                              • eBPF use in auto-instrumentation
                              • New OpenTelemetry SIG for Go auto-instrumentation
                              • Odigos open source project
                              • Best practices for instrumentation

                                • Resources:

                                • Instrumentation basics and OpenTelemetry support
                                • eBPF Automatic instrumentation for Go
                                • Odigos (Observability Control Plane)
                                • OpenTelemetry Go auto instrumentation SIG
                                • How OpenTelemetry works under the hood in JavaScript
                                • Spring Boot instrumentation
                                • Spring Cloud Sleuth 1.1.0 released


                                  • Socials:

                                  Twitter:⁠ https://twitter.com/OpenObserv⁠

                                  YouTube: ⁠https://www.youtube.com/@openobservabilitytalks⁠


                                  1 hr 1 min
                                • SigNoz: Open-source observability with Pranay - OpenObservability Talks S3E03

                                  In this episode, hosted by Jonah Kowall we are excited to introduce Pranay Prateek the co-founder and creator of SigNoz to the audience. He is an entrepreneur who has been building an interesting and exciting open-source observability platform. The goal is a unified interface and data storage system which is scalable and manageable much more easily than today’s disjointed open-source tools. SigNoz has a goal of unifying tracing, metrics, and logging in a single platform and interface. We’ll dig into the evolution of this unique project from its creation until today, along with the adoption of the technology and contribution from the community. We will discuss the data architecture and their move from Druid towards ClickHouse, which is always a fascinating topic. Finally, we’ll elaborate on the future of SigNoz and where things are going with the project. Join me, Jonah Kowall, as I speak with Pranay Prateek on this episode of OpenObservability Talks.

                                  The podcast episodes are available for listening on your favorite podcast app and on this YouTube channel.

                                  Socials:

                                  Twitter:⁠ https://twitter.com/OpenObserv⁠

                                  YouTube: ⁠https://www.youtube.com/@openobservabilitytalks⁠


                                  57 min
                                • Expensive Observability: The Cardinality Challenge - OpenObservability Talks S3E02

                                  We all collect logs, metrics and perhaps traces and other data types, in support of our observability. But this can get expensive pretty quickly, especially in microservices based systems, in what is commonly known as “the cardinality problem”.  

                                  On this episode of OpenObservability Talks I’ll host Ben Sigelman, co-founder and the GM of Lightstep, to discuss this data problem and how to overcome it. Ben architected Google’s own planet-scale metrics and distributed tracing systems (still in production today), and went on to co-create the open-source OpenTracing and OpenTelemetry projects, both part of the CNCF.

                                  The episode was live-streamed on 12 July 2022 and the video is available at https://youtu.be/gJhzwP-mZ2k

                                  OpenObservability Talks episodes are released monthly, on the last Thursday of each month and are available for listening on your favorite podcast app and on YouTube.

                                  We live-stream the episodes on Twitch and YouTube Live - tune in to see us live, and pitch in with your comments and questions on the live chat.
                                  https://www.twitch.tv/openobservability
                                  https://www.youtube.com/channel/UCLKOtaBdQAJVRJqhJDuOlPg

                                  Have you got an interesting topic you'd like to share in an episode? Reach out to us and submit your proposal at https://openobservability.io/

                                  Show Notes:

                                  • The difference between monitoring, observability and APM
                                  • What comprises the cost of observability
                                  • How common is the knowledge of cardinality and how to add metrics
                                  • Controlling cost with sampling, verbosity and retention
                                  • Lessons from Google’s metrics and tracing systems
                                  • Using metric rollups and aggregations intelligently
                                  • Semantic conventions for logs, metrics and traces
                                  • OpenCost project
                                  • New research paper by Meta on schema-first approach to application telemetry metadata
                                  • OTEL code contributions - published stats

                                    • Resources:

                                    • Monitoring vs. observability: https://twitter.com/el_bhs/status/1349406398388400128
                                    • The two drivers of cardinality: https://twitter.com/el_bhs/status/1360276734344450050
                                    • Sampling vs verbosity: https://twitter.com/el_bhs/status/1440750741384089608
                                    • Observing resources and transactions: https://twitter.com/el_bhs/status/1372636288021524482

                                      • Socials:

                                      • Twitter: https://twitter.com/OpenObserv
                                      • Twitch: https://www.twitch.tv/openobservability
                                      • YouTube: https://www.youtube.com/channel/UCLKOtaBdQAJVRJqhJDuOlPg


                                        • 1 hr 1 min
                                      • OpenTelemetry and the Vision for Unified Open Observability - OpenObservability Talks S3E01

                                        OpenTelemetry is one of the most fascinating and ambitious open source projects of this era. It’s currently the second most active project in the CNCF (the Cloud Native Computing Foundation), with only Kubernetes being more active. The entire industry is aligning behind this project, including incumbent monitoring vendors that were deeply vested in proprietary and closed-source agents to that end.

                                        In this episode of OpenObservability Talks I’ll host Alolita Sharma to discuss OpenTelemetry, its origins and mission statement, as well as updates hot off the press from the recent KubeCon conference in Valencia about releases and future plans.

                                        Alolita is co-chair of the CNCF Technical Advisory Group for Observability, member of the OpenTelemetry Governance Committee and a board director of the Unicode Consortium. She has served on the boards of the OSI and SFLC.in. Alolita has led engineering teams at Wikipedia, Twitter, PayPal, IBM and AWS. Two decades of doing open source continue to inspire her.

                                        The episode was live-streamed on 15 June 2022 and the video is available at https://youtu.be/IK2TWOzDUBI 

                                        OpenObservability Talks episodes are released monthly, on the last Thursday of each month and are available for listening on your favorite podcast app and on YouTube.

                                        We live-stream the episodes on Twitch and YouTube Live - tune in to see us live, and pitch in with your comments and questions on the live chat.
                                        https://www.twitch.tv/openobservability
                                        https://www.youtube.com/channel/UCLKOtaBdQAJVRJqhJDuOlPg


                                        You can read the recap post: https://logz.io/blog/opentelemetry-roadmap-and-latest-updates/?utm_source=devrel&utm_medium=devrel


                                        Show Notes:

                                        • Hot updates from KubeCon EMEA 2022
                                        • Alolita Sharma introduction
                                        • The state of OpenTelemetry
                                        • When OpenTelemetry Logging is expecting GA
                                        • The onboarding challenge of instrumentation
                                        • Client side instrumentation and real user monitoring
                                        • Adding continuous profiling telemetry to OpenTelemetry
                                        • Interoperability between OpenTelemetry and Prometheus
                                        • Challenges in OpenTelemetry and observability
                                        • Where OpenTelemetry is heading next
                                        • Jaeger OSS now accept OTLP (OpenTelemetry protocol)

                                          • Resources:

                                          • OpenTelemetry Metrics reaches RC: https://opentelemetry.io/blog/2022/metrics-announcement/
                                          • OpenTelemetry guide: https://logz.io/learn/opentelemetry-guide/
                                          • CI/CD Observability: https://horovits.medium.com/fighting-slow-and-flaky-ci-cd-pipelines-starts-with-observability-19da2ac94677
                                          • Jaeger can now accept OpenTelemetry protocol https://medium.com/jaegertracing/introducing-native-support-for-opentelemetry-in-jaeger-eb661be8183c
                                          • OTel Community Day summary: http://paulsbruce.io/blog/2022/06/opentelemetry-community-day-austin-2022
                                          • Contextual Logging in Kubernetes 1.24 https://kubernetes.io/blog/2022/05/25/contextual-logging/ 
                                          • PolarSignals announced FrostDB https://www.polarsignals.com/blog/posts/2022/05/04/introducing-arcticdb/ 

                                            • Socials:

                                            • Twitter: https://twitter.com/OpenObserv
                                            • Twitch: https://www.twitch.tv/openobservability
                                            • YouTube: https://www.youtube.com/channel/UCLKOtaBdQAJVRJqhJDuOlPg

                                              • Dotan Horovits

                                              ============

                                              Twitter: @horovits

                                              LinkedIn: in/horovits

                                              Mastodon: @horovits@fosstodon


                                              Alolita Sharma

                                              ============

                                              Twitter: @alolita

                                              LinkedIn: https://www.linkedin.com/in/alolita/


                                              1 hr 1 min
                                            • Observability for Developers Demystified - OpenObservability Talks E2E12

                                              Developers hate monitoring, but we need it. We need it in many points of the software development lifecycle: before deprecating an API, before launching a new feature, after launching the feature, and more. In fact, monitoring needs can vary much more than the classic Ops monitoring.

                                              In this episode I’ll host Liran Haimovitch to discuss how to determine what developers should be monitoring, the difference between observability for Dev and for Ops, and how observability fits into our current dev tools, dev stack and dev processes.

                                              Liran is the Co-Founder and CTO of Rookout. He’s an Observability and Instrumentation expert with a deep understanding of Java, Python, Node, and C++. Liran has broad experience in cybersecurity and compliance from his past roles. When not coding, you can find Liran hosting his podcast, speaking at conferences, writing about his tech adventures, and trying out the local cuisine when traveling.

                                              The episode was live-streamed on 10 May 2022 and the video is available at https://youtu.be/OaHQp-qnVN0 

                                              OpenObservability Talks episodes are released monthly, on the last Thursday of each month and are available for listening on your favorite podcast app and on YouTube.

                                              We live-stream the episodes on Twitch and YouTube Live - tune in to see us live, and pitch in with your comments and questions on the live chat.
                                              https://www.twitch.tv/openobservability
                                              https://www.youtube.com/channel/UCLKOtaBdQAJVRJqhJDuOlPg

                                              Have you got an interesting topic you'd like to share in an episode? Reach out to us and submit your proposal at https://openobservability.io/

                                              Show Notes:

                                              • Which data do we need to collect for our observability
                                              • How is observability for dev different from ops
                                              • How does observability fit into dev tool stack
                                              • Snapshots provide deep-dive telemetry signal
                                              • Dynamic instrumentation
                                              • Snapshots support in programming languages and runtimes
                                              • Open source standardization around snapshots
                                              • The cost associated with observability
                                              • Google is applying to contribute Istio to the CNCF
                                              • Shopify case study for observability team

                                                • Resources:

                                                • Istio applying to the CNCF: https://istio.io/latest/blog/2022/istio-has-applied-to-join-the-cncf/
                                                • Shopify case study for Observability team: https://ericmustin.substack.com/p/notes-on-an-observability-team?s=r

                                                  • Socials:

                                                  • Twitter: https://twitter.com/OpenObserv
                                                  • Twitch: https://www.twitch.tv/openobservability
                                                  • YouTube: https://www.youtube.com/channel/UCLKOtaBdQAJVRJqhJDuOlPg


                                                    • 59 min
                                                  • OpenSearch 2.0 and beyond with Eli - OpenObservability Talks E2E11

                                                    OpenSearch is a community-driven, open-source search and analytics suite derived from Apache 2.0 licensed Elasticsearch 7.10.2 & Kibana 7.10.2. The OpenSearch project started just over a year ago and is now the open-source alternative to ELK, which is no longer open source. The team has spent much of the last year getting the project going, but there was innovation as well. We will cover and discuss what OpenSearch has accomplished, but more importantly what’s coming next, including a big 2.0 release. We are joined in this episode by Eli Fisher, who is the product lead at AWS, working on the OpenSearch project. He’ll dive into recent launches, including several observability features, and innovations planned for 2.0 and beyond.   

                                                    The podcast episodes are available for listening on your favorite podcast app and on this YouTube channel.  

                                                    We live-stream the episodes, and you’re welcome to join the stream here on YouTube Live or at https://www.twitch.tv/openobservability​.   


                                                    1 hr 2 min

                                                  About OpenObservability Talks

                                                  From the publisher's feed

                                                  On OpenObservability Talks we discuss harnessing the power of open source to advance observability initiatives for developers, DevOps and SRE practitioners around the world.