July 28, 2022

Expensive Observability: The Cardinality Challenge - OpenObservability Talks S3E02

1 hour

We all collect logs, metrics and perhaps traces and other data types, in support of our observability. But this can get expensive pretty quickly, especially in microservices based systems, in what is commonly known as “the cardinality problem”.

On this episode of OpenObservability Talks I’ll host Ben Sigelman, co-founder and the GM of Lightstep, to discuss this data problem and how to overcome it. Ben architected Google’s own planet-scale metrics and distributed tracing systems (still in production today), and went on to co-create the open-source OpenTracing and OpenTelemetry projects, both part of the CNCF.

The episode was live-streamed on 12 July 2022 and the video is available at https://youtu.be/gJhzwP-mZ2k

OpenObservability Talks episodes are released monthly, on the last Thursday of each month and are available for listening on your favorite podcast app and on YouTube.

We live-stream the episodes on Twitch and YouTube Live - tune in to see us live, and pitch in with your comments and questions on the live chat.
https://www.twitch.tv/openobservability
https://www.youtube.com/channel/UCLKOtaBdQAJVRJqhJDuOlPg

Have you got an interesting topic you'd like to share in an episode? Reach out to us and submit your proposal at https://openobservability.io/

Show Notes:

The difference between monitoring, observability and APM

What comprises the cost of observability

How common is the knowledge of cardinality and how to add metrics

Controlling cost with sampling, verbosity and retention

Lessons from Google’s metrics and tracing systems

Using metric rollups and aggregations intelligently

Semantic conventions for logs, metrics and traces

OpenCost project

New research paper by Meta on schema-first approach to application telemetry metadata

OTEL code contributions - published stats

Resources:

Monitoring vs. observability: https://twitter.com/el_bhs/status/1349406398388400128

The two drivers of cardinality: https://twitter.com/el_bhs/status/1360276734344450050

Sampling vs verbosity: https://twitter.com/el_bhs/status/1440750741384089608

Observing resources and transactions: https://twitter.com/el_bhs/status/1372636288021524482

Socials:

Twitter: https://twitter.com/OpenObserv

Twitch: https://www.twitch.tv/openobservability

YouTube: https://www.youtube.com/channel/UCLKOtaBdQAJVRJqhJDuOlPg

...more

View all episodes

By Dotan Horovits

33 ratings