Generative AI is everywhere, but how do we monitor and observe it? OpenTelemetry has been a prominent tool and standard for observability, and recently the OTel community has been aiming to expand its scope and cover GenAI workloads with semantic conventions and tools.

In this episode, Horovits is joined by Nir Gazit, creator of the OpenLLMetry project, and member of the OpenTelemetry Generative AI SIG. We discuss new semantic conventions, tracing prompts and model behavior, the OpenLLMetry project’s journey, and what observability even means for modern AI systems.

Nir Gazit is the CEO and co-founder of Traceloop, and brings a wealth of data and AI experience, with previous experience leading AI teams at Google and serving as the Chief Architect at Fiverr.

You can read the recap post: https://medium.com/p/81b9cea6a771/

Show Notes:

00:00 - intro 

04:09 - what is observability for AI

18:07 - AI observability differences from traditional observability

25:22 - OpenLLMetry intro

41:21 - OpenLLMetry latest updates and roadmap

47:00 - OpenTelemetry GenAI Semantic Conventions SIG

56:03 - KubeCon updates: CrossPlane, Knative, Dragonfly, in-toto reached CNCF graduation 

1:00:08 - outro

Resources:

OpenTelemetry Generative AI Observability SIG: https://github.com/open-telemetry/community/blob/1c71595874e5d125ca92ec3b0e948c4325161c8a/projects/llm-semconv.md

https://github.com/traceloop/openllmetry

https://github.com/traceloop/hub

https://github.com/traceloop/opentelemetry-mcp-server

Socials:

BlueSky: https://bsky.app/profile/openobservability.bsky.social

Twitter: ⁠https://twitter.com/OpenObserv⁠

LinkedIn: https://www.linkedin.com/company/openobservability/

YouTube: ⁠https://www.youtube.com/@openobservabilitytalks⁠

Dotan Horovits

============

Twitter: @horovits

LinkedIn: www.linkedin.com/in/horovits

Mastodon: @horovits@fosstodon

BlueSky: @horovits.bsky.social

Nir Gazit

========

Twitter: https://x.com/nir_ga

LinkedIn: https://www.linkedin.com/in/nirga/

OpenObservability Talks episodes are released monthly, on the last Thursday of each month and are available for listening on your favorite podcast app and on YouTube.