Fluentd and Fluent Bit are two highly popular open source projects for data collection and log forwarding in the realm of observability. Fluentd's flexibility and scalability have led to seamless integration with diverse applications and data sources, while Fluent Bit's lightweight and efficient log forwarding have made it a preferred choice for modern observability pipelines. But Fluent Bit can process more than just logs. The recent release of Fluent Bit v2 added major new integrations with OpenTelemetry, Prometheus and more, as well as extensibility with WebAssembly plugins.

On this episode I hosted Eduardo Silva Pereira, one of Fluentd project maintainers and creator of Fluent Bit. He also is the founder of Calyptia, the Fluent company. Eduardo shared with us the latest updates of Fluentd and Fluent Bit, as well as valuable insights into the future roadmap of these projects.

OpenObservability Talks episodes are released monthly, on the last Thursday of each month and are available for listening on your favorite podcast app and on YouTube.

We live-stream the episodes on Twitch and YouTube Live - tune in to see us live, and chime in with your comments and questions on the live chat.

https://www.youtube.com/@openobservabilitytalks  

⁠https://www.twitch.tv/openobservability⁠

Show Notes:

02:11 - Fluentd and Fluent Bit background

09:51 - Should I choose FluentD or Fluent Bit?

13:26 - developing an active engaged OSS community

17:18 - enterprise needs and building commercial offering with Calyptia

19:54 - Fluent Bit v2 updates

29:22 - plugins, filters and processors in Fluent Bit

38:23 - A sneak peak into the planned announcements for KubeCon Chicago

44:16 - where to follow the community and Eduardo

47:43 - Prometheus now supports OTLP

48:57 - PromCon will take place in Berlin, 28-29 Sept.

50:11 - OpenTelemetry Semantic Conventions is separated from the Specification

53:38 - New in Kubernetes 1.27: Query node logs using the kubelet API

54:43 - Kelemetry: global tracing for Kubernetes control plane

Resources:

Resources

=========

Fluent Bit: https://fluentbit.io/

FluentD: https://www.fluentd.org/

Prometheus supports OTLP: https://horovits.medium.com/83f85878e46a

PromCon '23: https://promcon.io/2023-berlin/

Query node logs using the kubelet API: https://logz.io/blog/a-practical-guide-to-kubernetes-logging/#Kubernetes_1_27

Kelemetry project: https://www.cncf.io/blog/2023/07/27/kelemetry-global-tracing-for-kubernetes-control-plane/

Socials:

Twitter:⁠ https://twitter.com/OpenObserv⁠

YouTube: ⁠https://www.youtube.com/@openobservabilitytalks⁠