DevOps and Docker Talk: Cloud Native Interviews and Tooling

Better K8s Prometheus Alerts with Robusta


Listen Later

😇 My new GitHub Security workshop has launched! A free 2-hour workshop with hands-on labs to harden your repos and your workflows from common supply chain attacks. I'll cover how attackers are getting in, and then we'll lock down a sample repo so you know what needs to be done to protect your code. You'll leave with a deep understanding of risks and mitigations as well as a list of helpful tools to keep your repos safe, including my new "gasa" tool for scanning your repos and orgs.


Bret is joined by Natan Yellin, the co-founder of Robusta.dev to talk Kubernetes and Prometheus monitoring, alerting, and maybe some CPU limit ranting.

Robusta tries to fill the gap left by Kubernetes' own AlertManager which has a very specific and not so helpful way of describing events in your cluster. This makes it hard to diagnose the cause of the event and you're left with Google, StackOverflow and an awful lot of head-scratching. Robusta acts as a proxy between AlertManager and your notification platform of choice.

In the show we talk about what Robusta is, how to deploy it in your clusters, and Natan also details some of the enhancements in their cloud offering that you can layer on top of that, which has a generous free tier.

Streamed live on YouTube on January 5, 2023.


Unedited live recording of this show on YouTube (Ep. #197). Includes demos.

★Topics★
Robusta Website
Robusta on GitHub
KubeCon - Building a Runbook Automation System for Prometheus and Kubernetes
Stop using K8s CPU limits
Recommended Pod Spec
Send Push notifications to your phone
Prometheus AlertManager
Grafana Labs
Kubewatch

★Natan Yellin★
Natan on Twitter
Natan on LinkedIn

★Join my Community★
New live course on CI automation and gitops deployments
Best coupons for my Docker and Kubernetes courses
Chat with us and fellow students on our Discord Server DevOps Fans
Grab some merch at Bret's Loot Box

Homepage bretfisher.com

  • (00:00) - DDT MAIN
  • (00:04) - Intro
  • (00:53) - In today's episode
  • (02:59) - Main show
  • (03:27) - Introducing Natan
  • (03:53) - Alert fatigue
  • (04:29) - Where did the idea for Robusta come from?
  • (08:16) - Someone has to do the job
  • (09:17) - What does Robusta offer?
  • (10:25) - Proxying the alerts and providing context
  • (11:30) - Saving 10 to 30 minutes
  • (13:48) - The open source Robusta repo
  • (14:10) - The need to de-aggregate event data
  • (15:09) - Example or demo
  • (15:39) - Question about observability for microservices
  • (18:38) - Tip 1 Consider using silences
  • (19:49) - Tip 2 Monitor outcomes
  • (20:23) - Don't ignore alerts because of fatigue
  • (23:13) - Sending to different channels based on priority
  • (24:42) - Question about sending messages to destinations
  • (26:17) - Question
  • (26:49) - Installing Robusta
  • (27:42) - Demo set up commands
  • (27:54) - Questions
  • (28:11) - Demo Kubernetes-specific
  • (29:05) - Multi-cluster question
  • (31:32) - What does the SaaS platform do?
  • (32:44) - Demo with SaaS
  • (33:37) - kubectl not recommended
  • (35:03) - Breaking the glass
  • (38:15) - Question about notifications
  • (40:14) - Getting started
  • (41:24) - CPU limiting
  • (42:15) - Soft limits on CPU in Kubernetes
  • (44:35) - Bret's pod spec
  • (49:22) - Outro

  • You can also support this podcast by subscribing to my YouTube channel and my monthly newsletter at bret.news!

    Grab the best coupons for my GitHub, Agents, Docker, and Kubernetes courses.
    Join my cloud native DevOps community on Discord.
    Grab some merch at Bret's Loot Box
    Homepage bretfisher.com

    ...more
    View all episodesView all episodes
    Download on the App Store

    DevOps and Docker Talk: Cloud Native Interviews and ToolingBy Bret Fisher

    • 4.6
    • 4.6
    • 4.6
    • 4.6
    • 4.6

    4.6

    54 ratings


    More shows like DevOps and Docker Talk: Cloud Native Interviews and Tooling

    View all
    The Joe Rogan Experience by Joe Rogan

    The Joe Rogan Experience

    228,383 Listeners

    All Ears English Podcast by Lindsay McMahon and Michelle Kaplan

    All Ears English Podcast

    2,237 Listeners

    The Daily by The New York Times

    The Daily

    111,948 Listeners

    Agentic DevOps : AI Engineering for Infrastructure by Bret Fisher

    Agentic DevOps : AI Engineering for Infrastructure

    2 Listeners