Changelog Master Feed

Kaizen! Five incidents later (Ship It! #20)


Listen Later

This is our second Kaizen episode, where Adam, Jerod & Gerhard talk about changelog.com improvements since episode 10. OK, so Gerhard deleted the DNS API token. Not only did he take the time to understand how that happened, so that he could actually learn from his mistake, but now we have a system in place so that we can share learnings from incidents. By the way, these are publicly available in our #incidents Slack channel.

A great & unexpected thing that happened since we recorded this episode, is Jerod fixing 99% of all the errors that were happening in prod. The top error was the broken Twitter auth - sorry Matt - which was a result of us upgrading to OTP 24 a few months back. Episode 3 show notes include a YouTube stream which captures it all.

We wrap up this episode by each of us sharing the improvements that we would like to do until our next Kaizen. You heard it from Adam first: Ship It Driven Development

Join the discussion

Changelog++ members save 4 minutes on this episode because they made the ads disappear. Join today!

Sponsors:

  • Fly.io – Deploy your apps and databases close to your users. In minutes you can run your Ruby, Go, Node, Deno, Python, or Elixir app (and databases!) all over the world. No ops required. Learn more at fly.io/changelog and check out the speedrun in their docs.
  • PlanetScale – PlanetScale is the only serverless database platform you can start in an instant and scale indefinitely with unlimited connections. Never think about database servers again. Everything you want to control is available through the beautifully designed PlanetScale CLI. Learn more and start your database in seconds at planetscale.com
  • HoneycombGuess less, know more. When production is running slow, it’s hard to know where problems originate: is it your application code, users, or the underlying systems? With Honeycomb you get a fast, unified, and clear understanding of the one thing driving your business: production. Join the swarm and try Honeycomb free today at honeycomb.io/changelog
  • FastlyOur bandwidth partner. Fastly powers fast, secure, and scalable digital experiences. Move beyond your content delivery network to their powerful edge cloud platform. Learn more at fastly.com
  • Featuring:

    • Adam Stacoviak – Website, GitHub, LinkedIn, Mastodon, X
    • Jerod Santo – GitHub, LinkedIn, Mastodon, X
    • Gerhard Lazu – Website, GitHub, LinkedIn, X

    Show Notes:

    • changelog.com PR#378
    • Oban: Robust job processing for Elixir
    • changelog.slack.com #incidents
    • Incident.io: Our incidents platform
    • DNSimple: Our DNS provider
    • Fastly: Our content delivery network
    • Linode - Connectivity Issue - US-East (Newark)
    • Linode - Service Issue - Linode Kubernetes Engine
    • Linode Global Infrastructure
    • Sentry: Our error tracking
    • Jerod: This is what happens when @gerhardlazu…
    • Founderstalk #80: Iterating to globally distributed apps and databases
    • 🎬 Firecracker VMs on Metal, Oh My! - Kurt Mackey, March 2021
    • 🎬 Live Debugging the Changelog’s Production Kubernetes | Rawkode Live, September 2021
    • Something missing or broken? PRs welcome!

      ...more
      View all episodesView all episodes
      Download on the App Store

      Changelog Master FeedBy Changelog Media

      • 4.4
      • 4.4
      • 4.4
      • 4.4
      • 4.4

      4.4

      29 ratings


      More shows like Changelog Master Feed

      View all
      Software Engineering Radio - the podcast for professional software developers by team@se-radio.net (SE-Radio Team)

      Software Engineering Radio - the podcast for professional software developers

      272 Listeners

      Hanselminutes with Scott Hanselman by Scott Hanselman

      Hanselminutes with Scott Hanselman

      383 Listeners

      The Changelog: Software Development, Open Source by Changelog Media

      The Changelog: Software Development, Open Source

      290 Listeners

      Software Engineering Daily by Software Engineering Daily

      Software Engineering Daily

      622 Listeners

      Talk Python To Me by Michael Kennedy

      Talk Python To Me

      584 Listeners

      Soft Skills Engineering by Jamison Dance and Dave Smith

      Soft Skills Engineering

      288 Listeners

      Thoughtworks Technology Podcast by Thoughtworks

      Thoughtworks Technology Podcast

      43 Listeners

      The TWIML AI Podcast (formerly This Week in Machine Learning & Artificial Intelligence) by Sam Charrington

      The TWIML AI Podcast (formerly This Week in Machine Learning & Artificial Intelligence)

      437 Listeners

      Syntax - Tasty Web Development Treats by Wes Bos & Scott Tolinski - Full Stack JavaScript Web Developers

      Syntax - Tasty Web Development Treats

      987 Listeners

      CoRecursive: Coding Stories by Adam Gordon Bell - Software Developer

      CoRecursive: Coding Stories

      189 Listeners

      Kubernetes Podcast from Google by Abdel Sghiouar, Kaslin Fields

      Kubernetes Podcast from Google

      180 Listeners

      Practical AI by Practical AI LLC

      Practical AI

      204 Listeners

      The Stack Overflow Podcast by The Stack Overflow Podcast

      The Stack Overflow Podcast

      63 Listeners

      Big Technology Podcast by Alex Kantrowitz

      Big Technology Podcast

      501 Listeners

      Oxide and Friends by Oxide Computer Company

      Oxide and Friends

      66 Listeners