Maintainable

Brian Scanlan: Improving Oncall Support with Meaningful Alarms


Listen Later

In this episode of Maintainable, Robby is joined by Brian Scanlan, Principal Systems Engineer at Intercom. Brian shares insights into how Intercom has successfully implemented a volunteer-led on-call support system, emphasizing the importance of meaningful alarms and a consistent software environment.

Summary of Topics
  • [00:05:32] Introduction to the Guest's Background: Brian Scanlan discusses his role at Intercom and his experience with on-call practices.
  • [00:15:10] Developing a Volunteer-Led On-Call System: Brian explains how Intercom transitioned to a volunteer-led on-call system, ensuring that alarms are meaningful and actionable.
  • [00:20:00] The Role of Consistent Software Architecture: The impact of Intercom’s monolithic Ruby on Rails architecture on simplifying on-call duties.
  • [00:29:46] Managing Technical Debt as Velocity Risks: Brian describes how Intercom manages technical debt through a velocity risk framework.
  • [00:38:45] Improving Deployment Processes: The evolution of Intercom’s deployment processes, reducing the time from merge to production.
  • [00:43:32] Treating Internal Tools as a Product: The importance of treating internal tools with the same care as external products, focusing on usability and impact.
  • [00:50:56] Encouraging Small Wins in Productivity: How Intercom encourages engineers to address small productivity issues to prevent larger problems.
  • [00:51:39] Balancing Innovation with Stability: Intercom’s conservative approach to engineering and how it helps maintain a stable product.
Key Takeaways
  • Meaningful Alarms: Ensure that all alarms are actionable and represent real or inevitable customer pain.
  • Consistent Architecture: A consistent software environment, like Intercom's Ruby on Rails monolith, simplifies on-call duties and allows for greater flexibility across teams.
  • Velocity Risk Framework: Managing technical debt by quantifying its impact on velocity helps prioritize the most impactful work.
  • Continuous Improvement: Regular reviews and continuous improvement are essential for maintaining a sustainable on-call system.
  • Product-Focused Engineering: At Intercom, the emphasis is on building products, not just writing code, ensuring that engineers are focused on delivering value.
Helpful Links
  • Intercom's Engineering Site
  • Brian's Twitter
  • Brian's LinkedIn
  • [Book Recommendation] Choice Theory: A New Psychology of Personal Freedom, William Glasser

Subscribe to Maintainable on:

  • Apple Podcasts
  • Spotify

Or search "Maintainable" wherever you stream your podcasts.

Keep up to date with the Maintainable Podcast by joining the newsletter.

...more
View all episodesView all episodes
Download on the App Store

MaintainableBy Robby Russell

  • 5
  • 5
  • 5
  • 5
  • 5

5

32 ratings


More shows like Maintainable

View all
Software Engineering Radio - the podcast for professional software developers by se-radio@computer.org

Software Engineering Radio - the podcast for professional software developers

272 Listeners

The Changelog: Software Development, Open Source by Changelog Media

The Changelog: Software Development, Open Source

282 Listeners

LINUX Unplugged by Jupiter Broadcasting

LINUX Unplugged

265 Listeners

The Bike Shed by thoughtbot

The Bike Shed

121 Listeners

Software Engineering Daily by Software Engineering Daily

Software Engineering Daily

626 Listeners

Soft Skills Engineering by Jamison Dance and Dave Smith

Soft Skills Engineering

271 Listeners

Syntax - Tasty Web Development Treats by Wes Bos & Scott Tolinski - Full Stack JavaScript Web Developers

Syntax - Tasty Web Development Treats

984 Listeners

REWORK by 37signals

REWORK

210 Listeners

Remote Ruby by Chris Oliver, Andrew Mason

Remote Ruby

34 Listeners

Code with Jason by Jason Swett

Code with Jason

14 Listeners

The Stack Overflow Podcast by The Stack Overflow Podcast

The Stack Overflow Podcast

64 Listeners

ACQ2 by Acquired by Ben Gilbert and David Rosenthal

ACQ2 by Acquired

209 Listeners

Code and the Coding Coders who Code it by Drew Bragg

Code and the Coding Coders who Code it

6 Listeners

Oxide and Friends by Oxide Computer Company

Oxide and Friends

47 Listeners

The Pragmatic Engineer by Gergely Orosz

The Pragmatic Engineer

52 Listeners