Maintainable

Brian Scanlan: Improving Oncall Support with Meaningful Alarms


Listen Later

In this episode of Maintainable, Robby is joined by Brian Scanlan, Principal Systems Engineer at Intercom. Brian shares insights into how Intercom has successfully implemented a volunteer-led on-call support system, emphasizing the importance of meaningful alarms and a consistent software environment.

Summary of Topics
  • [00:05:32] Introduction to the Guest's Background: Brian Scanlan discusses his role at Intercom and his experience with on-call practices.
  • [00:15:10] Developing a Volunteer-Led On-Call System: Brian explains how Intercom transitioned to a volunteer-led on-call system, ensuring that alarms are meaningful and actionable.
  • [00:20:00] The Role of Consistent Software Architecture: The impact of Intercom’s monolithic Ruby on Rails architecture on simplifying on-call duties.
  • [00:29:46] Managing Technical Debt as Velocity Risks: Brian describes how Intercom manages technical debt through a velocity risk framework.
  • [00:38:45] Improving Deployment Processes: The evolution of Intercom’s deployment processes, reducing the time from merge to production.
  • [00:43:32] Treating Internal Tools as a Product: The importance of treating internal tools with the same care as external products, focusing on usability and impact.
  • [00:50:56] Encouraging Small Wins in Productivity: How Intercom encourages engineers to address small productivity issues to prevent larger problems.
  • [00:51:39] Balancing Innovation with Stability: Intercom’s conservative approach to engineering and how it helps maintain a stable product.
Key Takeaways
  • Meaningful Alarms: Ensure that all alarms are actionable and represent real or inevitable customer pain.
  • Consistent Architecture: A consistent software environment, like Intercom's Ruby on Rails monolith, simplifies on-call duties and allows for greater flexibility across teams.
  • Velocity Risk Framework: Managing technical debt by quantifying its impact on velocity helps prioritize the most impactful work.
  • Continuous Improvement: Regular reviews and continuous improvement are essential for maintaining a sustainable on-call system.
  • Product-Focused Engineering: At Intercom, the emphasis is on building products, not just writing code, ensuring that engineers are focused on delivering value.
Helpful Links
  • Intercom's Engineering Site
  • Brian's Twitter
  • Brian's LinkedIn
  • [Book Recommendation] Choice Theory: A New Psychology of Personal Freedom, William Glasser

Subscribe to Maintainable on:

  • Apple Podcasts
  • Spotify

Or search "Maintainable" wherever you stream your podcasts.

Keep up to date with the Maintainable Podcast by joining the newsletter.

...more
View all episodesView all episodes
Download on the App Store

MaintainableBy Robby Russell

  • 5
  • 5
  • 5
  • 5
  • 5

5

30 ratings


More shows like Maintainable

View all
Hanselminutes with Scott Hanselman by Scott Hanselman

Hanselminutes with Scott Hanselman

378 Listeners

Software Engineering Radio - the podcast for professional software developers by se-radio@computer.org

Software Engineering Radio - the podcast for professional software developers

262 Listeners

The Changelog: Software Development, Open Source by Changelog Media

The Changelog: Software Development, Open Source

286 Listeners

Thoughtworks Technology Podcast by Thoughtworks

Thoughtworks Technology Podcast

42 Listeners

Talk Python To Me by Michael Kennedy

Talk Python To Me

584 Listeners

Software Engineering Daily by Software Engineering Daily

Software Engineering Daily

630 Listeners

Soft Skills Engineering by Jamison Dance and Dave Smith

Soft Skills Engineering

272 Listeners

Go Time: Golang, Software Engineering by Changelog Media

Go Time: Golang, Software Engineering

128 Listeners

JS Party: JavaScript, CSS, Web Development by Changelog Media

JS Party: JavaScript, CSS, Web Development

91 Listeners

Syntax - Tasty Web Development Treats by Wes Bos & Scott Tolinski - Full Stack JavaScript Web Developers

Syntax - Tasty Web Development Treats

987 Listeners

REWORK by 37signals

REWORK

211 Listeners

CoRecursive: Coding Stories by Adam Gordon Bell - Software Developer

CoRecursive: Coding Stories

185 Listeners

The Stack Overflow Podcast by The Stack Overflow Podcast

The Stack Overflow Podcast

63 Listeners

Oxide and Friends by Oxide Computer Company

Oxide and Friends

47 Listeners

The Pragmatic Engineer by Gergely Orosz

The Pragmatic Engineer

48 Listeners