Maintainable

Brian Scanlan: Improving Oncall Support with Meaningful Alarms


Listen Later

In this episode of Maintainable, Robby is joined by Brian Scanlan, Principal Systems Engineer at Intercom. Brian shares insights into how Intercom has successfully implemented a volunteer-led on-call support system, emphasizing the importance of meaningful alarms and a consistent software environment.

Summary of Topics
  • [00:05:32] Introduction to the Guest's Background: Brian Scanlan discusses his role at Intercom and his experience with on-call practices.
  • [00:15:10] Developing a Volunteer-Led On-Call System: Brian explains how Intercom transitioned to a volunteer-led on-call system, ensuring that alarms are meaningful and actionable.
  • [00:20:00] The Role of Consistent Software Architecture: The impact of Intercom’s monolithic Ruby on Rails architecture on simplifying on-call duties.
  • [00:29:46] Managing Technical Debt as Velocity Risks: Brian describes how Intercom manages technical debt through a velocity risk framework.
  • [00:38:45] Improving Deployment Processes: The evolution of Intercom’s deployment processes, reducing the time from merge to production.
  • [00:43:32] Treating Internal Tools as a Product: The importance of treating internal tools with the same care as external products, focusing on usability and impact.
  • [00:50:56] Encouraging Small Wins in Productivity: How Intercom encourages engineers to address small productivity issues to prevent larger problems.
  • [00:51:39] Balancing Innovation with Stability: Intercom’s conservative approach to engineering and how it helps maintain a stable product.
Key Takeaways
  • Meaningful Alarms: Ensure that all alarms are actionable and represent real or inevitable customer pain.
  • Consistent Architecture: A consistent software environment, like Intercom's Ruby on Rails monolith, simplifies on-call duties and allows for greater flexibility across teams.
  • Velocity Risk Framework: Managing technical debt by quantifying its impact on velocity helps prioritize the most impactful work.
  • Continuous Improvement: Regular reviews and continuous improvement are essential for maintaining a sustainable on-call system.
  • Product-Focused Engineering: At Intercom, the emphasis is on building products, not just writing code, ensuring that engineers are focused on delivering value.
Helpful Links
  • Intercom's Engineering Site
  • Brian's Twitter
  • Brian's LinkedIn
  • [Book Recommendation] Choice Theory: A New Psychology of Personal Freedom, William Glasser

Subscribe to Maintainable on:

  • Apple Podcasts
  • Spotify

Or search "Maintainable" wherever you stream your podcasts.

Keep up to date with the Maintainable Podcast by joining the newsletter.

...more
View all episodesView all episodes
Download on the App Store

MaintainableBy Robby Russell

  • 5
  • 5
  • 5
  • 5
  • 5

5

32 ratings


More shows like Maintainable

View all
99% Invisible by Roman Mars

99% Invisible

26,174 Listeners

Hanselminutes with Scott Hanselman by Scott Hanselman

Hanselminutes with Scott Hanselman

380 Listeners

The Changelog: Software Development, Open Source by Changelog Media

The Changelog: Software Development, Open Source

291 Listeners

Software Engineering Daily by Software Engineering Daily

Software Engineering Daily

624 Listeners

Y Combinator Startup Podcast by Y Combinator

Y Combinator Startup Podcast

232 Listeners

Syntax - Tasty Web Development Treats by Wes Bos & Scott Tolinski - Full Stack JavaScript Web Developers

Syntax - Tasty Web Development Treats

987 Listeners

REWORK by 37signals

REWORK

210 Listeners

The Indicator from Planet Money by NPR

The Indicator from Planet Money

9,516 Listeners

Tech Brew Ride Home by Morning Brew

Tech Brew Ride Home

968 Listeners

Remote Ruby by Chris Oliver, Andrew Mason

Remote Ruby

35 Listeners

.NET Rocks! by Carl Franklin and Richard Campbell

.NET Rocks!

245 Listeners

Short Wave by NPR

Short Wave

6,362 Listeners

The Real Python Podcast by Real Python

The Real Python Podcast

141 Listeners

Hard Fork by The New York Times

Hard Fork

5,479 Listeners

The AI Daily Brief: Artificial Intelligence News and Analysis by Nathaniel Whittemore

The AI Daily Brief: Artificial Intelligence News and Analysis

559 Listeners