Engineering Enablement by DX

Tara Hernandez, the VP of Developer Productivity at MongoDB, joins the podcast to give an inside look at what the developer experience looks like at an organization that develops a database. Here, Tara shares what it looks like to develop, test, and release changes at MongoDB, while also providing insight into how the company invests in developer productivity more broadly.

Discussion points: 

• (0:57) What was going on at the time Tara joined 
• (4:37) Tara’s perspective on the buzz of platform engineering
• (7:38) What’s involved in building and testing a database
• (10:11) The development environment at MongoDB
• (13:14) How testing works
• (16:50) What the release process looks like
• (19:27) What goes into performance testing a release
• (21:31) MongoDB’s investment in engineering enablement 
• (22:39) Takeaways from working on databases
• (24:24) Affecting cultural change
• (26:40) Opportunities Tara’s team identified to change culture
• (29:12) Managing technical debt
• (33:06) MongoDB’s culture around developer experience 
• (34:59) Why Evergreen CI is open source

Mentions and links:
Follow Tara on LinkedIn or Twitter
Read more about MongoDB’s “Evergreen” Continuous Integration
Visit MongoDB’s engineering blog 

Max Kanat-Alexander, the Tech Lead for the Developer Productivity and Insights Team at LinkedIn, shares an inside look at LinkedIn’s metrics platform and how teams across the organization use it.

Discussion points: 

• (1:31) Why Max shares how his team is measuring productivity
• (3:20) Why some teams use metrics and some don’t 
• (6:03) The types of metrics Max’s team focuses on
• (12:59) The role of TPMs
• (17:05) How Max would measure productivity if he weren’t at LinkedIn
• (25:04) Surprises in how teams are using metrics at LinkedIn
• (31:27) The tooling required to enable metrics for teams to use
• (36:41) Qualitative versus quantitative metrics
• (40:39) Measuring code quality at Google 
• (46:16) Whether a centralized team should own measurement

Mentions and links:
Connect with Max on LinkedIn or Twitter
Read the article, Measuring Developer Productivity and Happiness at LinkedIn
Listen to the first interview with Max and his colleague Or Michael Berlowitz: Episode 23
Abi’s blog post on the Three-Bucket Framework for Engineering Metrics

Mike Fisher, the former CTO at Etsy, spearheaded a multi-year developer experience initiative aimed at improving developer happiness and efficiency during his time at Etsy. Here, he shares the story of that initiative, including the pillars of the program and the investment that went into it. Towards the end of the conversation, Mike also shares his perspective on measuring developer productivity.

Discussion points:

• (1:31) What was happening at Etsy when Mike joined 
• (4:08) The scaling challenges Etsy faced
• (6:08) Deciding on the term “developer experience” 
• (9:35) Whether developer experience is a new approach
• (11:24) The pillars of Etsy’s DevEx initiative 
• (15:49) Converting the length of time required for this initiative
• (18:11) The investment allocated to the initiative 
• (20:04) Talking about the ROI of devex initiatives 
• (22:50) Who was actually leading this work
• (24:37) Etsy’s experience with platform teams 
• (30:42) Advice for leaders championing DevEx initiatives
• (34:45) Framing the conversation about getting budget for a DevEx initiative
• (37:45) How leaders can address the efficiency conversation
• (42:00) Measuring productivity 
• (45:49) The “experiment velocity” metric 

‍

Mentions and links:
Follow Mike on LinkedIn or Twitter

Subscribe to Mike’s newsletter, Fish Food for Thought

Karl’s team at American Airlines were early adopters of Backstage, and in this episode he shares their journey of implementing and rolling out a developer portal. He also describes two of the extensions his team has built for their portal.

Discussion points:

• (1:24) Where the idea of building a developer portal came from
• (7:24) What the developer experience looked like before the portal 
• (10:41) Initiating the project
• (14:16) The decision to choose Backstage 
• (16:28) The V1 scope for the portal 
• (19:14) Getting adoption for the portal
• (23:35) Defining success for the portal’s adoption 
• (28:04) The ideal state for how developers will use the portal
• (30:56) Who should or shouldn’t invest in building a developer portal 
• (33:14) Custom extensions Karl’s team has developed for their portal
• (37:46) What’s difficult about developing a new plugin for the backstage platform

‍

Mentions and links:
Follow Karl on LinkedIn
The Runway platform at American Airlines
Read more on the engineering blog from American Airlines 

As product lead, Russ Nealis has been focused on introducing the discipline of product management in the Developer Foundations organization. This episode discusses the reasons why PMs are currently uncommon in platform organizations, examples of when having a PM has been helpful, and more.

Discussion points:

• (1:23) Russ’s role at Plaid 
• (2:49) Why platform product managers are uncommon
• (3:28) Backgrounds to look for when hiring a platform PM
• (4:58) Deciding whether to hire a platform PM
• (6:20) Signs that bringing in a Product Manager would be beneficial
• (9:16) How Russ personally became a platform PM
• (12:15) Whether a platform PM is a career path 
• (14:55) Articulating the business impact a platform PM has
• (18:56) Challenges Plaid’s platform team has faced without a PM  
• (19:19) Symptoms of a need for product management in an internal-facing team
• (30:15) Whether Twilio had platform PMs  
• (31:22) Example projects where PMs have been crucial
• (34:12) How the book “Ask Your Developer” influenced Twilio’s engineering culture 
• (36:13) Getting started with introducing a product management discipline to an organization 
• (38:33) Org structure and where platform PMs may report 
• (40:00) Career ladder for platform PM when reporting to engineering leadership
• (41:20) Being product-led or technology-led
• (43:14) How technical skills may help when in a platform PM role

‍

Mentions and links: 
Follow Russ on LinkedIn
Episode 7 with Will Larson - related to why it’s difficult to find Platform PMs
Episode 27 with Jean-Michel Lemieux - related to the percentage of investment that should be put towards platform investments
The Build Trap by Melissa Perri
Ask Your Developer by Jeff Lawson

In this deep-dive episode, Brian Scanlan, Principal Systems Engineer at Intercom, describes how the company’s on-call process works. He explains how the process started and key changes they’ve made over the years, including a new volunteer model, changes to compensation, and more.

Discussion points:

• (1:28) How on-call started at Intercom
• (10:11) Brian’s background and interest in being on-call
• (14:06) Getting engineers motivated to be on-call 
• (16:37) Challenges Intercom saw with on-call as it grew
• (19:53) Having too many people on-call
• (23:20) Having alarms that aren’t useful 
• (26:03) Recognizing uneven workload with compensation
• (27:22) Initiating changes to the on-call process 
• (30:08) Creating a volunteer model
• (33:02) Addressing concerns that volunteers wouldn’t take action on alarms 
• (34:40) Equitability in a volunteer model
• (36:36) Expectations of expertise for being on-call
• (40:56) How volunteers sign up 
• (44:15) The Incident Commander role 
• (46:19) Using code review for changes to alarms
• (50:02) On-call compensation 
• (52:50) Other approaches to compensating on-call
• (55:08) Whether other companies should compensate on-call
• (57:32) How Intercom’s on-call process compares to other companies 
• (1:00:46) Recent changes to the on-call process
• (1:04:13) Balancing responsiveness and burnout 
• (1:07:12) Signals for evaluating the on-call process 

‍

Mentions and links: 

• Follow Brian on LinkedIn or Twitter 
• Brian’s article: How we fixed our on call process to avoid engineer burnout
• Gergely Orosz’s On-Call Compensation 

Jack Li explains how his production engineering team rolled out a new incident review process, how they’ve made the case for investing in reliability, and specific tools his team has built to improve reliability.

—

Discussion points:

• (1:25) How Jack became interested in reliability 
• (3:24) Where the Instagram Reels team fits into the broader organization
• (4:05) What Jack’s team focuses on
• (4:55) The role of production engineering at Instagram versus Shopify 
• (8:32) The essence of DevOps
• (10:44) Pros and cons of having product-focused teams
• (13:35) How Jack’s team defines and tracks quality
• (15:46) Signals the team monitors outside of systems 
• (18:10) Revamping Instagram Reel’s incident management process
• (19:46) Making the case for improving the incident review process
• (28:10) How their incident review process works
• (31:55) The roles involved in an incident review 
• (33:40) The value of having incident reviews
• (35:55) Why leaders should be part of incident reviews 
• (38:34) Why Jack’s team builds tools for driving reliability goals
• (40:06) The types of tools Jack’s team focuses on 
• (43:09) What a merge queue is and why it was built at Shopify
• (51:20) Using a Slack bot for ‘failed build’ alerts
• (52:32) When a company should consider implementing a merge queue

—

Mentions and links:
Follow Jack on LinkedIn
Jack’s article from his time on Shopify about their Merge Queue
Jack’s talk on Shopify’s Merge Queue at GitHub Universe 2019

Nathen Harvey, who leads DORA at Google, explains what DORA is, how it has evolved in recent years, the common challenges companies face as they adopt DORA metrics, and where the program may be heading in the future.

—

Discussion points:

(1:48) What DORA is today and how it exists within Google

(3:37) The vision for Google and DORA coming together

(5:20) How the DORA research program works

(7:53) Who participates in the DORA survey

(9:28) How the industry benchmarks are identified 

(11:05) How the reports have evolved over recent years

(13:55) How reliability is measured 

(15:19) Why the 2022 report didn’t have an Elite category

(17:11) The new Slowing, Flowing, and Retiring clusters

(19:25) How to think about applying the benchmarks

(20:45) Challenges with how DORA metrics are used

(24:02) Why comparing teams’ DORA metrics is an antipattern 

(26:18) Why ‘industry’ doesn’t matter when comparing organizations to benchmarks 

(29:32) Moving beyond DORA metrics to optimize organizational performance 

(30:56) Defining different DORA metrics

(36:27) Measuring deployment frequency at the team level, not the organizational level

(38:29) The capabilities: there’s more to DORA than the four metrics 

(43:09) How DORA and SPACE are related

(47:58) DORA’s capabilities assessment tool 

(49:26) Where DORA is heading

—

Mentions and links:

Follow Nathen on LinkedIn or Twitter

Engineering Enablement episode with Dr. Nicole Forsgren

2022 State of DevOps report  

Bryan Finster’s How to Use & Abuse DORA Metrics (and Abi’s summary of the paper) 

Engineering Enablement episode with Dr. Margaret-Anne Storey

Join the DORA community for discussion and events: dora.community 

This week's guest is Dr. Margaret-Anne Storey, who goes by the name Peggy. Peggy is a professor of Computer Science at the University of Victoria, the Chief Scientist at DX, and co-author of the SPACE Framework, which is the topic of focus in this episode. Today’s conversation discusses what the SPACE framework is and what went into developing the metrics and categories. Peggy also shares where she sees this line of research heading next.  

—

Discussion points:
 
(1:29) Peggy’s background 
(4:01) What the SPACE framework is 
(5:55) Why the researchers came together for this paper
(7:27) The process of writing this paper
(9:52) How the SPACE categories and acronym emerged 
(11:50) The authors’ intention for how this framework would be received
(13:26) Finding a definition for what developer productivity is
(17:08) The metrics included in the SPACE framework 
(24:48) How SPACE is different from DORA
(26:17) Why lines of code and number of pull requests were included as example metrics
(27:14) What Peggy is thinking about next

—

Mentions and links:
Where to find Peggy: Twitter, Website
The SPACE of Developer Productivity: There’s more to it than you think by Nicole Forsgren, Margaret-Anne Storey, Chandra Madilla, Thomas Zimmerman, Brian Houck, and Jenna Butler
Abi’s summary of the SPACE paper
Peggy’s talk, What Does Productivity Actually Mean for Developers? 

This week’s guest is Jeremiah Lee, who was previously a manager at Stripe and product manager at Spotify. This conversation focuses on org structure, and specifically Jeremiah’s experience with the popular squad model from Spotify. Jeremiah provides the backstory on where the model came from, what parts of the model were a challenge, and advice for leaders either already adopting the model or considering doing so.

Discussion points:

(1:40) What the Spotify model is

(4:39) Jeremiah’s impression of the Spotify model as he joined the company

(7:29) Spotify’s progress in adopting the model as Jeremiah joined

(9:55) Challenges with matrix management

(12:02) The role of engineering managers 

(14:40) What the model was designed to solve 

(15:54) Good autonomy versus toxic autonomy 

(18:51) How Agile coaches were used at Spotify 

(21:39) Advice for teams who are struggling to implement the Spotify model

(24:50) Advice for leaders who are starting to think about org design

(27:30) How Stripe approached org structure 

(30:26) How org structure affects a platform team’s work 

(33:32) Tracking engineering org structures 

(36:02) Why the squad model became so popular

(39:37) What the original authors may have felt about the popularity of the model

Mentions and links: 

Follow Jeremiah on LinkedIn

Jeremiah’s Spotify’s Failed #SquadGoals

The original whitepaper on the Spotify model: Scaling Agile at Spotify

Team Topologies by Matthew Skelton and Manuel Pais

Essential Scrum by Kenneth S. Rubin