Engineering Enablement by DX

This week we’re joined by Preeti Kota, the Head of Engineering for Compass at Atlassian. Preeti walks us through Atlassian’s journey with developer experience: including how they measure DevEx, and how they drive improvements through efforts at both the organization and team levels. Preeti also talks about how this journey has led to the development of Atlassian’s newly released internal developer portal, Compass.

Mentions and links:

• Learn about Compass, Atlassian’s newly released internal developer portal  
• Connect with Preeti on LinkedIn
• Atlassian’s CTO, Rajeev Rajan, on the key to unlocking developer productivity 

Discussion points:

• (1:43) Where Atlassian’s journey with developer experience began
• (5:36) Who is championing the focus on DevEx at Atlassian
• (9:30) How the company arrived at their level of investment in DevEx
• (13:47) Defining developer experience
• (18:19) How the program for improving developer productivity is structured
• (21:19) The Developer Productivity Champions group
• (23:53) Two metrics in focus: Self-serve documentation and self-serve dependency maintenance
• (25:56) How Atlassian surveys developers 
• (29:59) Types of projects the centralized teams tackle 
• (31:19) Getting buy-in for investing 10% time toward DevEx projects
• (33:13) How leaders get teams to feel they have permission to invest 10% of their time toward DevEx projects
• (36:19) The backstory behind Compass, Atlassian’s new product 
• (38:10) What Compass is, who it’s for, and how it is unique

This week we’re joined by Mark Côté, who leads the Developer Infrastructure organization at Shopify, to learn about their developer survey program. Mark shares what went into designing and running the survey, what they’ve done to drive participation rates higher, and how they interpret their data.

Mentions and links:

• Follow Mark on LinkedIn
• Read Shopify’s engineering blog

Discussion points:

• (1:32) Starting the survey
• (3:20) How the survey has evolved
• (4:22) Three types of information gleaned from the survey
• (7:37) Designing and running the survey
• (12:28) Participation rates
• (15:12) Why there's an increase of interest in the results at Shopify
• (17:42) What's affecting participation rates
• (23:03) Selecting survey questions
• (27:01) Refining survey questions
• (28:54) Survey length
• (30:56) Analyzing the results 
• (33:31) How the data is stored and shared
• (35:56) Sending targeted surveys to the right developers 
• (37:40) Using the results as a Developer Acceleration organization
• (39:29) Confidence in the data
• (41:27) The value of a developer survey

Thomas Khalil, Head of Platform and SRE at Trivago, describes how the teams reporting into him are structured, the tactics they’re using to increase awareness of their work, and how they demonstrate their impact. 

Discussion points: 

• (1:17) The pillars of the Central Platform organization
• (2:18) The organization’s focus on time to market and efficiency
• (3:09) The differences in developer experience between teams
• (4:37) Deciding whether to consolidate services
• (5:57) How platform, developer experience, observability, and SRE teams interact
• (8:40) How these problems were being tackled previously 
• (10:09) A failed attempt at rolling out Backstage 
• (13:48) How SRE squads are organized
• (15:39) How to motivate platform teams 
• (17:23) Demonstrating the impact of the organization
• (18:42) How the data is collected
• (22:32) How they’re increasing awareness for their work 
• (23:42) The DevEx pillar
• (25:46) How the DevEx roadshow will work 
• (27:56) How DORA metrics fit into their measurement program 

Mentions and links:
Connect with Thomas on LinkedIn

This week’s guest is Jenny McClain, who leads R&D Team Enablement at Toast. Jenny’s team focuses on enabling individual teams at Toast to drive their own productivity improvements, and this conversation dives into how they tackle this problem.

Discussion points: 

• (1:19) How the R&D Enablement team works
• (2:50) Why the team was formed
• (4:31) The types of work the team focuses on
• (7:31) Identifying the problems this team would solve
• (11:23) How team embeds work
• (17:19) The learning resources the team develops and maintains
• (20:55) Who creates and maintains the learning resources
• (23:10) How enablement stays connected with teams at scale
• (25:51) How the team plans work with qualitative and quantitative measures 
• (29:37) Formats for sharing knowledge between teams
• (33:05) How other companies can think about the enablement function
• (37:40) Enablement as a career path

Mentions and links:
Follow Jenny on LinkedIn
Tuckman’s stages of group development

Working Agreements template from Steve Sobel, Director of Engineering at Toast - one of the resources featured in Toast’s Team Health Toolkit 

This week we’re joined by Ciera Jaspan and Collin Green, who lead the Engineering Productivity Research team at Google. Ciera and Collin have written several papers from studies they’ve conducted, and this discussion covers the insights from their research as well as their work more broadly at Google. 

Discussion points:

• (1:19) About the Engineering Productivity Research team
• (3:57) How the team interacts with the rest of the organization
• (5:58) The different backgrounds included on the team
• (13:11) How Google measures developer productivity
• (18:54) Evaluating discrepancies between qualitative and quantitative data 
• (28:40) Google’s quarterly developer survey
• (32:02) Distributing survey results back to the organization 
• (40:25) Misunderstandings about surveys
• (43:51) Ciera and Collin’s paper on why measuring productivity is difficult
• (50:35) Reductionist metrics for measuring productivity
• (55:26) Examples of other fields that have struggled with measurement
• (59:00) Google’s study on measuring technical debt
• (1:08:05) Human judgment in measurement

Mentions and links:
Follow Ciera and Collin on LinkedIn
A Human-Centered Approach to Measuring Developer Productivity - Paper, Abi’s summary
Enabling the Study of Software Development with Cross-Tool Logs - Paper
Defining, Measuring, and Managing Tech Debt - Paper, Abi’s summary
Google’s Goals, Signals, Metrics framework - Paper, Abi’s summary

This week we’re joined by Jason Kennedy, Senior Engineering Manager of Developer Experience at One Medical. Jason’s team takes a uniquely customer-driven approach to improving the developer experience, and in this episode he describes their philosophy and how it works in practice. Jason explains how they shadow developers, how they run surveys, and more.

Discussion points:

• (1:02) Renaming from Engineering Efficiency to Engineering Experience
• (4:17) How Platform and DevEx teams differ 
• (5:38) How One Medical’s approach to customer experience inspires this team’s work
• (7:01) Mapping out the developer journey
• (11:14) Jason’s career transition from VPE to a line manager role
• (14:14) Challenges some companies face with getting buy-in for a DevEx team
• (16:22) Taking a customer service approach to DevEx
• (19:12) Jason’s experience with DORA metrics
• (22:19) Lessons learned about ownership
• (24:18) The “Gemba” practice used at One Medical 
• (28:02) How information from the Gemba practice is stored
• (30:59) Using weekly polls to surface pain points
• (34:03) Tracking trends in the poll
• (35:00) Using a quarterly NPS survey for overall sentiment
• (37:08) How sentiment is measured and evaluated
• (41:44) The biggest challenges with surveys 

Mentions and links:
Follow Jason on LinkedIn
Listen to the podcast episode with Jasmine James

Book about Disney: Be Our Guest

Matthew and Luke lead Extend’s Developer Experience team, a team that has approached their work in a way that is more forward-thinking than most. In this episode, they cover how they deliver impact at multiple levels of the organization, their journey with productivity metrics, and how they’ve made DevEx a C-level concern. 

Discussion points:

• (1:40) How the DevEx team started and where it fits at Extend
• (5:08) Tradeoffs of DevEx reporting into Platform 
• (6:40) The mandate and tasks they focus on
• (12:07) The impact of learning and development efforts
• (16:33) How to drive team-level improvements 
• (18:44) Why developer experience is becoming more prevalent
• (26:17) How they made DevEx a C-level concern
• (30:27) Their journey with productivity metrics 
• (33:10) Advice for presenting DevEx data to executives 
• (34:52) The team’s experience using git metrics tools
• (48:30) Being rigorous in leveraging metrics 

Mentions and links:
Connect with Matthew and Luke on LinkedIn
Other podcasts mentioned: Manuel Pais; Peloton’s DevEx survey

Manuel Pais delves into one of the concepts covered in his book “Team Topologies”: platform and enabling work. Manuel shares how he views the strategy behind when and how to invest in platform or enabling work. This conversation also goes into each type of work in more detail, covering topics such as measuring cognitive load and where platform engineering may be heading in the future. 

• (2:13) How enabling teams and platform teams are different 
• (10:28) What it looks like for a team to own both platform and enabling work 
• (17:04) How to deliver enabling work in an organization
• (22:28) Whether enabling teams should be temporary
• (30:10) Platform team anti-patterns
• (47:10) Measuring cognitive load

Thansha Sadacharam, who leads Tech Learning and Insights at Peloton walks us through the journey of building the company’s developer experience survey. She shares what went into the survey’s design, rollout, and maintenance, as well as the different teams involved.

Discussion points: 

• (1:19) Where the idea for running a developer survey originated
• (6:36) Advice for other leaders getting buy-in for these initiatives
• (11:27) The first steps in designing the survey
• (18:21) How the survey incorporated benchmarking
• (20:30) Measuring developer satisfaction
• (22:37) Refining the question items 
• (25:50) How long the survey was
• (26:50) What was involved in trimming the questions 
• (29:28) Writing survey questions 
• (33:12) How much time was spent developing the survey
• (35:19) The communication plan for launching the survey
• (42:05) Driving participation rates  
• (45:21) Sampling and how often surveys are being sent 
• (49:21) How the information was presented 
• (54:10) Feeling nervous about sending out surveys 

Mentions and links

Follow Thansha on LinkedIn

In this episode, Abi is interviewed by Laura Tacho about the new paper he co-authored with Dr. Nicole Forsgren, Dr. Margaret-Anne Storey, and Dr. Michaela Greiler. Abi and Laura discuss the pitfalls of some of the common metrics organizations use, and how the new paper builds on prior frameworks such as DORA and SPACE to offer a new approach to measuring and improving developer productivity.

Discussion topics:

• (2:20) Laura’s background
• (3:59) Laura’s view on git metrics
• (11:05) What developer experience (DevEx) is 
• (14:37) How the authors came together for this paper 
• (18:55) How DORA and SPACE are different
• (22:38) Limitations of DORA metrics 
• (24:43) Employing the DORA metrics at GitHub
• (27:47) What the SPACE framework is
• (30:44) Whether to use DORA or SPACE or both
• (33:54) Limitations of the SPACE framework
• (37:29) The need for a new approach 
• (38:46) What the new DevEx paper solves 
• (40:13) The three dimensions of developer experience 
• (40:54) Flow state 
• (43:10) Feedback loops
• (43:52) Cognitive load 
• (44:51) Why developer sentiment matters
• (47:58) Using both perceptual and workflow measures
• (50:59) Examples of perceptual and workflow measures 
• (54:05) How to collect metrics 
• (59:47) How other companies are measuring and improving developer experience
• (01:02:56) Advice for earlier-stage or growing organizations

Resources for learning more about the DevEx framework:

Read the new paper on ACM Queue

Read Abi’s announcement about the new paper 

Read how top companies measure developer productivity 

Connect with Abi and Laura 

Sign up for Laura’s course, Measuring Development Team Performance

Connect with Laura on LinkedIn or Twitter

Connect with Abi on LinkedIn or Twitter