Sign up to save your podcasts
Or
Share Chapter 9: Apply the Changes and Repeat
Share to email
Share to Facebook
Share to X
Share to email
Share to Facebook
Share to X
Or
Chapter 9: Apply the Changes and Repeat
CHAPTER 10: Apply the Changes, Analyze the Results, and Repeat
I have no special talent. I am only passionately curious.
- Albert Einstein
A few years ago, the designer and engineer Peter Skillman held a competition to build the tallest possible structure in 20-minutes with the following items:
The marshmallow had to be placed on top to win the competition, and the entire structure had to remain motionless for three seconds.
After running this challenge with more than 2,000 people from various backgrounds (from some of the top universities like Stanford, the University of California, and more to boot), Skillman was surprised by the results.
Business school students performed the worst. What was even more shocking was that the best performing group were not engineering prodigies or physics majors. They were kindergarteners.
What happened?
The business students were talking and thinking strategically. They examined the materials. They brainstormed ideas, debated the best options, asked savvy questions, and honed in on the most promising ideas. It was professional, rational, and intelligent. The process resulted in a shared decision to pursue one strategy. Then, with five minutes left, they divided up the tasks and started to build.
The kindergartners took a different approach. Instead of strategizing, analyzing hypotheticals, debating, proposing, and asking questions, they simply start building. They rarely spoke to each other, and when they did, it was in short bursts, “Here! Now, here!”
By the time the 20 minutes were up, they had experimented with 8 to 10 different structures before landing on the most stable one.
Figure 10.1 The kindergartners in Skillman’s design challenge
In dozens of trials, kids built structures up to 26 inches tall, while business school students built structures less than 10 inches (on average). Kindergartners also beat out teams of lawyers (who built towers that averaged 15 inches), as well as teams of CEOs (22 inches).
The lesson here: multiple iterations usually beat a commitment to the first idea and making it work. It’s best to learn by doing.
This is an important principle to apply to user onboarding. From my experience, most teams do big launches to improve their user onboarding. Then, they iterate just once or twice a year. It’s much better to analyze the results, reiterate and implement changes quickly than to overthink and strategize about it for months.
We’re now at the final step of the EUREKA framework. It’s time to apply changes to your onboarding experience. In this chapter, we’ll discuss how to run rapid-fire experiments that will produce compounding wins.
The Growth Process
Companies that grow the fastest are also the ones that learn the quickest. They use this as a massive competitive advantage.
That’s where our “Triple A” sprint comes in. It focuses on rapidly identifying problems, building solutions, and measuring impact. The process follows a one-month sprint cycle to identify and deliver an improvement to a product. It consists of three A’s:
Analyze: Identify inputs that drive the outcomes you want for your business
Ask: Answer these three questions:
Where do you want to go?
Which levers can you pull to achieve the desired outcome?
Which inputs should you invest in?
Act: Choose an input and implement it.
Then, you go through the Triple A cycle again, considering any learnings from the previous iterations to future ones.
I use this process when working with ProductLed clients. But you can use any iterative growth process such as Sean Ellis’ high-tempo four-step process, Brian Balfour’s six-step growth machine process or Growth Tribe’s G.R.O.W.S. process. You’ll notice that most of the work you’ve completed for the EUREKA framework contributes to the Analyze and Ask steps of the Triple A process.
By now, the majority of the ideas to improve user onboarding should have emerged from the previous steps in the EUREKA framework. Those steps are focused on holistically analyzing your data and gathering insights as a cross-functional team.
Prioritizing Onboarding Improvements
From my experience, the problem in the growth process doesn’t lie with Ideation but in the Prioritization of growth ideas. For that, I use the Action Priority Matrix:
Here’s what this means:
Split Testing Growth Ideas
Ideally, you want to split test any onboarding changes. Let’s say you want to validate that your Straight-Line Onboarding performs better than your current onboarding experience. You can send half of your new users through the new onboarding and the other half through your current process. After a few weeks, compare the week-to-week retention rate to determine if the retention curve has improved.
At this stage, it’s critical to iterate quickly. If you’re stuck at this step due to engineering bandwidth, consider tools like Appcues, Pendo, and UserPilot to implement Product Bumpers. Userlist, Intercom or Drift can enact conversational bumpers.
Socializing Learnings
Another step that teams often miss is ensuring the learnings from each growth experiment are shared across the whole organization. Since changes in the user onboarding impact so many teams, it’s really important to share the process, progress, and learnings with them.
Sharing with teams needs to become a habit.
Doing this will accomplish a few things. First, it excites your teammates (and the executive team) about onboarding. It’s also a great way to create teamwide empathy to remind everyone that their work has a real impact on everyday lives. Finally, sharing the successes and impact of onboarding will ensure that you can continue to dedicate company resources to the project.
How you communicate this is entirely up to you, but here are a few things to try:
Beyond The Initial Onboarding
At this point, I want to stress once again that improving user onboarding is a continuous process. As your product evolves to the market it serves, the user onboarding experience must adapt to those product changes.
To that point, the process of onboarding users never ends.
As new users complete the initial onboarding and become regular users, your onboarding team can (and should) help them adopt new capabilities and use cases of the product.
You can apply the same EUREKA process to various steps in the user journey. Here are some common places to consider:
Different entry points to your product will require a different onboarding experience. The journey and knowledge of someone who signed up from an ad will likely be completely different from someone who signed up through a Google search or a webinar campaign.
Different Customer Jobs will also require personalized onboarding.
Invited users should be treated differently. You’ll need to give them more information about why they’re being invited and why they should use your product.
New UI and product changes for existing users should be communicated thoughtfully. You’ll find the most engaged users will be the most resistant to change. How can you implement the principles you’ve learned in the EUREKA framework to help them adopt those new changes?
Generally speaking, after the initial onboarding is completed, you can onboard users to go deeper or wider with your product.
Going Deeper
First, go deeper by educating new customers about advanced configurations and features to help them do a Customer Job more effectively.
For example, for sales executives who signed up for Calendly to meetings with prospects, the First Strike occurs when a prospect schedules a meeting using their Calendly link. For the next onboarding iteration, you could inform them about advanced features that would help them close more deals, such as:
Going Wider
The other option is to go wider by introducing solutions to different problems that your customers face.
For example, there are many reasons why people sign up for Hubspot:
Let’s say you know that a new customer signed up for Hubspot specifically to generate leads with landing pages. After achieving that objective, you can educate them on using social media to drive traffic to that landing page. If they have a sales team, you could also let them know about the benefits of subscribing to Hubspot Sales along with Hubspot Marketing.
What’s Next?
Before we wrap up, some final thoughts. So far, I’ve covered the steps to improve your user onboarding without any assistance from anyone in your team. But sometimes it’s best to have some help. Next, we’ll look at how sales and high-touch support fit into the user onboarding experience.
View all episodes
By Ramli JohnCHAPTER 10: Apply the Changes, Analyze the Results, and Repeat
I have no special talent. I am only passionately curious.
- Albert Einstein
A few years ago, the designer and engineer Peter Skillman held a competition to build the tallest possible structure in 20-minutes with the following items:
The marshmallow had to be placed on top to win the competition, and the entire structure had to remain motionless for three seconds.
After running this challenge with more than 2,000 people from various backgrounds (from some of the top universities like Stanford, the University of California, and more to boot), Skillman was surprised by the results.
Business school students performed the worst. What was even more shocking was that the best performing group were not engineering prodigies or physics majors. They were kindergarteners.
What happened?
The business students were talking and thinking strategically. They examined the materials. They brainstormed ideas, debated the best options, asked savvy questions, and honed in on the most promising ideas. It was professional, rational, and intelligent. The process resulted in a shared decision to pursue one strategy. Then, with five minutes left, they divided up the tasks and started to build.
The kindergartners took a different approach. Instead of strategizing, analyzing hypotheticals, debating, proposing, and asking questions, they simply start building. They rarely spoke to each other, and when they did, it was in short bursts, “Here! Now, here!”
By the time the 20 minutes were up, they had experimented with 8 to 10 different structures before landing on the most stable one.
Figure 10.1 The kindergartners in Skillman’s design challenge
In dozens of trials, kids built structures up to 26 inches tall, while business school students built structures less than 10 inches (on average). Kindergartners also beat out teams of lawyers (who built towers that averaged 15 inches), as well as teams of CEOs (22 inches).
The lesson here: multiple iterations usually beat a commitment to the first idea and making it work. It’s best to learn by doing.
This is an important principle to apply to user onboarding. From my experience, most teams do big launches to improve their user onboarding. Then, they iterate just once or twice a year. It’s much better to analyze the results, reiterate and implement changes quickly than to overthink and strategize about it for months.
We’re now at the final step of the EUREKA framework. It’s time to apply changes to your onboarding experience. In this chapter, we’ll discuss how to run rapid-fire experiments that will produce compounding wins.
The Growth Process
Companies that grow the fastest are also the ones that learn the quickest. They use this as a massive competitive advantage.
That’s where our “Triple A” sprint comes in. It focuses on rapidly identifying problems, building solutions, and measuring impact. The process follows a one-month sprint cycle to identify and deliver an improvement to a product. It consists of three A’s:
Analyze: Identify inputs that drive the outcomes you want for your business
Ask: Answer these three questions:
Where do you want to go?
Which levers can you pull to achieve the desired outcome?
Which inputs should you invest in?
Act: Choose an input and implement it.
Then, you go through the Triple A cycle again, considering any learnings from the previous iterations to future ones.
I use this process when working with ProductLed clients. But you can use any iterative growth process such as Sean Ellis’ high-tempo four-step process, Brian Balfour’s six-step growth machine process or Growth Tribe’s G.R.O.W.S. process. You’ll notice that most of the work you’ve completed for the EUREKA framework contributes to the Analyze and Ask steps of the Triple A process.
By now, the majority of the ideas to improve user onboarding should have emerged from the previous steps in the EUREKA framework. Those steps are focused on holistically analyzing your data and gathering insights as a cross-functional team.
Prioritizing Onboarding Improvements
From my experience, the problem in the growth process doesn’t lie with Ideation but in the Prioritization of growth ideas. For that, I use the Action Priority Matrix:
Here’s what this means:
Split Testing Growth Ideas
Ideally, you want to split test any onboarding changes. Let’s say you want to validate that your Straight-Line Onboarding performs better than your current onboarding experience. You can send half of your new users through the new onboarding and the other half through your current process. After a few weeks, compare the week-to-week retention rate to determine if the retention curve has improved.
At this stage, it’s critical to iterate quickly. If you’re stuck at this step due to engineering bandwidth, consider tools like Appcues, Pendo, and UserPilot to implement Product Bumpers. Userlist, Intercom or Drift can enact conversational bumpers.
Socializing Learnings
Another step that teams often miss is ensuring the learnings from each growth experiment are shared across the whole organization. Since changes in the user onboarding impact so many teams, it’s really important to share the process, progress, and learnings with them.
Sharing with teams needs to become a habit.
Doing this will accomplish a few things. First, it excites your teammates (and the executive team) about onboarding. It’s also a great way to create teamwide empathy to remind everyone that their work has a real impact on everyday lives. Finally, sharing the successes and impact of onboarding will ensure that you can continue to dedicate company resources to the project.
How you communicate this is entirely up to you, but here are a few things to try:
Beyond The Initial Onboarding
At this point, I want to stress once again that improving user onboarding is a continuous process. As your product evolves to the market it serves, the user onboarding experience must adapt to those product changes.
To that point, the process of onboarding users never ends.
As new users complete the initial onboarding and become regular users, your onboarding team can (and should) help them adopt new capabilities and use cases of the product.
You can apply the same EUREKA process to various steps in the user journey. Here are some common places to consider:
Different entry points to your product will require a different onboarding experience. The journey and knowledge of someone who signed up from an ad will likely be completely different from someone who signed up through a Google search or a webinar campaign.
Different Customer Jobs will also require personalized onboarding.
Invited users should be treated differently. You’ll need to give them more information about why they’re being invited and why they should use your product.
New UI and product changes for existing users should be communicated thoughtfully. You’ll find the most engaged users will be the most resistant to change. How can you implement the principles you’ve learned in the EUREKA framework to help them adopt those new changes?
Generally speaking, after the initial onboarding is completed, you can onboard users to go deeper or wider with your product.
Going Deeper
First, go deeper by educating new customers about advanced configurations and features to help them do a Customer Job more effectively.
For example, for sales executives who signed up for Calendly to meetings with prospects, the First Strike occurs when a prospect schedules a meeting using their Calendly link. For the next onboarding iteration, you could inform them about advanced features that would help them close more deals, such as:
Going Wider
The other option is to go wider by introducing solutions to different problems that your customers face.
For example, there are many reasons why people sign up for Hubspot:
Let’s say you know that a new customer signed up for Hubspot specifically to generate leads with landing pages. After achieving that objective, you can educate them on using social media to drive traffic to that landing page. If they have a sales team, you could also let them know about the benefits of subscribing to Hubspot Sales along with Hubspot Marketing.
What’s Next?
Before we wrap up, some final thoughts. So far, I’ve covered the steps to improve your user onboarding without any assistance from anyone in your team. But sometimes it’s best to have some help. Next, we’ll look at how sales and high-touch support fit into the user onboarding experience.