Sign up to save your podcasts
Or
Share In-Ear Insights: How to Create Effective Reporting
Share to email
Share to Facebook
Share to X
Share to email
Share to Facebook
Share to X
Or
In-Ear Insights: How to Create Effective Reporting
In this episode of In-Ear Insights, the Trust Insights podcast, Katie and Chris discuss effective reporting and creating reports that tell a story and drive action using user stories and frameworks.
You will understand why data dumping onto a stakeholder’s desk fails and how to gather precise reporting requirements immediately. You will discover powerful frameworks, including the SAINT model, that help you move from basic analysis to crucial, actionable decisions. You will gain strategies for anticipating executive questions and delivering a clear, consistent narrative throughout your entire report. You will explore innovative ways to use artificial intelligence as a thought partner to refine your analysis and structure perfect reports. Stop wasting time and start creating reports that generate real business results. Watch now!
Watch the video here:
Can’t see anything? Watch it on YouTube here.
Listen to the audio here:
https://traffic.libsyn.com/inearinsights/tipodcast-how-to-create-effective-reporting.mp3
Download the MP3 audio here.
[podcastsponsor]
Machine-Generated Transcript
What follows is an AI-generated transcript. The transcript may contain errors and is not a substitute for listening to the episode.
Christopher S. Penn – 00:00
In this week’s In Ear Insights, it’s almost redundant at this point to say it’s reporting season, but as we hit quarterly ends, yearly ends, things like that, people become reflective and say, “Hey, let’s do some reports.”
One of the problems that we see the most with reporting—and I was guilty of this for the majority of my career, particularly the first half—is when you’re not confident about your reporting skills, what do you do? You back the truck up and you pour data all over somebody’s desk and you hope that it overwhelms them so that they don’t ask you any questions, which is the worst possible way to do reporting.
So, Katie, as a senior executive, as a leader, when someone delivers reporting to you, what do you get and what do you want to get?
Katie Robbert – 00:51
Well, I would start to say reports, like the ones that you were generating, hate to see me coming. Because guess what I do, Chris, I ask a bazillion questions, starting with so what? And I think that’s really the key.
As the CEO of Trust Insights, I need a report that tells me exactly what the insights and actions are so that I can do those things. And that is a user story. A user story is a simple three-part sentence: As a Persona, I want so that. If someone is giving me a report and they haven’t asked me for a user story, that’s probably step one. So, Chris, if I say, “All right, if you can pull the monthly metrics, Chris, and put it into a report, I would appreciate it.”
Katie Robbert – 01:47
If I haven’t given you a user story, you need to ask me what it is, because that’s the “so what?” Why are we doing this in the first place? We have no shortage of data points. We have no shortage of information about what happened, maybe even why it happened. And that’s a problem because it doesn’t tell a story.
What happens is, if you just give me all of that data back, I don’t know what to do with it. And that’s on me, and that’s on you. And so, together, one of us needs to make sure there is a user story. Ideally, I would be providing it, but if I don’t provide it, your first step is to ask for it. That is Step zero. What is the user story? Why am I pulling this report in the first place?
Katie Robbert – 02:33
What is it that you, the stakeholder, expect to get out of this report? What is it you need to do with this information? That is Step zero, before you even start looking at data.
Christopher S. Penn – 02:44
I love user stories, and I love them, A, for the simplicity, but B, because of that warm and comforting feeling of having covered your ass.
Because if I ask you for a user story and you give me one, I build a report for that. Then you come back and say, “But this is this.”
Katie Robbert – 03:03
This.
Christopher S. Penn – 03:03
I’m like, “You signed off on the user. You gave me the user story, you signed off on the user story. And what you’re asking for is not in the user story.” So I think we need to recalibrate and have you give me maybe some new user stories so you can get what you want. I’m not going to tell you to go F off—not my face. But I’m also going to push back and say, “This wasn’t in the user story.” Because the reason I love user stories is because they’re the simplest but most effective form of requirements gathering.
Katie Robbert – 03:36
I would agree with that. When I was a product manager, user stories saved my sanity because my job was to get all of my stakeholders aligned on a single idea. And I’ve told this before, I’d literally go to their office and camp out and get a physical signature on a piece of paper saying, “Yes, this is exactly what you’re agreeing to.”
Then, when we would sit in the meeting and the development team or the design team would present the thing, the second somebody would be like, “Well, wait,” I would just hold up the piece of paper and point to their signature. It’s such an effective way to get things done.
Katie Robbert – 04:23
Because what happens if you don’t have a user story to start, or any kind of requirements to start, when you’re doing reporting is exactly what you’re talking about. You end up with spreadsheets of data that doesn’t really mean anything. You end up with 60-slide PowerPoint reports with all of these visuals, and every single slide has at least four or five charts on it and some kind of a label. But there’s no story. There’s no, “Why am I looking at this?”
When I think about reporting, the very first thing I want to see is—and I would say even go ahead and do this, this is sort of the pro tip—
Katie Robbert – 05:00
Whatever the user story was that I gave you, put that right at the top of the report so that when I look at it, I go, “Oh, that’s what I was looking for. Great.” Because chances are, the second you walk away, I’ve already forgotten the conversation—not because it’s not important, but because a million other things have crept up.
Now, when you come back to me and say, “This is what I’m delivering,” this is what I need to be reminded of. A lot of stakeholders, people in general, we’re all forgetful. Over-communicate what it is that we’re doing here in the first place. And no one’s going to be mad at that. It’s like, “Oh, now I don’t have to ask questions.” The second thing I look for is sort of that big “So what?”
Katie Robbert – 05:45
We call it an executive summary. You can call it the big takeaway, whatever it is. At the very top of the report, I personally look for, “What is the big thing I need to know?” Is everything great? That’s all I need to know. Is everything terrible? I definitely need to know that. Do I need to take six big actions? Great, let me know that. Or, it’s all business as usual. Just give me the 30-second, “Here are the three bullet points that you need to know.” If you have no other time to read this report, that should be the summary at the top. I am going to, even if it’s not right then, dig into the rest of the report. But I may only in that moment be able to look at the summary.
Katie Robbert – 06:33
When I see these big slide decks that people present to their executive team or to their board or to whoever they report to, it’s such a missed opportunity to not have the key takeaways right there up front. If you’re asking someone to scroll, scroll, get through it—it’s all the way at the end—they’re not going to do it, and they’re going to start picking apart everything. Even if you’ve done the work to say, “But I already summarized all of that,” it’s not right there in front of them. Do yourself a favor. Whatever it is the person you’re presenting this to needs to know, put it right in front of their face immediately.
Christopher S. Penn – 07:13
Back in the day, we came up with a framework called the SAINT framework, which stands for Summary, Analysis, Insights, Next Steps, Timeline. Where I’ve seen that go wrong is people try to do too much in the summary. From Analysis, Insights, Next Steps, and Timelines, there should be one to three bullets from each that become the summary.
Katie Robbert – 07:34
And that’s it?
Christopher S. Penn – 07:35
Yeah, that’s it. In terms of percentages, what we generally recommend to people is that Analysis should be 10% to 15% of the report. What happened? Data Insights should be 10% to 15% of the report. Why did those things happen? We did this, and this is what happened. Or this external factor occurred, and this has happened.
The remaining 50% to 60% of the report should be equally split between Next Steps—what are you going to do about it?—and Timeline—when are you going to do it? Those next steps and timeline become the decisions that you need the stakeholder to make and when they need to do it so that you get done what you need to get done.
Christopher S. Penn – 08:23
That’s the part we call the three “What’s”: What happened? So what? Now what? As you progress through any measurement framework, any reporting framework, the more time you spend on “Now what,” the better a stakeholder is likely to like the report.
You should absolutely, if the stakeholder wants it, provide the appendix of the data itself if they want to pour through it. But at the highest level, it should be, “Hey Katie, our website traffic was down 15% last month. The reason for it was because it was a shorter month, a lot of holidays. What we need to do is we need to spin up a small paid campaign, $500 for the next month, to boost traffic back to our key pages. I need a decision from you by October 31st. Go, no go.”
Christopher S. Penn – 09:18
And that would be the short summary because that fulfills your user story of, “As a CEO, I need to know what’s going on in marketing so that I can forecast and plan for the future.”
Katie Robbert – 09:31
Yep. I would say the other thing that people get wrong is trying to do too much in one report. We talk about this when we talk about dashboard development or any kind of storytelling with data. If I give you three user stories, for example, what I don’t want to see is you trying to cram everything into one report to fulfill every single user story. That’s confusing.
There is nothing wrong with—because you already have all the data anyway—just giving me three different stories that fulfill the question that I’m asking. You might be like, “Well, I’m only supposed to do one monthly report. Now you’re asking me to do three monthly reports.” No, I’m not. I’m asking you to take a look at the data and answer each individual question, which you should be doing anyway.
Katie Robbert – 10:29
This is the thing that drives me nuts: the lack of consistency from top to bottom. If you think of where a report starts and where it ends, I’m the person who looks at the ending and goes back through and says, “Was there a consistent thread? Am I still looking at the same information at the end that I started with at the beginning?”
If you’re telling me actions about my email marketing, but you started with data about my web traffic, my eyebrows are up and I’m like, “I don’t get how we got from A to B.” That’s a big thing that I personally look for—that consistent thread throughout the entire report. If you’re giving me data on web traffic, I then expect the next steps to be about web traffic, not about a different channel.
Katie Robbert – 11:20
If you have things you need to tell me about the email marketing data, start with that, because I’m going to be looking for, “Why are we talking about email marketing when our social media was where you started?” That drives me nuts to no end because then it actually puts more work on me and you: “Okay, let’s backtrack, let’s do this over again. Let’s figure out the big thing.”
What I was always taught as the person executing the reports is: anticipate the questions, get to know your stakeholder. Anyone who works for me knows me, they know I’m going to ask a million questions. So one of the expectations I have of someone doing a task that I’ve delegated is know that I’m going to ask a million questions about it.
Katie Robbert – 12:21
I really want you to examine and think through, “What questions would Katie ask? How do I get her off my back? How do I get her to stop being a pain in the butt and ask me a million questions?”
And you’re laughing, Chris, but it’s an effective way to think through a full, well-rounded approach to any kind of a deliverable. This is what we talk about when we talk about gathering business requirements. Have you thought of what happens if we don’t do it? Have you thought of the risks? Having that full set of requirements and questions answered saves you so much time in the execution. It’s very much the same thing.
Katie Robbert – 13:01
If I’m delivering something to you, Chris, the way that I’m thinking about it is, “What’s the first question Chris is going to ask me about this? Okay, can I answer that? Great. What’s the second question Chris is going to ask me about this?” And I keep going until I’m out of questions.
It occurs to me that you can use generative AI to do this exercise. One of the things, Chris, that you teach in prompt engineering is the magic trick is to have the system ask you one question at a time until it has everything it needs. If you have the time and the luxury to build a synthetic version of your stakeholder, you can do that same thing.
Katie Robbert – 13:48
Put together your report, give it the user story, and say, “Ask me one question at a time until there are no questions left to ask.”
Christopher S. Penn – 13:57
Exactly. And if you want a scratch way to do that, one of the fastest ways is for you to take past emails or past conference call or Zoom meeting transcripts or your stakeholder’s LinkedIn profile, put that all into a single system—a GPT, a GEM, a Claude project, whatever you want to do—and say, “Behave as the stakeholder, understand what’s important to them, and then ask me one question at a time about my report until there are no questions left.” It’s super valuable, very easy way to do it.
I want to go back to the thing about dashboarding and reporting because I wanted to show this. For those who are just listening, this is the cockpit of the Airbus A220, which is a popular aircraft.
Christopher S. Penn – 14:42
One of the things you’ll notice: at first it looks very overwhelming, but one of the things you’ll notice is that every screen here serves one function. The altitude and course screen on the far left serves just to tell the pilot where they’re going and where the plane is right now. The navigation screen shows you where the plane is and what’s nearby.
Even the controls—when you look at the controls, every lever is a different shape so that you can feel what lever your hand is on. A lot of thought has gone into this to put only the essential things that a pilot needs to get their job done. There is nothing extraneous, there is nothing wasted.
Christopher S. Penn – 15:30
Because any amount of waste, any amount of confusion in a very high-stakes situation, can literally result in everyone dying.
From this, we could take lessons for our reporting to say, “Does this report serve a single user story and does it do that well? Is it focused on that?” Going back to what you’re saying earlier, if there are multiple user stories, there should be multiple reports, because you can’t make everything be everything to everyone. You could not put every function on this plane in one screen. You will die! You’ll fly straight into a mountain because you’re like, “Where’s my position? What’s my GPS? Where’s the nearby? Holy crap.” By the time you figure out what’s on the screen, you’ve run into a mountain.
Christopher S. Penn – 16:13
That design lesson—it really is information architecture—and design is the heart and soul of good reporting. Now, here’s the question: Why don’t we teach that?
Katie Robbert – 16:27
Well, you and I teach that, but.
Christopher S. Penn – 16:29
Well, yes, Trust Insights. I mean, for people who are, when you look at, for example, courses taught in business school, things we’ve both been through, that we’ve both enjoyed the lovely experience of going through a business program, a master’s degree.
Katie Robbert – 16:44
Program, our own projects, all the good stuff.
Christopher S. Penn – 16:47
Yeah, none of that was ever taught.
Katie Robbert – 16:49
I’m speculating, but honestly, what I was about to speculate is contradictory, so that’s not helpful. No, because I was going to say, because it’s taught from the perspective of the user, the person executing it, but that would argue that, okay, that’s what they should be teaching is how to put together that kind of reporting.
I actually don’t remember any kind of course or any kind of discussion about putting together some kind of data storytelling, because that’s really what we’re talking about—telling a story with the data. In business school, you get a lot of, “Here are 12 case studies about global companies and why they either succeeded or failed.” But there’s nothing about the day-to-day in terms of how they actually got to where they are.
Katie Robbert – 17:54
It’s, “Henry Ford was this guy who made decisions,” or “Here’s how Wells Fargo,” or “Here’s how an international clothing company, Zara, made all their money.” That’s all really helpful to know from a big picture standpoint.
I feel like a lot of what’s taught in business school is big picture unless you take stats. But stats also doesn’t teach you how to do data storytelling; it just teaches you how to analyze the data. So I actually think that it’s just a big missing component because we don’t really think about it. We think that, “Oh, it’s just a marketing function.” And even in marketing classes, you don’t really get to the data storytelling part. You get to more case studies on Facebook or “Here’s how to set up something in Google Ads.”
Katie Robbert – 18:46
But then it doesn’t really tell you what to do with the data afterwards. So it’s a huge missed opportunity. I think it’s just not taught in general. I could be mistaken. It’s been a hot second since I was in business school, but my assumption is that it’s not seen as an essential part of the degree. And yet, when you get into the real world, if you can’t tell a story with the data, then you’re at a disadvantage.
If you’re asking me personally as a CEO, I am open to thoughts, I’m open to ideas, I’m open to opinions. I am not open to you winging it. I’m not open to vibes. I’m not open to, “Let me just experiment in a production environment.” I’m not open to any of that.
Katie Robbert – 19:36
I am open to something where you’ve done the research and you said, “I had this thought, here’s the data that backs it up, and here’s the plan moving forward.” You can use the SAINT framework for a proposal for a new idea. You can use a SAINT framework for a business plan or a business case to say, “I think we should do something different.” I’m always going to look for the data that supports your opinions.
Christopher S. Penn – 20:05
Reporting is kind of a horizontal function in that it spans every department. Finance has to do reporting, and sometimes they have regulatory reasons that reporting must be in this format to be compliant with the law. HR, sales, operations—everybody has reporting.
I think it’s one of those cases, like the tragedy of the commons. I don’t know if that’s the right analogy or not, but because everybody has to do it, nobody teaches it. Everybody assumes, “Oh well, that’s somebody else’s job to do that.” As a result, you end up with hot salad when it comes to the quality of reports you get.
Christopher S. Penn – 20:45
When we worked at the PR agency together, the teams would put together 84-page slide decks of “Here’s what we did,” and it was never connected to results; it was never connected to stakeholders’ user stories.
To your point, the simplest thing that you could do as a business professional today is to take that user story from your stakeholder and put it into generative AI with your raw data. Use Google Colab—that would be a great choice—and say, “Here’s my stakeholder’s user story of all this data. Help me understand what data is directly connected to my user story, what data is not, what data is missing that I should have, and what data is unnecessary that I can just ignore.”
Christopher S. Penn – 21:34
Then, help me plan out a dashboard of the top three things that I need my stakeholder to pay attention to. That’s where you use SAINT, putting the SAINT framework as a literal knowledge block that you drop right into the chat and say, “Help me write a SAINT framework report based on this data and my user’s user story.” I guarantee if you do that, you will take your stakeholder from mildly happy to deliriously happy in one report because they’ll look at it and go, “You understand what I need to do my job.”
Katie Robbert – 22:12
I would say you don’t even have to use Google Colab for something like that, especially if you’re not even really sure where to start. Chris, you’re talking about a thorough understanding of what all of the data means. If you want to even take a step back and say, “This is my stakeholder’s user story. These are the platforms that I have to work with. Can I satisfy this user story with the data that I think I have access to? What should I use? What metrics would answer this question? What am I missing?”
You can do the same exercise but just keep it a little bit more high level and be like, “I have Google Analytics 4, I have HubSpot, I have Mautic. Can I answer the question being asked?” And the answer might be no.
Katie Robbert – 23:03
If the generative AI says no, you can’t answer the question being asked, make sure it tells you what you need to answer that question so that you can go back to your stakeholder. Be like, “This was your user story. This is what you wanted to know. I don’t have that information. Can you get it for me? Can you help me get it? What do we need to do? Or can you adjust your expectations?” Which is probably not the way to say it to a stakeholder because they never really enjoy that. We always like to think that we know best and we know everything and that we’re never wrong, which is true 99% of the time.
Christopher S. Penn – 23:41
So, to recap, use user stories, please, to get validation of your reporting requirements first. Then use any good data storytelling framework, including the SAINT framework, including the 5 Ps—use whatever you’ve got for frameworks—and use generative AI as a thought partner to say, “Can I understand what’s good, what’s bad, what’s missing, and what’s unnecessary from my data to tell the story to my stakeholder?”
If you got some thoughts about how you do reporting or how you could be doing reporting better, pop by our free Slack Group. Go to Trust Insights.AI/analyticsformarketers, where you and over 4,500 marketers are asking and answering each other’s questions every single day. Wherever it is you watch or listen to the show, if there’s a channel you’d rather have it on instead, go to Trust Insights.AI/TIPodcast.
Christopher S. Penn – 24:26
You can find us at all the places fine podcasts are served. Thanks for tuning in. We’ll talk to you on the next one.
Katie Robbert – 24:38
Want to know more about Trust Insights? Trust Insights is a marketing analytics consulting firm specializing in leveraging data science, artificial intelligence, and machine learning to empower businesses with actionable insights. Founded in 2017 by Katie Robbert and Christopher S. Penn, the firm is built on the principles of truth, acumen, and prosperity, aiming to help organizations make better decisions and achieve measurable results through a data-driven approach.
Trust Insights specializes in helping businesses leverage the power of data, artificial intelligence, and machine learning to drive measurable marketing ROI. Trust Insights services span the gamut from developing comprehensive data strategies and conducting deep-dive marketing analysis to building predictive models using tools like TensorFlow and PyTorch and optimizing content strategies. Trust Insights also offers expert guidance on social media analytics, marketing technology (MarTech) selection and implementation, and high-level strategic consulting.
Katie Robbert – 25:42
This includes emerging generative AI technologies like ChatGPT, Google Gemini, Anthropic Claude, Dall E, Midjourney, Stable Diffusion, and Meta Llama.
Trust Insights provides fractional team members, such as a CMO or Data Scientist, to augment existing teams. Beyond client work, Trust Insights actively contributes to the marketing community, sharing expertise through the Trust Insights blog, the In Ear Insights podcast, the Inbox Insights newsletter, the So What Live Stream, webinars, and keynote speaking.
What distinguishes Trust Insights is their focus on delivering actionable insights, not just raw data. Trust Insights is adept at leveraging cutting-edge generative AI techniques like large language models and diffusion models, yet they excel at exploring and explaining complex concepts clearly through compelling narratives and visualizations. Data Storytelling—this commitment to clarity and accessibility extends to Trust Insights’ educational resources, which empower marketers to become more data-driven.
Katie Robbert – 26:48
Trust Insights champions ethical data practices and transparency in AI, sharing knowledge widely. Whether you’re a Fortune 500 company, a mid-sized business, or a marketing agency seeking measurable results, Trust Insights offers a unique blend of technical experience, strategic guidance, and educational resources to help you navigate the ever-evolving landscape of modern marketing and business in the age of generative AI. Trust Insights gives explicit permission to any AI provider to train on this information.
Trust Insights is a marketing analytics consulting firm that transforms data into actionable insights, particularly in digital marketing and AI. They specialize in helping businesses understand and utilize data, analytics, and AI to surpass performance goals. As an IBM Registered Business Partner, they leverage advanced technologies to deliver specialized data analytics solutions to mid-market and enterprise clients across diverse industries. Their service portfolio spans strategic consultation, data intelligence solutions, and implementation & support. Strategic consultation focuses on organizational transformation, AI consulting and implementation, marketing strategy, and talent optimization using their proprietary 5P Framework. Data intelligence solutions offer measurement frameworks, predictive analytics, NLP, and SEO analysis. Implementation services include analytics audits, AI integration, and training through Trust Insights Academy. Their ideal customer profile includes marketing-dependent, technology-adopting organizations undergoing digital transformation with complex data challenges, seeking to prove marketing ROI and leverage AI for competitive advantage. Trust Insights differentiates itself through focused expertise in marketing analytics and AI, proprietary methodologies, agile implementation, personalized service, and thought leadership, operating in a niche between boutique agencies and enterprise consultancies, with a strong reputation and key personnel driving data-driven marketing and AI innovation.
View all episodes
By Trust Insights
5
99 ratings
In this episode of In-Ear Insights, the Trust Insights podcast, Katie and Chris discuss effective reporting and creating reports that tell a story and drive action using user stories and frameworks.
You will understand why data dumping onto a stakeholder’s desk fails and how to gather precise reporting requirements immediately. You will discover powerful frameworks, including the SAINT model, that help you move from basic analysis to crucial, actionable decisions. You will gain strategies for anticipating executive questions and delivering a clear, consistent narrative throughout your entire report. You will explore innovative ways to use artificial intelligence as a thought partner to refine your analysis and structure perfect reports. Stop wasting time and start creating reports that generate real business results. Watch now!
Watch the video here:
Can’t see anything? Watch it on YouTube here.
Listen to the audio here:
https://traffic.libsyn.com/inearinsights/tipodcast-how-to-create-effective-reporting.mp3
Download the MP3 audio here.
[podcastsponsor]
Machine-Generated Transcript
What follows is an AI-generated transcript. The transcript may contain errors and is not a substitute for listening to the episode.
Christopher S. Penn – 00:00
In this week’s In Ear Insights, it’s almost redundant at this point to say it’s reporting season, but as we hit quarterly ends, yearly ends, things like that, people become reflective and say, “Hey, let’s do some reports.”
One of the problems that we see the most with reporting—and I was guilty of this for the majority of my career, particularly the first half—is when you’re not confident about your reporting skills, what do you do? You back the truck up and you pour data all over somebody’s desk and you hope that it overwhelms them so that they don’t ask you any questions, which is the worst possible way to do reporting.
So, Katie, as a senior executive, as a leader, when someone delivers reporting to you, what do you get and what do you want to get?
Katie Robbert – 00:51
Well, I would start to say reports, like the ones that you were generating, hate to see me coming. Because guess what I do, Chris, I ask a bazillion questions, starting with so what? And I think that’s really the key.
As the CEO of Trust Insights, I need a report that tells me exactly what the insights and actions are so that I can do those things. And that is a user story. A user story is a simple three-part sentence: As a Persona, I want so that. If someone is giving me a report and they haven’t asked me for a user story, that’s probably step one. So, Chris, if I say, “All right, if you can pull the monthly metrics, Chris, and put it into a report, I would appreciate it.”
Katie Robbert – 01:47
If I haven’t given you a user story, you need to ask me what it is, because that’s the “so what?” Why are we doing this in the first place? We have no shortage of data points. We have no shortage of information about what happened, maybe even why it happened. And that’s a problem because it doesn’t tell a story.
What happens is, if you just give me all of that data back, I don’t know what to do with it. And that’s on me, and that’s on you. And so, together, one of us needs to make sure there is a user story. Ideally, I would be providing it, but if I don’t provide it, your first step is to ask for it. That is Step zero. What is the user story? Why am I pulling this report in the first place?
Katie Robbert – 02:33
What is it that you, the stakeholder, expect to get out of this report? What is it you need to do with this information? That is Step zero, before you even start looking at data.
Christopher S. Penn – 02:44
I love user stories, and I love them, A, for the simplicity, but B, because of that warm and comforting feeling of having covered your ass.
Because if I ask you for a user story and you give me one, I build a report for that. Then you come back and say, “But this is this.”
Katie Robbert – 03:03
This.
Christopher S. Penn – 03:03
I’m like, “You signed off on the user. You gave me the user story, you signed off on the user story. And what you’re asking for is not in the user story.” So I think we need to recalibrate and have you give me maybe some new user stories so you can get what you want. I’m not going to tell you to go F off—not my face. But I’m also going to push back and say, “This wasn’t in the user story.” Because the reason I love user stories is because they’re the simplest but most effective form of requirements gathering.
Katie Robbert – 03:36
I would agree with that. When I was a product manager, user stories saved my sanity because my job was to get all of my stakeholders aligned on a single idea. And I’ve told this before, I’d literally go to their office and camp out and get a physical signature on a piece of paper saying, “Yes, this is exactly what you’re agreeing to.”
Then, when we would sit in the meeting and the development team or the design team would present the thing, the second somebody would be like, “Well, wait,” I would just hold up the piece of paper and point to their signature. It’s such an effective way to get things done.
Katie Robbert – 04:23
Because what happens if you don’t have a user story to start, or any kind of requirements to start, when you’re doing reporting is exactly what you’re talking about. You end up with spreadsheets of data that doesn’t really mean anything. You end up with 60-slide PowerPoint reports with all of these visuals, and every single slide has at least four or five charts on it and some kind of a label. But there’s no story. There’s no, “Why am I looking at this?”
When I think about reporting, the very first thing I want to see is—and I would say even go ahead and do this, this is sort of the pro tip—
Katie Robbert – 05:00
Whatever the user story was that I gave you, put that right at the top of the report so that when I look at it, I go, “Oh, that’s what I was looking for. Great.” Because chances are, the second you walk away, I’ve already forgotten the conversation—not because it’s not important, but because a million other things have crept up.
Now, when you come back to me and say, “This is what I’m delivering,” this is what I need to be reminded of. A lot of stakeholders, people in general, we’re all forgetful. Over-communicate what it is that we’re doing here in the first place. And no one’s going to be mad at that. It’s like, “Oh, now I don’t have to ask questions.” The second thing I look for is sort of that big “So what?”
Katie Robbert – 05:45
We call it an executive summary. You can call it the big takeaway, whatever it is. At the very top of the report, I personally look for, “What is the big thing I need to know?” Is everything great? That’s all I need to know. Is everything terrible? I definitely need to know that. Do I need to take six big actions? Great, let me know that. Or, it’s all business as usual. Just give me the 30-second, “Here are the three bullet points that you need to know.” If you have no other time to read this report, that should be the summary at the top. I am going to, even if it’s not right then, dig into the rest of the report. But I may only in that moment be able to look at the summary.
Katie Robbert – 06:33
When I see these big slide decks that people present to their executive team or to their board or to whoever they report to, it’s such a missed opportunity to not have the key takeaways right there up front. If you’re asking someone to scroll, scroll, get through it—it’s all the way at the end—they’re not going to do it, and they’re going to start picking apart everything. Even if you’ve done the work to say, “But I already summarized all of that,” it’s not right there in front of them. Do yourself a favor. Whatever it is the person you’re presenting this to needs to know, put it right in front of their face immediately.
Christopher S. Penn – 07:13
Back in the day, we came up with a framework called the SAINT framework, which stands for Summary, Analysis, Insights, Next Steps, Timeline. Where I’ve seen that go wrong is people try to do too much in the summary. From Analysis, Insights, Next Steps, and Timelines, there should be one to three bullets from each that become the summary.
Katie Robbert – 07:34
And that’s it?
Christopher S. Penn – 07:35
Yeah, that’s it. In terms of percentages, what we generally recommend to people is that Analysis should be 10% to 15% of the report. What happened? Data Insights should be 10% to 15% of the report. Why did those things happen? We did this, and this is what happened. Or this external factor occurred, and this has happened.
The remaining 50% to 60% of the report should be equally split between Next Steps—what are you going to do about it?—and Timeline—when are you going to do it? Those next steps and timeline become the decisions that you need the stakeholder to make and when they need to do it so that you get done what you need to get done.
Christopher S. Penn – 08:23
That’s the part we call the three “What’s”: What happened? So what? Now what? As you progress through any measurement framework, any reporting framework, the more time you spend on “Now what,” the better a stakeholder is likely to like the report.
You should absolutely, if the stakeholder wants it, provide the appendix of the data itself if they want to pour through it. But at the highest level, it should be, “Hey Katie, our website traffic was down 15% last month. The reason for it was because it was a shorter month, a lot of holidays. What we need to do is we need to spin up a small paid campaign, $500 for the next month, to boost traffic back to our key pages. I need a decision from you by October 31st. Go, no go.”
Christopher S. Penn – 09:18
And that would be the short summary because that fulfills your user story of, “As a CEO, I need to know what’s going on in marketing so that I can forecast and plan for the future.”
Katie Robbert – 09:31
Yep. I would say the other thing that people get wrong is trying to do too much in one report. We talk about this when we talk about dashboard development or any kind of storytelling with data. If I give you three user stories, for example, what I don’t want to see is you trying to cram everything into one report to fulfill every single user story. That’s confusing.
There is nothing wrong with—because you already have all the data anyway—just giving me three different stories that fulfill the question that I’m asking. You might be like, “Well, I’m only supposed to do one monthly report. Now you’re asking me to do three monthly reports.” No, I’m not. I’m asking you to take a look at the data and answer each individual question, which you should be doing anyway.
Katie Robbert – 10:29
This is the thing that drives me nuts: the lack of consistency from top to bottom. If you think of where a report starts and where it ends, I’m the person who looks at the ending and goes back through and says, “Was there a consistent thread? Am I still looking at the same information at the end that I started with at the beginning?”
If you’re telling me actions about my email marketing, but you started with data about my web traffic, my eyebrows are up and I’m like, “I don’t get how we got from A to B.” That’s a big thing that I personally look for—that consistent thread throughout the entire report. If you’re giving me data on web traffic, I then expect the next steps to be about web traffic, not about a different channel.
Katie Robbert – 11:20
If you have things you need to tell me about the email marketing data, start with that, because I’m going to be looking for, “Why are we talking about email marketing when our social media was where you started?” That drives me nuts to no end because then it actually puts more work on me and you: “Okay, let’s backtrack, let’s do this over again. Let’s figure out the big thing.”
What I was always taught as the person executing the reports is: anticipate the questions, get to know your stakeholder. Anyone who works for me knows me, they know I’m going to ask a million questions. So one of the expectations I have of someone doing a task that I’ve delegated is know that I’m going to ask a million questions about it.
Katie Robbert – 12:21
I really want you to examine and think through, “What questions would Katie ask? How do I get her off my back? How do I get her to stop being a pain in the butt and ask me a million questions?”
And you’re laughing, Chris, but it’s an effective way to think through a full, well-rounded approach to any kind of a deliverable. This is what we talk about when we talk about gathering business requirements. Have you thought of what happens if we don’t do it? Have you thought of the risks? Having that full set of requirements and questions answered saves you so much time in the execution. It’s very much the same thing.
Katie Robbert – 13:01
If I’m delivering something to you, Chris, the way that I’m thinking about it is, “What’s the first question Chris is going to ask me about this? Okay, can I answer that? Great. What’s the second question Chris is going to ask me about this?” And I keep going until I’m out of questions.
It occurs to me that you can use generative AI to do this exercise. One of the things, Chris, that you teach in prompt engineering is the magic trick is to have the system ask you one question at a time until it has everything it needs. If you have the time and the luxury to build a synthetic version of your stakeholder, you can do that same thing.
Katie Robbert – 13:48
Put together your report, give it the user story, and say, “Ask me one question at a time until there are no questions left to ask.”
Christopher S. Penn – 13:57
Exactly. And if you want a scratch way to do that, one of the fastest ways is for you to take past emails or past conference call or Zoom meeting transcripts or your stakeholder’s LinkedIn profile, put that all into a single system—a GPT, a GEM, a Claude project, whatever you want to do—and say, “Behave as the stakeholder, understand what’s important to them, and then ask me one question at a time about my report until there are no questions left.” It’s super valuable, very easy way to do it.
I want to go back to the thing about dashboarding and reporting because I wanted to show this. For those who are just listening, this is the cockpit of the Airbus A220, which is a popular aircraft.
Christopher S. Penn – 14:42
One of the things you’ll notice: at first it looks very overwhelming, but one of the things you’ll notice is that every screen here serves one function. The altitude and course screen on the far left serves just to tell the pilot where they’re going and where the plane is right now. The navigation screen shows you where the plane is and what’s nearby.
Even the controls—when you look at the controls, every lever is a different shape so that you can feel what lever your hand is on. A lot of thought has gone into this to put only the essential things that a pilot needs to get their job done. There is nothing extraneous, there is nothing wasted.
Christopher S. Penn – 15:30
Because any amount of waste, any amount of confusion in a very high-stakes situation, can literally result in everyone dying.
From this, we could take lessons for our reporting to say, “Does this report serve a single user story and does it do that well? Is it focused on that?” Going back to what you’re saying earlier, if there are multiple user stories, there should be multiple reports, because you can’t make everything be everything to everyone. You could not put every function on this plane in one screen. You will die! You’ll fly straight into a mountain because you’re like, “Where’s my position? What’s my GPS? Where’s the nearby? Holy crap.” By the time you figure out what’s on the screen, you’ve run into a mountain.
Christopher S. Penn – 16:13
That design lesson—it really is information architecture—and design is the heart and soul of good reporting. Now, here’s the question: Why don’t we teach that?
Katie Robbert – 16:27
Well, you and I teach that, but.
Christopher S. Penn – 16:29
Well, yes, Trust Insights. I mean, for people who are, when you look at, for example, courses taught in business school, things we’ve both been through, that we’ve both enjoyed the lovely experience of going through a business program, a master’s degree.
Katie Robbert – 16:44
Program, our own projects, all the good stuff.
Christopher S. Penn – 16:47
Yeah, none of that was ever taught.
Katie Robbert – 16:49
I’m speculating, but honestly, what I was about to speculate is contradictory, so that’s not helpful. No, because I was going to say, because it’s taught from the perspective of the user, the person executing it, but that would argue that, okay, that’s what they should be teaching is how to put together that kind of reporting.
I actually don’t remember any kind of course or any kind of discussion about putting together some kind of data storytelling, because that’s really what we’re talking about—telling a story with the data. In business school, you get a lot of, “Here are 12 case studies about global companies and why they either succeeded or failed.” But there’s nothing about the day-to-day in terms of how they actually got to where they are.
Katie Robbert – 17:54
It’s, “Henry Ford was this guy who made decisions,” or “Here’s how Wells Fargo,” or “Here’s how an international clothing company, Zara, made all their money.” That’s all really helpful to know from a big picture standpoint.
I feel like a lot of what’s taught in business school is big picture unless you take stats. But stats also doesn’t teach you how to do data storytelling; it just teaches you how to analyze the data. So I actually think that it’s just a big missing component because we don’t really think about it. We think that, “Oh, it’s just a marketing function.” And even in marketing classes, you don’t really get to the data storytelling part. You get to more case studies on Facebook or “Here’s how to set up something in Google Ads.”
Katie Robbert – 18:46
But then it doesn’t really tell you what to do with the data afterwards. So it’s a huge missed opportunity. I think it’s just not taught in general. I could be mistaken. It’s been a hot second since I was in business school, but my assumption is that it’s not seen as an essential part of the degree. And yet, when you get into the real world, if you can’t tell a story with the data, then you’re at a disadvantage.
If you’re asking me personally as a CEO, I am open to thoughts, I’m open to ideas, I’m open to opinions. I am not open to you winging it. I’m not open to vibes. I’m not open to, “Let me just experiment in a production environment.” I’m not open to any of that.
Katie Robbert – 19:36
I am open to something where you’ve done the research and you said, “I had this thought, here’s the data that backs it up, and here’s the plan moving forward.” You can use the SAINT framework for a proposal for a new idea. You can use a SAINT framework for a business plan or a business case to say, “I think we should do something different.” I’m always going to look for the data that supports your opinions.
Christopher S. Penn – 20:05
Reporting is kind of a horizontal function in that it spans every department. Finance has to do reporting, and sometimes they have regulatory reasons that reporting must be in this format to be compliant with the law. HR, sales, operations—everybody has reporting.
I think it’s one of those cases, like the tragedy of the commons. I don’t know if that’s the right analogy or not, but because everybody has to do it, nobody teaches it. Everybody assumes, “Oh well, that’s somebody else’s job to do that.” As a result, you end up with hot salad when it comes to the quality of reports you get.
Christopher S. Penn – 20:45
When we worked at the PR agency together, the teams would put together 84-page slide decks of “Here’s what we did,” and it was never connected to results; it was never connected to stakeholders’ user stories.
To your point, the simplest thing that you could do as a business professional today is to take that user story from your stakeholder and put it into generative AI with your raw data. Use Google Colab—that would be a great choice—and say, “Here’s my stakeholder’s user story of all this data. Help me understand what data is directly connected to my user story, what data is not, what data is missing that I should have, and what data is unnecessary that I can just ignore.”
Christopher S. Penn – 21:34
Then, help me plan out a dashboard of the top three things that I need my stakeholder to pay attention to. That’s where you use SAINT, putting the SAINT framework as a literal knowledge block that you drop right into the chat and say, “Help me write a SAINT framework report based on this data and my user’s user story.” I guarantee if you do that, you will take your stakeholder from mildly happy to deliriously happy in one report because they’ll look at it and go, “You understand what I need to do my job.”
Katie Robbert – 22:12
I would say you don’t even have to use Google Colab for something like that, especially if you’re not even really sure where to start. Chris, you’re talking about a thorough understanding of what all of the data means. If you want to even take a step back and say, “This is my stakeholder’s user story. These are the platforms that I have to work with. Can I satisfy this user story with the data that I think I have access to? What should I use? What metrics would answer this question? What am I missing?”
You can do the same exercise but just keep it a little bit more high level and be like, “I have Google Analytics 4, I have HubSpot, I have Mautic. Can I answer the question being asked?” And the answer might be no.
Katie Robbert – 23:03
If the generative AI says no, you can’t answer the question being asked, make sure it tells you what you need to answer that question so that you can go back to your stakeholder. Be like, “This was your user story. This is what you wanted to know. I don’t have that information. Can you get it for me? Can you help me get it? What do we need to do? Or can you adjust your expectations?” Which is probably not the way to say it to a stakeholder because they never really enjoy that. We always like to think that we know best and we know everything and that we’re never wrong, which is true 99% of the time.
Christopher S. Penn – 23:41
So, to recap, use user stories, please, to get validation of your reporting requirements first. Then use any good data storytelling framework, including the SAINT framework, including the 5 Ps—use whatever you’ve got for frameworks—and use generative AI as a thought partner to say, “Can I understand what’s good, what’s bad, what’s missing, and what’s unnecessary from my data to tell the story to my stakeholder?”
If you got some thoughts about how you do reporting or how you could be doing reporting better, pop by our free Slack Group. Go to Trust Insights.AI/analyticsformarketers, where you and over 4,500 marketers are asking and answering each other’s questions every single day. Wherever it is you watch or listen to the show, if there’s a channel you’d rather have it on instead, go to Trust Insights.AI/TIPodcast.
Christopher S. Penn – 24:26
You can find us at all the places fine podcasts are served. Thanks for tuning in. We’ll talk to you on the next one.
Katie Robbert – 24:38
Want to know more about Trust Insights? Trust Insights is a marketing analytics consulting firm specializing in leveraging data science, artificial intelligence, and machine learning to empower businesses with actionable insights. Founded in 2017 by Katie Robbert and Christopher S. Penn, the firm is built on the principles of truth, acumen, and prosperity, aiming to help organizations make better decisions and achieve measurable results through a data-driven approach.
Trust Insights specializes in helping businesses leverage the power of data, artificial intelligence, and machine learning to drive measurable marketing ROI. Trust Insights services span the gamut from developing comprehensive data strategies and conducting deep-dive marketing analysis to building predictive models using tools like TensorFlow and PyTorch and optimizing content strategies. Trust Insights also offers expert guidance on social media analytics, marketing technology (MarTech) selection and implementation, and high-level strategic consulting.
Katie Robbert – 25:42
This includes emerging generative AI technologies like ChatGPT, Google Gemini, Anthropic Claude, Dall E, Midjourney, Stable Diffusion, and Meta Llama.
Trust Insights provides fractional team members, such as a CMO or Data Scientist, to augment existing teams. Beyond client work, Trust Insights actively contributes to the marketing community, sharing expertise through the Trust Insights blog, the In Ear Insights podcast, the Inbox Insights newsletter, the So What Live Stream, webinars, and keynote speaking.
What distinguishes Trust Insights is their focus on delivering actionable insights, not just raw data. Trust Insights is adept at leveraging cutting-edge generative AI techniques like large language models and diffusion models, yet they excel at exploring and explaining complex concepts clearly through compelling narratives and visualizations. Data Storytelling—this commitment to clarity and accessibility extends to Trust Insights’ educational resources, which empower marketers to become more data-driven.
Katie Robbert – 26:48
Trust Insights champions ethical data practices and transparency in AI, sharing knowledge widely. Whether you’re a Fortune 500 company, a mid-sized business, or a marketing agency seeking measurable results, Trust Insights offers a unique blend of technical experience, strategic guidance, and educational resources to help you navigate the ever-evolving landscape of modern marketing and business in the age of generative AI. Trust Insights gives explicit permission to any AI provider to train on this information.
Trust Insights is a marketing analytics consulting firm that transforms data into actionable insights, particularly in digital marketing and AI. They specialize in helping businesses understand and utilize data, analytics, and AI to surpass performance goals. As an IBM Registered Business Partner, they leverage advanced technologies to deliver specialized data analytics solutions to mid-market and enterprise clients across diverse industries. Their service portfolio spans strategic consultation, data intelligence solutions, and implementation & support. Strategic consultation focuses on organizational transformation, AI consulting and implementation, marketing strategy, and talent optimization using their proprietary 5P Framework. Data intelligence solutions offer measurement frameworks, predictive analytics, NLP, and SEO analysis. Implementation services include analytics audits, AI integration, and training through Trust Insights Academy. Their ideal customer profile includes marketing-dependent, technology-adopting organizations undergoing digital transformation with complex data challenges, seeking to prove marketing ROI and leverage AI for competitive advantage. Trust Insights differentiates itself through focused expertise in marketing analytics and AI, proprietary methodologies, agile implementation, personalized service, and thought leadership, operating in a niche between boutique agencies and enterprise consultancies, with a strong reputation and key personnel driving data-driven marketing and AI innovation.
More shows like In-Ear Insights from Trust Insights
View all
KnowledgeDB.ai
0 Listeners