Why do more pressure, more meetings, and more accountability so often produce the same outcomes? John Dues and Andrew Stotz explore Deming's overlooked insight that results are created by systems — not effort alone. Learn why reacting to variation often makes performance worse, how leaders unintentionally create noise through "tampering," and what it takes to build improvement that actually lasts.

TRANSCRIPT

0:00:02.6 Andrew Stotz: My name is Andrew Stotz, and I'll be your host as we dive deeper into the teachings of Dr. W. Edwards Deming. Today I'm continuing my discussion with John Dues, who is part of the new generation of educators striving to apply Dr. Deming's principles to unleash student joy in learning. And the topic for today is why reacting to results won't improve your system. John, take it away.

0:00:25.6 John Dues: Hey, Andrew. It's good to be back.

0:00:28.6 Andrew Stotz: Yeah, it has been a while.

0:00:30.5 John Dues: It has been a while. We missed a couple months for scheduling stuff, so we're fitting it in on Memorial Day here.

0:00:38.1 Andrew Stotz: Hard working. Even on a holiday.

0:00:41.1 John Dues: Even on a holiday, yep. No doubt. I stumbled across this, I'd seen this a number of times, but I thought I'd start with this quote from Deming. He would often sort of pose this simple question at his seminars. He would, you know, kind of ask the crowd, "what will it take to take an organization to unprecedented levels of quality?" And he was, you know, truth be told, he was kind of setting the crowd up because he knew inevitably someone in the crowd would say, you know, by everyone doing their best. And he would immediately respond then, "they already are, and that's the problem," right? So that's kind of the focus today. And, you know, that sort of exchange to me exposes a belief that still shapes in my world how many schools are led today, and I'm sure many businesses as well. And that is this idea that when results fall short, the instinct is to push harder, you know, respond faster, demand more from people. You know, it feels responsible, it looks decisive, but it rarely, very rarely produces better outcomes, especially on the long term. You know, in many schools, you know, leadership revolves around reviewing outcomes.

0:02:05.8 John Dues: You know, just like probably in your business, you know, we're examining test scores, attendance rates, discipline data, you know, lots of other types of indicators, and we're often comparing those results to what came before. And then we have all these meetings and we have charts and explanations and action steps. And, you know, despite all this attention, all these best efforts, results often remain unchanged.

0:02:30.3 Andrew Stotz: Yeah, it made me think about when, you know, let's just say that a tragedy happens and then everybody wants the government to tighten the laws. And then they're oftentimes responding to a short term, or let's say, normal common cause variation. And next thing you know, you have 10 laws coming down on society that nobody can untangle.

0:02:56.3 John Dues: Yeah.

0:02:56.8 Andrew Stotz: And if you were to actually stand up... And this is, I think, to me, some of the crux of what made Deming different and difficult, was that if you were to actually stand up and say, "my proposal is to do nothing."

0:03:04.0 John Dues: Yeah.

0:03:16.3 Andrew Stotz: Everybody wants action.

0:03:17.3 John Dues: Yep, everybody wants action. It's, you know, the issue is certainly not a lack of effort. You know, I mean, I see it every day, you know, leaders, educators, they work hard. The vast majority, you know, work very, very hard, which is probably the case in most businesses. And you know, in most cases people are already doing their best. And that's kind of the point, right? The issue is that the results are those outputs of those systems. You know, they're produced by the system and they can't be improved directly, the results, that is. You know, but that's what we focus on. As leaders we focus typically on results and, you know, we end up reacting to what the system produces rather than changing, you know, how that system works. And I think that's probably, if not the, one of the key lessons that, you know, Dr. Deming taught in his four-day seminars. And it's just like what you said, you know, that reaction, it feels like action, but it doesn't change, you know, the performance of the system. So, you know, over the past several months, I've argued, you know, as I've been writing about this, that leaders often respond too quickly.

0:04:32.4 John Dues: Just like what you were, you know, talking about in your example there. When the numbers change, it's so often just that common cause, that routine variation, and they don't have any tools to distinguish signal from noise. That's sort of one characterization. So, you know, what happens is these common cause patterns just remain. And when results are not where we want them to be, we just respond to the data itself, right? Instead of actually working towards the system. And we, you know, in my world, it's lots of meetings, you know, we ask... As leaders, we ask for explanations. I definitely did this before I discovered this methodology. We adjust expectations. You know, we in education are sort of notorious for new initiatives piled on old initiatives, but none of these actions, none of these things, it feels productive, but none of them are actually changing that underlying system. And I think that's really where the problem lies in my mind.

0:05:39.5 Andrew Stotz: Yeah, the concept of tampering is such an interesting one, you know, that he talks about, about tampering with a system, you know, just does more damage.

0:05:49.2 John Dues: More damage. Yeah, that's exactly right. That's kind of the irony or the paradox that you often find in the Deming philosophy is that until people actually stop and think about it, you know, if they stop long enough to consider what he was saying, then they start to sort of come around to those ideas. But we often don't slow down enough to actually do that, right? And so, you know, it begs the questions, if reacting to results is ineffective, and we've said that, you know, here before, what is it leaders should do instead? And I, you know, I think a really useful, different starting point is a question to ask yourself. And you know, that first question is, is the process that produced this observation the same as the process that produced the others, right? Is this actually something different being produced in our system or, when we really stop and think about it, is it more of the same? You know, and the answer to that question is going to dictate your next steps. But the key thing is that that question shifts attention away from the most recent data point and toward the system that generated it. You know, it forces you to look back more than just, you know, last month or last year.

0:07:07.7 John Dues: Now you're looking at what's happened the last several months, what's happened in the last several years in this system. You know, I think then if the process has not changed or the system has not changed, I kind of use those interchangeably, then what you often discover is that the results are likely consistent with what the system has been producing all along. And so in that case, which is again so often the case in a common cause system, asking for explanations or making immediate adjustments doesn't address that underlying issue. And it, just like you were saying, it's what Deming called tampering. And it actually makes things worse. All this action, all this activity, it feels good in the moment, but you're actually making things worse in your organization by overreacting or reacting to the wrong things. Now, on the other hand, if the process has changed, then there might be something to investigate, but the goal is not to explain the result, it's to understand what is different in your system. So in either case, whether it's a change or something hasn't changed, I think the key thing is the focus moves from the data to that underlying process or that underlying system.

0:08:29.4 Andrew Stotz: It's... I've been working at my coffee factory with the accounting team using the accounts receivable days and the inventory days as a measure that we can track over time. And then I've, you know, developed a pretty simplified PDSA for the team, considering they've never heard any of this stuff. And so, and then, you know, first thing we saw when we looked at the data was that the inventory days really went down a lot in December. We're like... And that was because we wrote off a lot of inventory at the end of the year that was obsolete or whatever. So there was a... And that's where I could say there is an example of a special cause. There's no sense in changing the system because of that one write-off, although that can give us some indication like, we need to be better in some other areas. But to look at that one special cause as unique wouldn't make sense.

0:09:35.7 John Dues: Yeah. And in that case, is the data point from December being produced by the same system that the other data points came from.

0:09:45.6 Andrew Stotz: It's the same system except there's an extreme adjustment to the system.

0:09:53.1 John Dues: Right.

0:09:53.3 Andrew Stotz: Which is the write off.

0:09:55.5 John Dues: Does that happen every year?

0:09:58.3 Andrew Stotz: Yeah, it's gonna happen in every business that has a warehouse and production because, you know, there's gonna be waste, there's gonna be obsolescence, there's gonna be mistakes, and you... It's just very hard to get it perfect. My first job at Pepsi was counting the inventory, basically, and I ran a team of seven people that counted the inventory every single day. And you know, you just, you know, you can see the whole concept of, you know, that you're never gonna get it perfectly right. But the objective is to minimize and minimize and minimize that, you know, variation. We don't really want a large, the reason why a large hit happened at the end of the year was ultimately because of the management decisions that we made throughout the year to either avoid it, not take, you know, not write it off, or not try to sell it at a discounted price or something like that. So yeah, there's lots of different factors.

0:10:58.3 John Dues: Yeah, sounds like some seasonality in that case is probably the primary driver. Yeah, and I think that's a good segue into this idea that I think when we've talked about systems, you know, they all have sort of a certain level of performance they're capable of producing at any given time. And you know, the key thing is looking at that capability not just at a single point in time, but over, you know, an extended period of time. And then by seeing the patterns that sort of emerge over time, you sort of start to really grasp what the capability of your, you know, your system is. And then, you know, at the same time as an organization we have expectations for those processes or systems, what the results should be. And I think the starting point for improvement is where you start to compare those two things. You know, what is the system currently producing, what do we want it to produce? And then what's the gap between those two levels? And that's where we, you know, often that's where the goal setting and things like that, you know, where our expectation setting falls off track because we've set those goals without studying, without understanding that capability.

0:12:14.0 John Dues: You know, we've talked about this here, it's so often leaders are establishing targets based on aspiration or pressure or, you know, external demands without understanding what the current system can actually deliver. And so then when those results fall short, the response is often to push harder or react more quickly, you know, reactive. We got to do something. Don't just stand there. You know, these are all things that we've talked about multiple times. I think, however, you know, that gap between current and desired performance, it can't be closed by reacting to outcomes. That's the whole point. It can only be closed by changing that underlying system that produces those outcomes. And I think that's what Deming was talking about when he said "substitute leadership." Right. Leadership is about understanding the system, understanding what the system is capable of. And you're a part of that system, you're a part of that understanding. And so you have to sort of lead that understanding, lead that capability understanding, and then start to help lead with the changing of that underlying system. It's not just the frontline workers, you know, in our case, the teachers, they can't be left to their own devices because they don't have control over so much of the system just like, you know, the production workers in your Pepsi example.

0:13:36.0 Andrew Stotz: Yeah, it's interesting because I've been working with this, my accounting and finance and inventory team, and you can definitely see, like, they cannot produce a different result than what they're producing right now. Like, I just can see that. And even thinking about how do we do that is a challenge. It needs some time. But I also don't want to come in and give solutions. I want to teach them how to use PDSA and how to think about, you know, the variation. And so, you know, but I did, like, I told them a story about, I went to visit a credit card collection company and they were very successful in Thailand and they were a Japanese company. And the way that they did it is they set up the collection date was the 15th, starting, or your credit card, basically you gotta pay on the 15th. And their goal was, of course, to be at 100% collection by the end of the month. So they made a whiteboard, and they just marked down each day from 15, 16, 17, all the way to 31. And then they had eight teams, and each team each day would post the percent collection.

0:14:52.8 Andrew Stotz: So one day they had 50%, you know, 50% of people just pay, and then 60 and 70. And so there's a natural, you know, increase as people are paying their bills. But then they start doing calls and other things, and then they can look at other teams and see how's each team doing. And I asked the Japanese manager of it, this was in Thailand many years ago, and I asked him, what do you do if one team's doing really well and the other one's collecting, you know, a lot less? He said, well, we have the better team help the weaker team.

0:15:23.6 John Dues: Hmm. Imagine that.

0:15:25.2 Andrew Stotz: And I was like, that would just never happen in America.

0:15:27.6 John Dues: Yeah.

0:15:31.9 Andrew Stotz: It's like, not my problem, dude. You're not doing it.

0:15:48.6 John Dues: Yeah. You know, when you're talking about, you know, a special cause, you know, a key point is that a special cause could be positive, you know, that positive deviance. And so what do you do? Exactly what you just said. You study it, and then you share it with, with, you know, with everybody else, right? And it's not in a gloating way. It's just like what you described, you know, they were helping another team because, you know, a few months from now, maybe it's this, you know, the team that needed help, they may then be, you know, helping others. And that's such a much better way to sort of operate, you know.

0:16:08.3 Andrew Stotz: Yeah, by giving some ideas like that. And this is why I love reading and I love podcasts, and I love that because, you know, you hear different ideas and you think, hmm, I wonder, you know, why don't we... We could do a PDSA on that and say, what could we track on a daily basis that would keep us all kind of seeing the progression? Would that help? Maybe not.

0:16:30.9 John Dues: Well, at least it puts it in front of you. So everybody sort of, you know, closes that feedback loop, it makes it visual. We're actually doing something like that right now with our student recruitment department, where we have a weekly meeting, we have a board where, you know, all of our sort of leading and lagging indicators are visualized, and we can see right away, like, where are the gaps. And then immediately with that team, we're problem solving. Okay, this campus wasn't able to make, you know, or spend as much time on recruitment, like what's getting in the way, and then they can immediately problem solve with some of their peers from other campuses that are doing the same work. It's been really powerful to sort of operate like that.

0:17:09.2 Andrew Stotz: Yeah. And some people may look at it as just a torture to say, why is this up? Why is this down? You know, as you're talking about reacting. That's not the purpose. The purpose is... And I think for the purposes in my own situation, the purpose is awareness.

0:17:23.1 John Dues: Yeah, yeah. And it's in our case, you know, this is not a gotcha. It's not an accountability measure where, oh, you didn't hit, you know, a certain target. It's not like that. It's more along the lines of what got in your way, what didn't allow you, you know, we, we kind of, especially with the process-oriented targets, we kind of came up with them based on the end goal. So how much work do we need to be doing now in different areas to hit the end goal before the new school year starts? And we know that that requires a certain amount of, you know, hours, a certain number of calls we have to make, a certain number of doors we have to knock on, and those types of things. And so, you know, if we wait weeks before we sort of attend to some of the gaps where we're seeing instead of, you know, doing it immediately, then we're just gonna fall farther and farther behind from our goal. And so that's, you know, again, part of the power of doing it like what you're describing. And I think it really just goes back to this idea that, you know, better results require a better system.

0:18:22.9 John Dues: And then that means that the improvement work has to focus on how the system is operating. And we're talking about ways to visualize what the system is producing and in a very quick way then attend to those areas where things aren't where we want them to be. You know, so there's all kinds of ways to do that. But, you know, the key is the leader is not walking in and saying, why aren't the results where I want them, that's the key here. There's so many other things that we can do, kind of like what you were just talking about. We can, you know, study how the current system functions. We can identify areas in the systems where the performance is being limited. We can test small changes through a PDSA to improve outcomes. You know, we can repeat those cycles of learning to, you know, build knowledge. So there's many other things that we can do besides just applying pressure and say, you know, I don't know how you're gonna do it. I don't care how you do it. Just figure it out and get it done. You know, that's sort of the opposite of what Deming was talking about.

0:19:26.3 Andrew Stotz: Yeah. And when you work with them on it, you create lasting change.

0:19:31.9 John Dues: Well, I think, yeah. And when you're working with people on it, you know, they're much more likely to be bought into it, and the change is much more likely to stick because they were part of producing that solution in the first place, right? We're not trying to force better results, but we're trying to design a better system that regularly is capable of producing those results. That's kind of how I think about it.

0:19:55.7 Andrew Stotz: So what would be your parting words to the audience here?

0:19:59.4 John Dues: Yeah, I mean, I think most of what I've seen is that improvement efforts also often fall short, you know, when we are focused on results instead of that system that produces them. So there's sort of three big ideas for me when I'm thinking about this other way of doing things. So big idea one would be we have to realize that results are produced by systems, not individual effort alone. And so pushing harder on outcomes does not change system performance. I think that'd be big idea number one. Big idea number two would be reacting to results, whether they are surprising or stable, does not improve capability. It often creates more noise without addressing the underlying causes. And then big idea number three is that improvement requires understanding current system performance and redesigning the system through disciplined experimentation. Deming's preferred method, the one we've talked about, is that Plan-Do-Study-Act cycle. And so I think when leaders shift their focus from reacting to results to improving the system, that's really when we move from activity to learning and from effort to effectiveness. And that's just what I've found as I've continued to try to apply the Deming philosophy in my own work here in Columbus.

0:21:21.2 Andrew Stotz: Yeah. And for the listeners out there, imagine if each of the challenges you faced, you created a permanent solution to through the process of, you know, not reacting, understanding variation, right, maybe using PDSA. But what happened was you permanently dealt with that particular issue. Imagine where you would be if you never had to deal with the same problem twice.

0:21:51.9 John Dues: It's learning your way to a better system.

0:21:54.4 Andrew Stotz: Yeah. Just imagine, I mean, you would be at the moon by now. So, but instead, most of the time people are stumbling through. So, well, on behalf of everyone at The Deming Institute, John, I want to thank you again for the discussion. And for listeners, remember to go to deming.org and jump into DemingNEXT to continue your journey. You can find John's book, Win-Win, W. Edwards Deming, the System of Profound Knowledge, and the Science of Improving Schools on amazon.com. This is your host, Andrew Stotz, and I'll leave you with one of my favorite quotes from Dr. Deming, and that is, "people are entitled to joy in work."