Tech Transforms

Tracy Bannon Senior Principal / Software Architect & DevOps Strategic Advisor at MITRE and ambassador for the DevOps Institute talks through the original DevOps timeline. Join as Carolyn and guest host Steve Mazzuca find out what happens when Dev fraternizes with Ops.

Episode Table of Contents

• [00:48] DevOps Strategic Advisor and Ambassador
• [10:34] Respected DevOps
• [18:35] The DevOps Pipeline
• [24:05] DevOps Institute

Episode Links and Resources

• DevOps Institute
• Project To Product

DevOps Strategic Advisor and Ambassador

Carolyn: Today, I have Steve Mazucca or The Mas as I like to call him, co-hosting with me. It's always fun to have a conversation with you Steve. The hard part is going to be getting you to be quiet, so we can get our guest Tracy Bannon. He is Senior Principal, Software Architect and DevOps Strategic Advisor at MITRE, as well as an ambassador for the DevOps Institute. So welcome Tracy.

Tracy: I'm thrilled to be here today. It's always fun to have these conversations.

Carolyn: You are a striking woman with pink hair and you were in development, which makes you in my mind, kind of a unicorn. I would really love to hear your story.

Tracy: I'll start with the pink hair and go backwards from there. I've had little bits of color in my hair for years. My mom was an art teacher. My dad's more on the math and the sciences side of it. I kind of have that left brain, right brain, need to express myself. Over probably the last two or three years, as I've been doing more remote work, I was having more fun with the pink and decided that it's the pandemic. Let's stretch things a little bit more. I'm just loving it. So that's a little bit about that piece of it.

But as for me being a woman in technology, I actually like to come at it in reverse. To say that I'm a real technologist and not say I'm a woman technologist. It matters, but it doesn't matter. What's important to me, is I've always been so interested in tech.A Woman Developer

Tracy: Someone asked me, "When was the first time you realized that you liked computers and that you were into computers?" It's a long story, but I'll make it very short. I can remember building a computer out of a box and cutting and putting mag tapes on the outside. Yes, I just told you how old I was. And arguing with my brother on who got to sit inside it and be the brains. So I remember being real and I couldn't read yet. I remember that very vividly. It goes a long way back.

Carolyn: Did you end up being the brains?

Tracy: Yes I did. I happened to be a little bit bigger than him. Even though he's two years older, I happened to have the weight advantage. As for being a woman developer, I've always been in tech. I never thought anything about the makeup of the team. That’s because I always tagged around with my older brother and his buddies. I considered myself one of the guys. One of the gang would be a better way to put it. I realized about midway through my career that there was a little bit of uniqueness to it. As I would look around the room, I would be the only woman on the team.

Now, occasionally there would be fantastic women involved, more on the database side of things, who had grown into that. Very few from a development perspective. We did see some spikes in industry, we saw that. But we're seeing that decline recently. But across my career, I tend to come at it that I'm a technologist. If you need to give me an adjective, make it real, instead of woman. But that's a little bit about me.

A Technologist at DevOps Who Happens to Be a Woman

Carolyn: I love that you want to take the emphasis off women. That you're a technologist, you're a developer and you happen to be a woman. You're often the only woman in the room. I'm often the only woman in the room and it will be a room of many people. But I do love that you've always just thought of yourself this way. What was your first development? Well, what was your first job actually?

Tracy: First paying job, was actually a lifeguard, but we won't go that far back. If we go forward, I worked throughout college in different corporate settings, always related to technology. When I graduated, I actually was independent before anybody was doing anything independent. And then it happened on my way into the engineering department at AccuWeather. My husband and I were already married and he was in operation. You can tell there was a little bit of, hey, take a look at this resume. That helped get me in the door in a very heavily male-populated tech group.

Yes, we were both within AccuWeather. That's actually a segue to a fun story. He's in operations, I'm in engineering. You could actually say that we're the original DevOps because we've been married for a couple of decades now before we said DevOps. So something would happen with engineering and they would call because there was a production problem. They would call him and he would realize that we had talked about something that week. Dinner time chatter.

Debugging Things Together at DevOps

Tracy: He'd say, "Didn't you guys roll out a change to XYZ." I say, "Oh yes." "Can you open up a window?" It wasn't a browser at that point. "Can you open up a window, let's take a look. We need to look at this queue or this record or this log." And we would end up debugging things together. They started to no longer just call and ask for him. They’d call the Bannon house because they would get both of us to solve a problem, it was really cool.

That showed me how important it was that I was writing software that could be operated, could be managed, could be maintained, could be debugged. It also taught him how important it was to give me access. I didn't have to have right access. But I needed to be able to check the cues. I needed to be able to look at these different things. That started me having that fraternizing with Ops. I've always been an advocate of having Ops at the table, even before. Well, before we coined that wonky phrase DevOps.

Steve: When did we start coming out with that? Is that Gene Kim, is he famous for that? Or is that even before Gene Kim, as far as when he started coming out with that?

Tracy: It was before Gene, but I cannot remember the fellow's name off-hand. I'd have to Google it. It strikes me that it's Patrick something. It wasn't that long ago. Maybe in 2011, 2012, somewhere in that, he made a comment about it. It really resonated with folks.

An Evolution

Tracy: Gene was brilliant to realize that it's a true and valuable story. He started really to table pound to get the message out and started this. I'll call it an evolution. I hate the revolution thing. But he started us evolving and thinking about it.

Steve: It's certainly come a long way. You and I obviously have done a lot of work together over the years and a couple of different iterations across government. I've only known you supporting the government, but I know you did some other things before. How long have you supported the federal government? I don't actually know that story.

Tracy: Federal 2015. Not that long, not an entire career focused on the federal government. But I was working across state governments starting in 1999.

Carolyn: Is it really, really different to be a state versus federal?

Tracy: There are parallels and there are differences. In the federal government, we have a mandate on citizenship. You don't have the same mandates for a state-level data center. I can have a foreign nationals, I can have different types of folks coming in with different types of visas, to be able to support that. That's starting to grow and evolve and change. The policies are much stricter at the federal level and the size.

Some of the biggest states are similar to some of the smallest federal agencies. But think about the economies of scale. It's just that much bigger at the federal level. In defense, not a whole lot of defense at the state level. That's probably the most interesting, different mission that I've been involved with.

Respected DevOps

Tracy: Being involved on the public sector side of the government, the civilian side of the federal government, that's where Steve and I got to know each other. It was with the IRS and treasury and around that side of it. Changing the focus more into defense has been very humbling just because of the sheer complexity. Think about NATO with 30 different nations involved in technical decisions and discussions and the complexities that come with them. It's just mind-boggling.

Carolyn: Just think of communication alone. Being able to communicate across all those different groups, it breaks my head. Can you think of any use cases between state government, federal government vice versa? A best practice that one should pick up from the other.

Tracy: Leading practices abound across both of them. Whether it is with respected DevOps, whether it's with respect to leveraging AI and ML, to improve. One of the things that I'm seeing in both places. I experienced this first with the state of Colorado and that was embracing Cloud. This was multiple years ago, they looked at it and this was one particular agency. They said, "In order for us to be secure enough, we actually need to go to the Cloud." Now that seems to be the opposite of what people think about right now.

Oh no, we don't want to go to the Cloud. That could be a breach. But they looked at it from an economics perspective and said, "For us to have the same number of professionals, with the same level of training, with the same SLAs, with the same contractual obligation to keep us safe, would cost us this many millions. Hundreds of millions."

A Part of the Opportunity

Tracy: Where if I have that through my contract with a provider like Salesforce or Service Now, or AWS, or Google, any of them, I have that out of the box. I have that as part of it. For the most part, I have that as a part of the opportunity. That's one thing that I saw the states do a little bit before the federal government. I’d say the federal government is tighter in its cyber practice, absolutely. Tighter in its cyber practice, but it's like any technology. It's not about the tech, it's about how we apply it. So how about we go about problem-solving. We do a lot of things in the commercial space. I did a lot of commercial work.

The commercial is a little less fettered. They're not as tethered to what the policies and the bureaucracy can be. Federal bureaucracy is a little more impacted by the administration changes. Trying to think of what the acronym was the other day. Somebody was joking with me. Instead of meantime to recover MTTR as a DevOps, it was a meantime to command change. It was MTTC or MTCC. Because you can figure out how much disruption was going to happen, and that's something very different.

The states have much more continuity with their technology leadership in specific, than I see in the federal space. The defense definitely, because of the time you do rotations, you may be there for six months, a year, or two years. You see it more in this less civilian side, less frequently. But still the federal is much more changeover, than I see at the state level.The Challenge With DoD

Carolyn: The defense, Mas, that's your world. I could see a little bit of two things with that command change. Fresh eyes, means fresh ideas. Also, it feels like it could be a huge setback.

Steve: The other challenge with DoD that we run into is it's everything about the network edge security. Everybody is commercial and never gone in civilian, not all a bunch of places, but the vast majority is civilian. We're using the public internet, we're using everything that's out there. They have access to everything that is commercially sound. In defense, it starts with, we don't have what we don't have access to. You start with what you don't have access to and you've got to build off from there.

The problems are obviously incrementally different. But the problem is also we can't take advantage of a lot of things that we can. We have to constantly fight that battle on both. As technology advisers, we have to go in there with that understanding that we're talking about solutions and products and technologies that sometimes you can't utilize all of the capability. Can we utilize enough to actually make a difference in your mission? Even beyond the fact that the kernels aren't going to be rotating in and out correct.

Tracy: It's interesting. There's always an opportunity for those fresh eyes to come in and infuse a lot of new energy and thought. But that depending on how large a program is, it takes them maybe a year to 18 months, to really hit stride. And so, there's a lot of churn that can happen during that time. Sometimes you'll see that folks will entrench themselves like, oh no, there's a change of leadership coming. I'm going to stay the course.

Specific Thing About DevOps

Tracy: While there's a leader that's talking about this amazing different type of infusing, of innovation, those in the trenches are saying, "Okay, I'm going to stay the course until we figure out how this solidifies, or how it plays out a little bit more." It causes a little bit of tension back and forth. But Steve, you bring up a really good point. I think about DevOps in specific. On the defense side, I jokingly say that they like the term DevSecOps.

But it's really DevSec, pause, wait, there's some other things going on, then SecOps. So the idea that we see of that infinity loop, right the figure eight, for there to be constant and continual feedback from the war fighter, from the constituency, to the developer. It's more difficult because there are different groups that are in charge of it.

I don't mean a guy in the other room is in charge of operations, versus me here in charge of Dev. They may be contractually a different part, they may be totally a different part of the service. Think about deploying onto a Naval vessel. Ops, there is a lot different than Ops that would be CONUS. Continental US sitting right beside me. It does have some pretty different challenges and is not insurmountable.

There's so much goodness that we should be looking at North Star to Steve's point. What's happening in commercials and then, what's my problem? What is the actual thing I'm trying to do? Then looking at commercials and saying, "What applies here? And what could be tailored? What could be improved and brought in?"

The DevOps Pipeline

Tracy: As opposed to, I'm going to go do what they did over here. I always use the example and it's trite at this point because so many people have heard it. But we're not just going to Netflix this. It's not just 50 releases a day. If I'm putting software onto a tank, if I'm putting it onto a jet, I'm not going to release 50 times a day. I may release a couple of times a week, but not 50 times a day. And so, there's economies of scale and things to learn with that.

Carolyn: I want to go back to that feedback loop that you talked about. Have you seen an agency that does that really well or a group? We don't even have to name names. Because just what you said, putting those software releases onto a tank, where we're talking about lives on the line. You need feedback from the guys operating it. Have you been in situations where that's just really smooth and they figured it out as a well-oiled machine?

Tracy: Yes, at scale. Not as much, but definitely there're amazing pockets of goodness. If I think about the things that are happening within the Air Force, if I think about things that are happening within the Navy and specific things with the Space Force, there are some fantastic loops that are going on. It depends on what the type of software is.

Also, it depends on what has to be done for fielding it ahead of time. There're policies and procedures that are in place, that would say I can't take that code that Tracy wrote. That she committed, that it was unit tested, that it automatically went through all of those things that we think of as the DevSecOps pipeline.

An Operational Fielding Exercise

Tracy: It actually has to go through an operational fielding exercise, before it could actually go into a war protection type scenario. Think of it like when people talk about the sequence and getting to pre-production. A lot of things happen in a beautiful figure eight up to this pre-production. Then there's one additional step, which is to live in the field. It's almost as though you have the figure eight and then another little punctuation off to the side. Steve, is that your experience too?

Steve: I think so. You differentiate when we think of war fighters, we always go right away to the weapon system to the plane or the tank. And now those software engines are a little bit different than all of the other business systems that are still out there. The vast majority of software being developed for them is still through the software batteries. It’s really more about the business logistics of doing everything else. Not necessarily command and control that the missile fly straight, that's being done at a very discreet kind of lab oriented.

But everything else, which at least looks on the surface anyway, with all of these software factories, the government is trying to move faster in that it embraces the CIT, the pipeline, and does more things in the Cloud. So yes, I'm encouraged by it. There's still a little bit of a disconnect between the CIC of the pipeline and then the ATO process. There is always this big Cloud around the ATO process, which does put a monkey wrench into things. Because every time you change any aspect, could we have broken something that could cause a security vulnerability?

The Purpose of Doing DevOps

Steve: How do we get around that? How do we make that faster?

Tracy: There's so much goodness that's happening now, to focus on CATO, Continuous ATO, the authorization to operate. It's a good debate on how real that CATO is, the ATO process as well as a platform that's underneath it. Then the thing that you need to look at and audit and be super focused on, is what's moving across the top of the Delta and the change? But that means that your pipeline needs to have a tremendous amount of auditability. Instantaneous audit ability throughout that process.

The RMF process in and of itself, is a good and strong framework. What's difficult, is helping the cyber professionals become part of the earlier parts of the design. I did a Navy project, I guess this was about two and a half years ago. And I really learned so much about the RMF process during that.

RMF, it's a Risk Management Framework. It is a way that you assess and evaluate a project, or a system, or a product before it goes to production.

Normally, the feed into RMF is that you have all of the designs complete, all of the boundaries, all of the information flows, everything complete. I thought I had a brilliant idea that I said, "Okay, guys, I've been hanging out during DevOps things. Not for the purpose of doing DevOps, but I've been leveraging those capabilities for many years, to take systems into production." I have horizontal teams, I want transparency for anybody who's involved in this. Hey, you RMF guys, those folks are going to work on our ATO, come on...

What matters most when it comes to providing the better experiences for our citizens? Listen as Jonathan Alboum of ServiceNow outlines the importance of preventing information from getting lost in the sea of data.

Episode Table of Contents

• [00:45] Moving Fully Into Better Experiences
• [09:15] Providing Better Experiences More Effectively
• [14:41] A Control Tower Approach

Episode Links and Resources

• Sapiens: A Brief History of Humankind
• Transforming citizen experiences in federal healthcare agencies

Moving Fully Into Better Experiences

Carolyn: We're excited to have Jonathan Alboum, who is the federal CTO of ServiceNow. He was formerly the CIO of the U.S. Department of Agriculture before moving fully into the service industry in 2019. I don't like that term, "moving fully into the service industry". When I read through your bio, I feel like you've been in the service industry your whole career.

Jonathan: Good point, I began my career after college. I went to UVA, same place that Mark went to school, but he went a few years before I did.

Mark: You had to point that out Jonathan, thank you.

Jonathan: He went when it was really, really fun. I went when it was just fun. But you know, when I exited college with a systems engineering degree, I joined a professional services firm. I worked at Pricewaterhouse and I learned management consulting, and we were doing services. We were implementing systems and different technologies for customers. Eventually, I moved to a smaller company. I was working on the same kinds of projects, doing professional services, helping organizations do modernization.

We weren't focused on this big grandiose term of IT modernization or digital transformation, which were the same kinds of things we were doing. We wanted people to have better access to data and systems so things could happen more efficiently.

It's the same kinds of things we're doing today, even if we use different words. I was in a service role before the government. When I joined the government at the food and nutrition service part of USDA versus the deputy CIO and then the CIO for that agency.

Providing a Strong Service and Better Experiences

Jonathan: We're a service provider to the programs at the food and nutrition service. So if the office of information technology at food and nutrition that I was responsible for wasn't providing a strong service, the programs would find another way to get their job done.

We could be shut out of that conversation. That's a terrible place for a CIO to be at. Cut out of the technology or the core business processes or the budget. So I always had a strong focus on service and that reality continued through my career. As I moved to general services administration, or back to the department of agriculture, to be the CIO for the entirety of the department, you have to be able to provide a service.

I eventually exited government and found my way to ServiceNow. I’d say the lessons I learned along the way are in terms of what it takes to provide a service that people want to use, and that they will partner with you on. Those things really informed the way I go about doing my job at ServiceNow. ServiceNow, it's a technology provider that supports this idea of service management. It's not customer relationship management.

We have tools for that, but you have customers, you want to provide them a service. Well, you have to think about the end to end workflow. How does the person interact with the system and what are they trying to get out of it? You think about it comprehensively.

I feel like I'm well positioned to do that based on these other roles that I've had. I've either been the creator of a service or the provider of the service. Now I can really think about it in a holistic manner.A Pioneer of Digital Transformation

Carolyn: I like what you said about digital transformation. We didn't call it that at the beginning of your career, but really you're a pioneer of digital transformation. So not to call you old.

You've really built your career, figuring out how to make things better and faster and really through digital. As I looked at some of your stuff online, you're all about data.

Jonathan: I have a person I want to thank for that. Her name is Kelly Chambliss. She was one of the first managers that I worked for at Pricewaterhouse and I haven't seen her in many years. I always remember her because when I was very early in my career, it was my first Pricewaterhouse project for an insurance company. It was a visual basic project and it was like 1995 or so, it was a long time ago. I remember telling her, "I'm going to really learn visual basics. I'm going to be this great visual basic developer".

She looked at me, sort of funny, and said, "you know, maybe that's not the best path for your career, technologies come and go. But if you focus on process and data, you'll be really valuable on this project or any future project that you're on." That was really excellent advice. As you point out, a lot of what I talk about is data and how data moves through the environment. How the work moves through an environment and thinking about how you can automate some of those things.

The Data and the Process

Jonathan: It's about the data and the process, and you can swap out one technology for the other. You're still trying to solve that same problem. We just have better ways of interacting with the data or the process today. Or we have more ways to automate it or more people can be dealing with the same data at the same time. We can have cloud technologies and other things that give us the ability to go faster, maybe.

But still, at the core are these things that I learned on that first project about thinking about a process. Thinking about data and trying to improve it. We have more tools to do that now. It's the same motion, so it's a really good observation, Carolyn.

Carolyn: I heard an interview with the author of Sapiens. I don't know if you've read that book. It doesn't really matter. What he said was, there we go!

Jonathan: I just recommended this book and it arrived the other day. But I haven't picked it up. I haven't had a chance to read it yet, but it was highly recommended by a friend of mine.

Carolyn: So, same. I have not read it, but it keeps coming at me, which for our listeners, Jonathan just held up a copy of the book. Now, I got to get it too. What he said was "Whoever owns the data, controls the data. And here's the key, can understand the data is going to win." Then something else said what was really funny to me. It was, "Never underestimate the power and the limitlessness of human stupidity." Looking at your profile, it just made me think of the data. Like, what do we do with all this data?

Establishing Better Experiences Is a Big Challenge

Jonathan: That's a big challenge. There's more data today than there was yesterday. Tomorrow there'll be even more data in the world and on and on and on.

Carolyn: Let's qualify it. When we say data, like we kick that word around, but give it to us in billions. What are we talking about here?

Jonathan: Well, when I talk about data, it's both the data that's in a system. We might be interacting with the system or we're putting data into a system. It's also all of the things that get created as we do a particular business process. The forms we fill out, the pictures we take, the conversations we have, the videos we create, all these things are data. They all have a role in informing someone about the process, the person, and all of that.

It's very easy for the information to get lost in the data. Because, some of the data's not important, some of it is really important. Figuring out what's really important is part of the key.

You go back to that process of conversation a little bit. If we put it in a business or a government setting, we're trying to improve a service we're providing to a customer or a citizen. Thinking about it that way, you can really understand how the work flows through the organization, which is the data's representation of what that work is. You can be focused on the things that make the biggest impact and you can really find, sometimes we'll call it "the moments that matter". Where either data is getting created or data is needed.

Providing Better Experiences More Effectively

Jonathan: If you can get quick access to it, now you can provide that service much more effectively. One of the ways that I think about this is all the data that exists in the environment. I'm trying to provide that service to a citizen. It might be in different systems, or it might be in spreadsheets. Or it might be on a piece of paper that I printed out.

If I have to move between all those sources of data to get you the answer that you need, I'm not really providing you a very great service. So one thing is understanding where all the data is and the next part is being able to connect it. Once that data is connected and you can bring it together on a single platform, a single place to go to see it, I can now interact with it and get you an answer much faster. I can provide you a service much faster.

When we're doing that in a government setting, we're providing a really great service to the people we serve as government employees or people that support the government. Well, now you begin to change the way people think about their government and you begin to inspire more trust. We create more engagement.

If we can think about technology and data in that sense, now our jobs, in the federal IT community and in the broader technology community, they take on a little bit of a higher calling. Cause we're really focused on ways that we can strive to a better society. That might be a grandiose idea, a pie in the sky. But I mean, I believe those things, cause I think it does make a real difference.

Prioritize the Citizen’s Better Experiences

Mark: When you talk about prioritizing the citizen experience to improve quality of the services or what they're experiencing, are you talking about applications in technology? Are you talking about the quality of the actual service that they're trying to receive or both?

Jonathan: I always try to think about it from the customer's perspective. I want to try and take an outside-in view of these things we're trying to automate or digitize. If I'm thinking inside-out, I'm thinking about my agency or my government program, I'm creating a system that works in a way I want it to, as the government program manager. There might be a citizen interaction, but I'm thinking first about me and my job.

I'm not necessarily thinking about that person who's using the system, who might also say, I'm at my government program and I have a set of customers. They are probably customers of other similar programs. If you think about entitlement programs, say SNAP or WIC or unemployment insurance or Medicaid, you may have the same person in multiple programs. And if I'm a state government and I'm creating opportunities for people to apply for these programs, there might be four or five different applications.

If I can think about it from that customer's perspective, they're trying to do three or four or five things kind of around the same time. Can I design with the human in the middle, with the human at the front? A human-centered design approach that creates opportunity for them to be a lot more efficient and effective than signing up for these programs. Can I take information from one application and apply it to the other application?

Privacy Requirements

Jonathan: These things are very logical and it makes sense. Sometimes there are regulations or privacy requirements that might get in the way of that. You know part of it's having the conversation and the dialect to understand what's possible. What are the limitations? And engaging in a discussion about, "can we change some of these things to make it easier for people to interact with their government?" People shouldn't be forced to interact with the government.

People should have the opportunity to interact with the government in the way they want to interact. They shouldn't be forced into a certain way because we have legacy technologies that don't interact.

The state is not integrated, that Jonathan is represented differently across five systems and someone who tries to look at the opportunities from a comprehensive, we can't do so easily. It's not easy to see that I'm the same person trying to do multiple programs because I have a particular need at a particular time. That's the idea of citizens-centered services.

Carolyn: I love that idea of citizen-centered services. Then I think about the poor guys trying to make this citizen-centered services because now you're talking about all these different applications. The data just got exponentially bigger. How do we manage it?

Jonathan: If you’re starting from the very beginning, you’d take a different approach than, maybe you have to take today because we have what we have. If you are going to build it from the start, you could design it with a mindset of "I'm going to create a central hub with these different capabilities and build it a certain way." But we can't undo or we can't change all the systems at once.

A Control Tower Approach

Jonathan: What I’d like to talk about is this idea of having something that is a connected tissue across programs, the connected tissue across systems. It's almost like a control tower approach for these capabilities. You can connect systems and data. You understand what the processes are, how that work flows. Now you're able to begin a conversation about the customer experience and how it may work across these systems.

You don't have to be focused on individual system modernization multiple times, so across all your systems. And you may want to upgrade those systems over time. They may need modernization to become more secure, or for some, maybe the technology is no longer supported. But, now you're doing that in a more controlled manner. You're not doing that with the urgency of providing a better service because you've taken that interaction layer and integration layer. You've moved it up, you've connected the systems. Now over time, you can swap out the things underneath that are connected to that platform.

That's sort of a platform as a strategy. Sometimes I'm out there talking, because there's a faster, lower-risk path to doing some of these more digital services. It's a faster way to digitally transform the way we interact with our citizens. It drives us towards more of an anticipatory government approach when you think about the citizen comprehensively, I can predict some things they might need. Or I can reach out to them proactively and provide them information. I'm thinking about the citizens as a whole. That's what we want to strive for.

Mark: Are you seeing this concept, these concepts get adopted in government?

Better Experiences At the State and Local Level

Jonathan: Yes, I am. As I talk with different agencies, I interact with some of my colleagues who are thinking about this at the state and local level. We have these conversations with CIOs. From their perspective, the modernization of all the individual systems is not the way to speed that digital transformation.

Carolyn: I want you to repeat what you just said. Modernization is not the way to speed.

Jonathan: Let me elaborate on that. If I'm going to try and modernize individual systems in the example I was talking about, I'm going to go system by system. Try to modernize from a mainframe to some cloud-based modern system. I have to repeat that for all of these programs or all of these services I provide. Those are really hard, complicated projects that require new technology skills that require a lot of change management.

You may be running systems in parallel for a period of time. There's a lot of risk with that. So you can't do that across all your systems. The conversations that I've had with different CIOs at the state and local level, and the federal government, it's about how do we provide digital services faster?

What's a way to speed the digital transformation of the way we interact with our customers? That's more of a platform approach that is the connective tissue across systems where the data can be integrated. The systems can be integrated and you can drive to a citizen-focused outcome a lot faster. Again, we still have to think about modernizing those underlying systems. But if that's where you're starting, you can easily get bogged down and the results you're after.

Legacy Systems

Carolyn: Well, I asked you to stop so we could repeat it because what you said feels a lot more doable. I literally felt myself breathe, like, "oh, we can do that." But to take on the legacy systems is overwhelming.

Jonathan: I totally agree, you have to understand them for sure. You have to know what you're dealing with. And you have to understand the data in those systems. We were talking about data before. If you don't understand what and where all the data is, it's really going to be very hard for you to protect all of that data. Make sure you have good data protection to make sure that you have the right level of security.

But there's a difference between understanding all of that information on the systems. And taking a perspective that I need to move off of all of these legacy technologies at once, so I can have a modern environment. I just don't think that's a practical approach. Because you know, CIO's, they have to operate very complex environments every day. Those technologies and those complicated environments only exist because there are technologies.

There are government programs that require those technologies. It's not IT for the sake of IT, it's IT in the spirit of customer service, citizen service. So you can't commit to modernizing everything and expect to be able to provide un-disrupted services. It just feels too risky from our perspective.

Mark: Or maybe agencies were forced to embark on this transformation before they may be ready to. For example, the pandemic. I know you recently gave an interview that you talked about.

Digital Transformation For Better Experiences

Mark: You said you felt like agencies should take a pause and their momentum or their digital transformation. Consider whether those processes were put in place because they were forced to. Given the circumstances with the pandemic or whether they're sustainable, or they make sense.

Jonathan: I stand by those statements and I talk about it as a strategic pause. One of the things that I've learned is that sometimes if you go too fast, you can create some blind spots in your professional life, your personal life, whatever. We are pushing towards a goal. We're not necessarily considering everything else that's happening. We had to do that, we had great urgency in the early days of the pandemic. There are some technology opportunities and service opportunities that emerged from there.

Carolyn: We've already seen some huge technology advancements coming out of this pandemic. Thank you, Jonathan, for this great insight. It's been a pleasure to have you on the show and...

Listen in as Carolyn and Willie find out the true power of data. Sean Applegate, CTO of SwishData, explains how data can be utilized across an entire mission to empower the warfighter.

Episode Table of Contents

• [01:00] Smart People With Great Ideas
• [07:29] Your Data Can Work for You up to Some Extent
• [14:15] Securing the Application So Your Data Can Work for You
• [22:08] Eight Guiding Principles
• [28:03] Applicability of AI
• Episode Links and Resources

Smart People With Great Ideas

Carolyn: I'm Carolyn Ford and this week, my guest co-host is Willie Hicks, public sector CTO at Dynatrace. I'm super excited that we get to talk to Sean Applegate, CTO of SwishData.

Sean: I'm excited to be here, it should be a blast.

Carolyn: Honestly, this is the best part of my week. This is the best part of what I do. I love talking to really smart people with great ideas about how technology can better our lives and how the government specifically can do that. So, Sean, you've written a lot of stuff. You're a pretty prolific writer, blogs articles, and a recent blog that I saw, I'm not going to lie, it kind of broke my head. It was a lot of technical stuff, but there were a couple of things in it that were kind of gotchas for me. I'd love for you to drill down into a little bit.

At the beginning of your blog, you write, the name of the blog is Optimizing Mission Outcomes with Intelligent Insights. In one of the beginning paragraphs, you say transforming the DoD to a data-centric organization requires that data is visible, accessible, understandable, linked, trustworthy, interoperable, and secure. So I would love for you to dive into that.

Sean: I would say the one thing that DoD is noticing, and you'll see this with some of their DevSecOps reference architectures is it requires culture change. Whether that's the business leaders or the mission leaders, the contractors, the developers, the people running infrastructure, or delivering a service.

Your Data Can Work For You But It Has To Be AccessibleSean: They've identified that the data has to be accessible across all of those different parts of the mission. That getting that data collectively together is extremely important. It's valuable for both mission velocity and a competitive advantage around the world, whether that's DoD or civilian agencies, we see that as well. So data is critical, be able to find it first.

Carolyn: If you've got the data, what do you mean it's not accessible? Do you mean like across agencies or across groups?

Sean: A lot of it is making it not just within your command, but outside the command. So it's trusted. For example, I'm using an application performance management issue. I'm delivering an application, I have lots of stuff on the application. I may not have a lot of stuff on the user community, or maybe somebody wants to analyze the success of my mission. That mission can be measured, lots of different ways.

How do I merge those data points together? So I can draw, make a business decision from that, that's very impactful. That may be something at a very strategic echelon such as the Pentagon. Or maybe very tactical, down at the tip of the sphere, the unit deployed overseas. I need to make a decision right this minute. How do we do that? That's very complex.

Carolyn: One of the things that gets bounced around a lot these days, you guys have both talked about AIOps. Using AIOps to get us to this place, all of these things that you list. Can you talk about how AI ops enables this?

Your Data Can Work for You Through Problem Solving Complex Things

Sean: On the AIOps side, what we find is, it allows our human workers to better focus on problem-solving and the complex things you can easily do with the computer. The AI piece allows us to typically make linkage. If you think of linked data, it's the dependencies between data points or systems. In many cases, when we look at application performance management, a user might have an issue on the front end. We have to go, what was that issue? The network, is that the desktop? Is it the application web front end, or is it deep back in the database?

Being able to draw that picture out in the end so you can analyze what their dependencies are and understand them, and then do the root cause analysis to figure out where the problem is at is absolutely critical so we can solve those problems faster. And that's really what it's aboutl solving problems quicker, or building better performing team systems so that we can achieve our mission and make citizens happy or empower the war fighter at the tip of the spear.

Carolyn: I was going to ask you to jump in because the whole linkage thing, that's kind of boring when I want to talk about AI. I want to talk about the Terminator or data from Star Trek.

Willie: And Jarvis.

Carolyn: Exactly. Willie, first level, set us on AI. I'll try to keep my fantasies out of the podcast and then respond to what Sean said.

Willie: Just kind of a level set on AI. It's more of a question. All through these conversations with people, there's always a lot of misconceptions about AI, what AI is.

A Narrow Approach to AI

Willie: A lot of names get thrown around machine learning versus are we talking about more of a discreet AI model. All of these different types of AI models that might be out there. We haven't reached what you're talking about with this data-centric view and kind of utilizing AI. I see that more as a very narrow approach to AI. Very narrowly focused on that skillset and not what people think of as general AI, which we haven't reached yet. You hear about IBM Watson, that's a long way from doing the Terminator. That's kind of what I see. Is that the case?

Sean: That's a good summary of it. If we looked at solving problems quickly with technology or making things better with technology, if you consider that artificial intelligence, you can really do things with basic AI today. So if you thought of, I found a problem, the next logical step might be, can I fix it automatically? Or can I build a little bot that can go fix it automatically? We're starting to see that with things like robotic process automation, for things that maybe aren't easily scriptable.

But the citizen developer might be able to build that process into their day-to-day job. That job might be IT operations or application development or running some infrastructure. We've done that in the past with some existing government clients, maybe writing something to analyze complex analysis. When you think of site reliability engineering, you could write some really basic AI or machine learning scripts. Where you could analyze dependencies across functions that you need to monitor in your job that are unique.

Your Data Can Work for You up to Some Extent

Sean: That industry can't do themselves, and you can take something like a TensorFlow or a Pytorch. Do some analysis of basic data sets and do that enroll at your own to some extent. Or unlock those things in a cloud, if you have access to something like an Azure or an AWS where you can do some AI things in the cloud, but you can easily get the data that's accessible and understandable and snap it into that fairly easily.

Willie: Just touching on then taking that to what we're talking about from your blog. I did not read the subsequent DoD document, which was about 700 plus pages. Thank you for that breakdown. What I saw there and why I think AI clicked is that you're talking about at first, the data. We're just talking about data, but we're talking about the full, you're talking about infrastructure. You're talking about the columns between all of these data points. How we secure access, everything needs to be CAC enabled and authenticated and all that good stuff.

But also it came to mind, I've been reading and talking a lot lately about the fall of JEDI, which sounds like a movie. But the new JCW, the new contract that is coming out as a replacement for the JEDI. A lot of that is going to drive this new joint, all domain command, the JADC2 initiative where it's all data-centric. That's tying a lot of data from the battlespace and the tactical edge with a lot of what I don't think we talked about yet.

Sensor Data

Willie: Even a lot of this is going to be sensor data. We started talking about IOT. You started talking about bringing all of this data in from, could be tens of thousands, hundreds of thousands, maybe millions of data points. Something has to parse all of that information to get the right relevant information to the battle. The commanders and the people who need that are allied forces or whoever subscribes to that data. That's where AI also needs to be leveraged, especially when we're talking about what you wrote in your article. Am I close to accurate?

Sean: Absolutely. In fact, there's a program called Advana, which is the DoD program. There's a bunch of approved tools for big data in AI that are included in that. One of the biggest challenges is how do you do that at scale? They're at the very early part of that journey where they figure out, how am I going to do this? They are doing it at the OSD level today, but how do I didn't do that at a mid-tier command? Or how do I do it at the edge of the battlespace? How do you do it in a jet as you're flying and you have to get telemetry off the jet at the end of the mission and analyze as part of the mission.

Those are not small challenges when you think of the massive amount of data across the Department of Defense. How do you make sense of that as a community? Part of that is getting that data into a data lake or a data warehouse somewhere where people can access it. And then do things with it that are valuable because that data has value to it.

Your Data Can Work for You During Time-Sensitive Situations

Sean: Often that's a time-sensitive situation where you need to analyze it within minutes or days, not months or years.

Carolyn: JC2. I love the idea of it. Define JC2 for me, Willie.

Willie: The joint all domain kind of command and control initiative, or it's JADC2. It's this idea born out of the communication signaling part of the military where they have all of this data and Sean is spot on. I'm not a military person, not trying to pretend like I'm military. But I can see, the vision here is that the wars of the future are not just going to be fought with bullets and putting steel on target. But it's also who's going to have control of the data and that space, and who's going to be able to find answers and execute on a mission faster.

This is a new arms race, this is why AI is so important. This is why you see all of this talk about AI and how the Department of Defense and the US need to be really focused on our AI capabilities. We can bring to bear the technology we're going to need to analyze all of this data from the data space. To Sean's point, this data could be coming from land, sea, under the sea from, literally soldiers on the ground who are wearing sensors.

It could be coming from satellites, whether all of this data has to go in. Again, not being military, but I understand that all of these pieces have to be aligned. To get a good view of the battle space, understand where you need to have your troops, how you need to have them there.

How Best Decisions Are Made

Willie: What do they need to be equipped with? All of these things need to be understood. The best decisions can be made and whoever can make those decisions the fastest. To Sean's point, there used to be a time we could make these decisions on days. I'm sure back in world war two, and you look at the planning of D-Day and things like that.

Things that were in the works for days and months, we might have hours or minutes to make a decision. Humans just can't, it is just impossible to parse all of that data. To make a good decision without decision support services from something like an AI. Does that make sense?

Carolyn: Yes and what you said, I translate it as we're going to have this central command and control for all DoD, maybe even some fed civ agencies. Here's why I say it kind of scares me and coming back to you, Sean, how do we secure it? How do we trust it? If we've got it, it's coming in from everywhere.

Sean: Generally speaking, if you look at the zero trust piece of it first, let's break that down. There's the zero trust architecture. So NIST 800-207 and there's the DOD zero trust reference architecture, which came out about three months ago. There are seven pillars. But if you break them down to the most basic functions, it's about securing a device. Making sure it's not compromised before you let in the environment, securing the users and the applications. Typically what we find is securing users and devices are the easier pieces.

Securing the Application So Your Data Can Work for You

Sean: Securing the application and when we say securing, not just compliance, but actually knowing and measuring that it is secure in real time. Finding the open-source module or the function or method that is secure, that the developer can rapidly fix on their own. It’s where AI can definitely help us find that because they can measure those things with APM technologies, with integrated security. We can trigger and tell the dev ops team or the no ops team, you have an issue.

Go take care of it immediately. If they're managing a small service or a function, they can go fix that in a couple of hours and it's fixed. We're starting to see that in the platform one environment and DoD, where they're patching containers every day in 24 hours. You have a team patching the Tomcat container for the web front end every 24 hours. But the rest of DoD subscribes to that container, that hardened container.

They're getting that patch and leveraging that fix without having to do actual work themselves. Getting that team that runs that container, that owns the security for that container DoD-wide. Where they can patch it as fast as possible and know the exact function or method they have to fix is important.

More importantly, if a large percentage of DoD applications rely on a core set of containers being shared in the community, you also have to make sure those containers operate and perform meticulously. If I have a team that's supporting that, they have to make sure it runs well. They QA it properly, they pull their performance testing left into their dev cycles.

The Integrity of Data

Sean: When they publish it every 24 hours, which is a lot of publishing a year, that it is running smoothly and not having any problems. Then the question becomes, how do those teams then monitor those in production at scale, if they're across hundreds of applications, for example, across DoD.

Carolyn: I heard a couple of things from another show. My past life is an insider threat. For you to say that it's harder to secure the app than the user, we can debate that later. To be secured, what I heard was AI DevSecOps. We're baking it in at this ground level, we're using AI to do it. That means it's coming, it's the integrity of the data. The integrity of the containers are built from birth.

Sean: That's a general way to approach it. It depends on what you mean by birth, but yes. If you've mentioned having a birthing a new baby, every 24 hours, sure. It comes from the top down. Because the team's going to turn over, in this case, we're really treating them more like cattle. If you want to use the DevOps term, we're going to not treat it like a pet. But to that team that manages that one container or those five containers that a lot of people use, that's a very important asset in their life that they have to care and feed for and nurture.

Those things come in lots of different flavors. But if you're a developer, you have to own everything about that container, that function that you're going to share with the rest of DoD and the community. So how do we make sure it runs well and it's secure? Then the question might be, would you?

How Your Data Can Work for You From a Data Accessibility Standpoint

Sean: From a data accessibility standpoint, I'd like to know how those containers are working not just in your application, but in other applications around DoD. You can make a lot more decisions and support it better if you can then access data across the organization and pull and work together across say a 10 application team. You're supporting them in ways that they care about that affect the mission.

Carolyn: Once you plug your container into the mothership, then you can send sensors out and see how it's integrating, assimilating to everybody else.

Sean: Sure, if you want to get the sensors for a minute. If we talk about applications, specifically containers, you can either go with some type of open sensor for application performance management to get things like metrics, logs, and transactions out of it. You could use something like OpenTelemetry or Fluentd or Telegraph or Statsd. There's lots of options that are open source that are supportable across different application performance management platforms. Or if you want to use the word observability platforms, those as well.

The question for the government might be why wouldn't you embed that inherently, those sensors, if you offer for a DevOps team or an SRE team inherently as part of your build cycle. They are there, and then you can leverage them across lots of observability platforms. Then an organization can pick the one that's best for them that they liked the best. Or maybe they can pick the one with the most advanced AI functionality.

One of the Hardest Problems

Sean: That's why sometimes getting the data into your platforms from lots of systems in DoD is one of the hardest problems. Because they've got to go through an authority to operate the process and to get that approved. It's great, but making changes to it after the ATO sometimes can be a little challenging. If you build your sensors and as part of that process inherently, it becomes a lot easier to get the data out later, in a more open fashion, potentially.

Willie: That actually begs a question I had and this is a slight tangent, but I'm just curious. You triggered a thought in my mind, Sean. Are you seeing from your customers and the clients you work with, are speaking or baking this into the product? So there's these new concepts around like open telemetry and building this type of telemetry into the application. It can be exposed at runtime and be pulled in by any number of these tools that you're talking about. You can get a complete, more full picture of the landscape. Is that something you're seeing as well?

Sean: We are seeing it in different agencies. Some of the civilian agencies have been more focused on things like distributed tracing and writing things into their application code. While that's a noble effort, it leaves a lot of the infrastructure not covered. It becomes hard to connect the dots from your application code or the web front end, which normally the app guys have pretty well covered. But then, if you look at cloud infrastructure or on-prem hardware infrastructure, it leaves it uncovered in most cases. If you consider the network, another piece of that you want to pay attention...

Risk taking is unavoidable when it comes to modernization. Best selling author and retired four-star U.S. general Stan McChrystal outlines 10 control factors to help citizens and agencies alike take smarter risks. Carolyn and Mark also get some early insight on Stan's upcoming book Risk: A User's Guide.

Episode Table of Contents

• [01:44] Stan McChrystal of the Team of Teams
• [10:54] A Story of a Defensive System by Stan McChrystal
• [18:26] Stan McChrystal Talks About Inertia
• [25:57] Why Stan McChrystal Doesn’t Want to Worry About External Threats
• [35:40] Artificial Intelligence Based

Episode Links and Resources

• Risk: A User's Guide
• Team of Teams

Stan McChrystal of the Team of Teams

Carolyn: We are joined this morning by retired four-star U.S Army General Stanley McChrystal. We've had the pleasure of talking to General McChrystal on a few occasions. Good morning, General McChrystal.

Stan: Call me Stan, and it's an honor to be with you again.

Carolyn: Let me give our audience for those that have been living under a rock, just a few more of your credentials. So Stan is a former commander of the U.S and International Security Assistance Forces, ISAF Afghanistan. He’s the former commander of the nation's premier military counter-terrorism force, JSOC. He is best known for developing and implementing a comprehensive counter-insurgency strategy in Afghanistan. For creating a cohesive counter-terrorism organization that revolutionized the inter-agency operating culture. Is it fair to say, Stan, that’s the basis for your book Team of Teams?

Stan: It was certainly the foundation of it, and then our study beyond that.

Carolyn: Honestly, Team of Teams, just a little plug here, is the best book on leadership that I have read. If you haven't read that one, do that. But we're here to talk about a new book that will be out this October that is also sure to be a bestseller. Mark and I got to have a sneak peek. We got to read an early copy of the manuscript. We're here to talk about your new book, Risk: A User's Guide. We’d like you to talk to us a bit about the 10 risk control factors, and the four measures that are the foundation for your new book Risk.

Calculating Risks

Stan: We decided to take on Risk as a subject because through my career, there had been processes to follow. Calculating risk and acting on that too, to be able to measure the threats or risks to your organization. But it never connected with how we actually did it. Now, certainly there are some financial firms that use financial models that theoretically do this. But if you look at so many things in our lives, there's one way we talk about risk. Then there's another way we actually respond to it.

I wanted to understand what the disconnect was. Of course, that had been my experience as well. In most cases in my career, we had done checklists or matrices, and calculations to come out with a risk score. But the reality was most of our reaction was intuitive. And so we decided to study risk. What we came away with was the idea that each organization and individual actually, but organizations particularly have something which I'll call a risk immune system.

It's a system consisting of 10 factors, such as communication, diversity, bias, timing. Those things interact together to determine the health of the system to respond to and prevent threats from undermining the organization. It's very much akin to the human immune system. If you're not familiar with it, the human immune system is a marvel. We face about 10,000 pathogens a day that come to our body. Any one of which could harm us or kill us, but we don't think much about it.

The Theory Behind Vaccines

Stan: We don't have to, because the body has got a process in which our immune system detects all the threats that come. Assesses each one, responds, kills off the ones that need to be, and then learns from the process. The miracle is it gets smarter and that's the theory behind vaccines. You build up an immunity so that the next time it's very easy to fight off that known threat.

The human immune system is this marvel that we sort of go through most of our lives taking for granted until it's compromised. Then it's compromised like with HIV/AIDs or another assault on our system. When it's weakened, suddenly we fall prey to threats that otherwise wouldn't be a problem for us. Really, nobody ever died of HIV/AIDs itself. What they died from was other lesser things that the body was unable to combat.

Now we come back to organizations. It was interesting we started to write this book and do the research and COVID arrived. It became almost a perfect example of what we were talking about. If we think about COVID-19 someone says it was a black swan event. Who knew that a pandemic was coming? The answer was, everybody knew. We'd been through it before many times in world history, particularly the Spanish flu.

Most notably in 1918 and in a smaller sense, many times since. In fact, in 2019, just a few months before COVID-19 arrived, the Department of Health and Human Services ran a series of exercises. They were called Crimson Contagion and they were based on a scenario of a viral threat. Pathogen coming out of China, coming to the United States, going around the world and wreaking havoc.

Stan McChrystal Talks About COVID-19

Stan: The lessons from that set of exercises was, the United States had not done enough preparation. Had not stockpiled enough supplies, had not worked enough of the processes, and therefore paid a heavy cost. That was only a few months before COVID-19 arrived. The interesting thing about COVID-19 is we knew the threat was inevitable. You don't know the exact strain of the pathogen, but you know that kind of assault is inevitable.

The second thing is you know exactly what to do about it. Public health is not a new science, we knew the basics that we had to do. We had to stockpile things that would take certain steps. Of course, we pulled a rabbit out of a hat in terms of developing vaccines faster than any time in history. But except for that, the world's response to COVID-19 has been very weak. It's been very weak because of many of the risks in the immune system.

I would argue the society's ability to communicate effectively, to make decisions on time, to overcome the inertia against inaction. To have the kind of leadership that emerges, that brings all capabilities together. We literally stumbled on every one of them on COVID-19. It's a tremendous, but sad example of how important, because this wasn't a scientific failure. In fact, COVID-19 is a scientific triumph. It is a societal and governance failure in our analysis.

Mark: Or leadership failure, if you look at the 10 dimensions of control. How did you come up with the 10 dimensions of control?

The Antidote Stan McChrystal Concocted

Stan: When we decided to take a look at those things which were important factors. They were distinct or different enough to be categorized, you probably could have had 12. You probably could have had 8, if you'd put something together. There was some bias and diversity. They're a little bit akin to each other. If you have biases, the antidote is diversity, different perspectives.

They could be linked to communication and narrative, but they're also distinct into themselves. We wanted readers to understand, these factors are all things which would be consciously addressed by an organization. Trying to both be sure it has a healthy risk immune system and then improving or strengthening it.

Carolyn: One of the risk factors, control factors is technology, which I want to focus on since this is Tech Transforms. You start that chapter of technology with a quote that I loved. So you said, "Technology raises a new question, who or what is in control?" This is something that I think about almost every day and have since I was a kid watching Star Trek. And this got my head spinning about how we ensure that technology is an advantage instead of a disadvantage.

Who's in control? Is it helping the agencies that are using the technology within the government? Is it helping the citizens and warfighters that those agencies are serving? Can you talk about that? Who is in control when it comes to technology?

Stan: The answer is it should be us. But if we go back in our history, we refer to the movie Fail Safe, an early 1960s movie in the book. If anyone hasn't seen it, you ought to watch it.

A Story of a Defensive System by Stan McChrystal

Stan: It's a story of a defensive system that has been implemented by the United States based on technology. It allows the United States to essentially be able to strike the Soviet Union without being stopped. It's got a defensive aspect to it, so it can analyze whether there's a threat coming and then launch a counter-threat.

Of course, it malfunctions in signaling that there is an attack and then it launches a counter strike. Humans are not able to recall the counterstrike. So in the desperately tragic final scene of the movie, the president of the United States works and deals with Soviet leaders. After the United States bombs Russia, which we cannot stop our plane from doing, we bomb New York City ourselves as a tit for tat to prevent further war.

Now that question you say, who's in charge? The answer is, because of the dependence upon very highly technical devices, they can get ahead of us. If we fast forward to 60 years and we've got artificial intelligence, and we've got things like hypervelocity missiles, we've now got response systems where you have to let the machine respond based upon inputs from its collection. There's no time to put a human in the loop.

We always say, we'll always put a human in the loop if it's got anything to do with lethal effects. You can't do it and make it work. The reality is you either depend upon that, or you have a much slower human system. Which probably is not fast enough to deal with some of these threats. We're building threats that make us dependent upon technology-based responses.

Human’s Last Touch

Stan: At a certain point, the human's last touch of this thing is when we craft the system and if we get it wrong, or if someone spoofs the system or corrupts it in some way, there's tremendous vulnerability.

Mark: I think of the movie WarGames in the '80s, where they had the WOPR. They were simulating nuclear attacks, ominously.

Stan: It's terrifying. We also refer in the book to some things that are more mundane, but they're pretty important. For example, most companies have implemented automated voice or automated telephone systems. You call your favorite company and they say, if your problem is X, dial one, or press one. You sit through this thing and you get more and more frustrated. And you want somebody to fix your problem.

It's much cheaper for the firm to do, but how many times have you taken your business elsewhere? How many times do you just say, "I give, I want to talk to someone who will accept my problem and fix it." That's a hidden cost or a hidden risk that technology gives us that we're not even sure we can measure.

Mark: It seemed to me, reading through the book, that the way you laid out the 10 dimensions of control in the different use cases as human analysis in decision-making, et cetera. Across that, the question I've got as it relates to technology and AI is, is there possibility of taking the risk immune system and the 10 dimensions of control? And apply that into artificial intelligence so that artificial intelligence can assist humans in this process move faster. Even in the book, a couple of examples, things start moving so fast. I wonder if we're too slow in that game.

Stan McChrystal Reveals What AI Could Do

Stan: I think we are. The first thing AI could do for a system like that is tell us what we're not doing. If you think about it, a problem comes, a fire starts in your kitchen, and you're worried about getting your kids out first. An AI system could pull all the factors together and it could remind you. It could say, "Wait a minute, you got this wrong. You haven't done this." Et cetera.

As conditions change, AI with the right amount of detection out could bring that information in. So that it could widen the aperture of the organization or individual making the decision and potentially respond more effectively. The problem is, we as leaders, have to understand AI much better than we do right now. We are going to get instructions from artificial intelligence in the future.

We're not going to be able to have the time to dissect them or through our own processes, compete with that. We are either going to have to accept it or not. It will say, do this. We're going to have almost an act of faith because artificial intelligence can bring so many data sources together. Draw conclusions, and make a recommendation. We can't compete with that so we're going to have to, at a certain pace, take it as an article of faith. That means we really need to understand what our data sources are, how the system is, how the system works.

Carolyn: It takes us down a very scary rabbit hole too. Where we've seen arrogance, laziness, I don't know how you want to label it.

Brains in the Foot Locker

Carolyn: But for whatever reasons, we just put our brains in the footlocker to quote one of my favorite quotes from your father. We want somebody else to do the thinking for us, we want the technology to do the thinking for us. Which is why we've heard many stories of people driving off the pier into the ocean because the GPS told them to. I think the idea of, can we get AI to the point of making all these decisions for us? It's a scary thought to me. It's the stuff that science fiction has been built on for the last 50 years.

Stan: No, you're exactly right. My wife and I were driving just last weekend. And my pet peeve about most of the GPS systems is they drill in and they show you a very small area. You just turn right or turn left. With my background, from the military, I want to see the big map, I want to see where I am. I want to see the route it's chosen, I want to do that the whole way. It automatically doesn't want to do that, it just wants to tell you what to do. They may be right, and they may not be right. That's the issue.

Carolyn: My dad, before we would go on any trip, he'd pull out the map and he'd make me look at it. I'm like, "I've got GPS. I don't need to do this." And you know what? Technology is awesome. It gets me where I want to go until it doesn't. Single sign-on is awesome until it doesn't work.

Stan McChrystal Talks About Inertia

Carolyn: I want to shift gears a little bit and talk about inertia. This is something that you brought up multiple times in the book. You said, in my Physics class at West Point, we learned that in the most basic terms. Inertia tells us that absent external forces, things will keep doing whatever they're doing.

That's true not only if what they're doing is brilliant and successful, but also if what they're up to is silly and destined for failure. To use the military axiom, never interrupt your enemy when they're making a mistake. But shouldn't we interrupt ourselves? Do you think that we are allowing tech to move us forward from momentum rather than deciding our own direction?

Stan: Potentially, it can. Of course, we proved it in history. We didn't need tech to do that either, we could make all those things. The problem we'd make is it can reinforce that. We build processes, we do things a certain way, and we do them to be more efficient with technology. To a certain degree, an operator of a machine or a computer does certain things, gets certain guidance or responses. We think we're not smart enough to say, no, we shouldn't do that right now.

Often, we give people instructions in their position. No, this is what you follow. This is the process. If it comes, if they say two plus two equals five, you use five and move on. The problem is really sociological, it's our leadership and the human side, but it is aided and abetted by technology. It's easier to go faster, further and get it way off track.

A Small Problem in a High-Speed System

Stan: There's a great story on the news recently about a high-frequency trading company. It was some years back when it happened, but one of their algorithms got off in the course of 40 minutes. They lost billions of dollars. It was because of a small problem in a very high-speed system, jumping the track and boom. That can happen in our economy. It can happen in almost anything.

Carolyn: Back to what you were talking about with just leveraging tech. One of the things you also say in the book, I assume this was early 2003 ish, when you were first kind of reorganizing the way JSOC operated. You said that the tech and the hardware, you were able to have communications, which is another one of the risk factors. And you were able to have communications all over the world, cross teams because of technology.

You said that technology was just as important as food and ammo. As I read the book, I kept thinking all of these things are interconnected. They rely on each other. Even as we're developing tech, it would be prudent. We should be applying this model to the development of the technology.

Stan: I'd throw out a couple of ideas. What would have happened a year and a half ago as COVID-19 sent us home, dispersed us, if we didn't have the level of technology that we enjoyed at that point? If let's say we didn't have the internet and whatnot, let's say we didn't have telephones. Literally society at the size and interconnected may have, would have stopped. Our Achilles' heel at that point became our ability to communicate because it built our confidence.

Stan McChrystal Communicates the First Step

Stan: If we weren't being communicated with all the time, most of us would have panicked in some way. Society would likely have done that as well. We learned something very interesting in JSOC, and I'm not sure we're the first people to learn it. It became really clear to me, that as we were dispersed and we had to implement a lot more technology than ever, to

When you have billions of data dependencies, and one goes amiss, how do you figure out where the issue is? Listen as Mark and Carolyn are joined by Andrey Zhuk of CTG Federal to discover how artificial intelligence is opening new doors for data security and recovery.

Episode Table of Contents

• [00:46] The Road to AIOps
• [09:07] Overarching Umbrella
• [17:25] Knowing the Unknown From Poorly Written Codes
• [27:55] Knowing the Unknown in the World of Tech

Episode Links and Resources

• Andre’s e-Book Software Intelligence for the Federal Government: The Road to AIOps
• Connect With Andrey Zhuk on LinkedIn

The Road to AIOps

Carolyn: I'm excited to introduce today's guest, , principal solutions architect at CTG and author of several eBooks. Today, we're going to talk about one of his latest eBooks, Software Intelligence for the Federal Government: The Road to AIOps. It focuses on cloud development migration in the federal government.

Carolyn: Let's start with the easy stuff. Tell us your story. What do you do? Where are you talking to us from? How did you get to where you are now?

Andrey: Sure. My background is actually electrical engineering. I used to design satellite systems and the networks that go along with them for the Department of Defense. From that, I went to the side of sales.

I was actually selling a lot of Palo Alto products and some optimization solutions. From there, I transitioned to the world of cloud. That's kind of where I got into the whole application performance management space. I was at a startup called Skyhigh Networks. They were one of the early cloud X security brokers. We were dealing with cloud apps and security cloud apps for government customers.

That's where I had the experience of dealing with the federal government workers, trying to modernize their applications. Then Skyhigh Networks got bought by McAfee. I was a solutions architect for cloud technologies with McAfee for a year or so. Then I moved on to CTG Federal to take on a principal architect position to help build their cybersecurity business with a little bit of the APM sprinkled in. We had a Dynatrace partner.

Knowing the Unknown About Satellite Stuff

Carolyn: Yes, we wish we could say that right upfront that we are partners. But before we get into it, I got to go back to the satellite stuff. How does that compare to what you're doing now, how long did you do that?

Andrey: Wow, probably six years, but ultimately everybody needs Facebook and satellite platforms get outdated like once every 10 years. So it's all about software.

Carolyn: Oh, so this is a lot faster for you then like, quicker pace.

Andrey: Yes.

Mark: The world that you're playing in now, Andrey, is it more high-level conceptual as opposed to the engineering work that you might have done, working on satellites?

Andrey: So it's interesting. The world is more about software now than it ever was before. Just to give you an example, YouTube platform. They're the one that when it got shut down by the Soviets, that is still in operation. So that airborne frames still exist, but the internals get modernized. The internals get modernized with new circuits, new equipment, but the software on those circuits gets changed quite frequently.

Mark: We truly are living in a software world, are we?

Andrey: Yes. Which makes application performance management and software intelligence that much more important.

Carolyn: It gets us right to one of the first things we want to talk to you about is the title of this eBook that you wrote, Software Intelligence for Federal Government: The Road to AIOps. I'd like you to unpack it a little bit for me. So first, what does it mean? What do you mean by software intelligence for the federal government? And then what do you mean by the road to AIOps?

A World Powered by ApplicationsAndrey: So we live in a world powered by applications. The applications are everywhere. They control how we work and collaborate, how we book travel, how we get our entertainment, how we get medical care, and taxes. But the thing is these applications have grown in scale. It used to be not so long ago, even as far back as 10 years ago, we would have just a couple of servers with some backend storage. Interact somewhere, maybe at clinics or something like that, our rec space, but now that's changing.

All of a sudden the applications are running on top of complex computing infrastructures that are dynamic, hybrid, and multi-cloud. And now these environments contain hundreds, if not thousands of technologies. Million lines of codes, and literally billions of dependencies. So traditionally we manage all these applications with a set of disparate point tools.

This is what we call, tooling. Each one required human involvement. This is also what we used to call application performance management, different tools reporting back to us. But as applications grew, the traditional data centers into cloud and subsequently public and hybrid clouds. So the old ways of doing things could no longer scale. The volume of data aggregated by all these monitoring and observability point tools quickly became so immense that no human can make sense of it.

There's also an issue of the unknown unknown, so you can't stage for something that you have no clue can even happen. So we needed a new approach. This is where we need to make all these sensors to be intelligent and make sense of all this data. And so this is where the software intelligence comes into play.Knowing the Unknown With Intelligence

Andrey: With software intelligence, we need to imbue these monitoring solutions with intelligence to make sense of all this stuff.

Carolyn: So have you seen that shift? You started the story where it was a lot more simple, was that the case when you were working on the satellites?

Andrey: Yes.

Carolyn: You've seen this shift through your career.

Andrey: Yes. So I'll give you an example. The old school application performance management circa 2010, it was primarily network-based. For example, I worked with a company called Riverbed. Back then, Riverbed was one of the hottest companies in Silicon Valley. They bought the startup from Cambridge, Massachusetts, which was called Cascade. All it did is it took packets from the network and made predictions about application performance based on flows and packet capture. That's it.

There are also a lot of competing solutions from the likes of NetScout, JDSU, and a couple of others. So it all just flows in the network. But now as things are changing going to the cloud, those solutions are no longer relevant. So you need something new.

Carolyn: So this isn't just a fundamental shift, I feel like this is a leap forward. I wouldn't say that it's equivalent to the invention of fire, but maybe.

Andrey: Maybe.

Carolyn: It's really huge to me.

Andrey: It's interesting. This is where the world of AIOps came to be. So artificial intelligence operations, I think it's a gardener term. But ultimately it's referred to a suite of products or software platforms that bring together, act as a force multiplier for correlating data across application performance management tools, IT infrastructure monitoring. This would be like your SolarWinds type stuff and network monitoring and the other diagnostics.

Overarching UmbrellaAt this point, AIOps is moving forward to now being this overarching umbrella which now encompasses application performance, infrastructure monitoring, AIOps. We used to think of it as opening tickets, for example, and maybe automating service, ticketing and response. But now we also include digital business analytics, which is especially important to the likes of say, Uber and the digital experience.

Mark: Yes, and user experience.

Andrey: Right. Especially if we think about the likes of Uber, this is where these kinds of platforms really come into play, just behemoths. How do you make sense of it all?

Carolyn: You mentioned something about our lives are run by,I mean, you said the word millions and billions, even at one point.

Andrey: Billions of codes and dependencies. Yes

Carolyn: Yes, and dependencies on an application. So give me some examples.

Mark: Some use cases.

Carolyn: Tell me a story about AIOps and I guess, how we're using it now, compared to what we were doing before?

Andrey: Sure.

Carolyn: You kind of already talked about that.

Andrey: I feel like we were talking more about the public sector or federal space. One of the best use cases that's easy to latch onto, even before undergoing through this transformation technology and re-architecting all your software. It's probably just a meantime to recover. So meantime to recover your MTTR, that refers to ability to recover a system back to an operational state. When you have billions of dependencies, something goes amiss, how do you figure out what the issue is?

Where AIOps Come Into Play

Andrey: So this is where an AIOps can come into play and be able to help you figure out, find that needle in the haystack. You go from weeks of teams trying to figure out what the issue is. Isolating components and testing to literally hours, because you have all this data and it's not just a sea of information. It's very precise nuggets of information telling you when something is amiss. So that's kind of the easiest to digest use case. The other one, which was very much a federal, I feel like federal specific is mission continuity between contracts.

So imagine you have a big contractor. Like Nordstrom come and run the show at one of the satellite facilities I worked with for the Department of Defense. Then that contract expires and maybe a Raytheon comes in to run the show. In an ideal world, that'd be documentation, kind of to bridge from one contractor to the next. But in the real world, documentation is usually lacking and key personnel have moved on. So how do you go about figuring out all the dependencies and process to process relationships in a satellite imaging system, for example, that one of the software factories is developing.

Actually, we talked about this pre-show. DoD is probably the biggest proponent of DevOps. They're on the forefront of re-architecting a lot of these legacy applications and making new ones, cloud-native. The big software factories, the ones like Platform One, Castle Run, Thunder Campbell Sky, Sonic, there are a bunch of them. So these guys support these numerous applications. How do you make sense of these applications when the contracts change?

Knowing the Unknown About the AIOps Tool

Andrey: This is where an AIOps tool is of great help. I can keep going. Is there any question?

Carolyn: How does the AIOps tool help that shift from contractor A to contractor B?

Andrey: The tool would help with mapping out dependencies, both horizontally between components at the same time and vertically.

Carolyn: Okay. It's keeping track of everything.

Andrey: Yes, between components of different types. And then you will get a real-time map of the entire application stack end-to-end from your customer's web browser. All the way to the application, down to the underlying containers and infrastructure, cloud resources, and so on. All of a sudden, that stack of hay is no longer a stack of hay, but a logical interconnection of resources for consumers.

Carolyn: Yes. See, I guess this is going to reveal how much of a developer I am not. I thought all that stuff had to be linked. Anyway, but you're telling me it's not, you need some kind of third entity to map it.

Andrey: In an ideal world, you're right. But I’ve been in situations where there was a hasty hire and the developer did not provide any inline documentation of any kind. No comments within the code. So making sense of that is the most impossible without a third-party tool mapping all those dependencies.

Mark: How do you think agencies are managing through these use cases like, we're talking about this digital transformation today?

Carolyn: Yes. What if they don't have that tool in place?

Andrey: Right now, it's very much just human elbow grease.

Carolyn: No. That's not even possible.

Science Fiction Movies

Andrey: I can't reveal a lot of things that I experienced firsthand, but you'd be surprised. So it's what you see in science fiction movies. These security operation centers like Starship Enterprise, those do exist, but they are far and few in between. The reality of things, especially with the smaller agencies. It's like three or four guys in a cube farm, looking at alerts and dashboards every couple of times a week. So without a tool like this, it's very difficult.

Actually, this brings up the next use case. Well, maybe the next two use cases. So one is helping augment the IT staff shortages. For example, I get hit up on LinkedIn several times a day. The federal government can compete with the commercial vendors like Dynatraces of the world. They have to do more with less. For example, we have a civilian agency that was piloting a Dynatrace tool for intelligent observability. They were using Dynatrace with Ansible to proactively detect and remediate memory leaks in a large enterprise application.

So Dynatrace would receive telemetry from the running processes. Then we'll use AI to determine if the telemetry received is indicative of a memory leak. So, this is not something that human can do. But if you're an AI operations platform, you have a broad statistical data set to make a decision upon. And so you say, "Okay, this is indicative of a memory leak at a very early stage. Then the tool integrates with Ansible to restart the call process without any human intervention required."

Carolyn: What do you mean restart the call process? Like, fix it?

Knowing the Unknown From Poorly Written Codes

Andrey: Yes. For example, a container that has some poorly written code on it is over utilizing the memory of the hypervisor it's running on. Sometimes the easiest way to fix these problems is just killing the process and bringing it back up, simple as that. But if you're a human and you have hundreds of containers running a copy of this microservice to support many threads, you don't know which one is problematic.

So you have an AIOps to tell you exactly, "Hey, this container has a memory leak. Let's kill it and bring it back up." And so you're also minimizing human error. That's another huge thing. This actually ties back to the next use case of how best automation works. So all these agencies usually rely on IT service management platforms like, ServiceNow, or they used to be BMC Remedy. It's now rebranded as BMC Helix, which you can run on prem.

There's also SolarWinds, which has a couple tools like that. So an AIOps tool would be able to detect and create and it will parse its content and apply AI to take an appropriate action. And maybe send an email for Joe, the engineer, to go do action X. So you may have an application that worked prior to a new code release.

Then you have a code upgrade and users are experiencing issues. Well, you can configure the solution. The AIOps tool or the software intelligence platform to automatically roll back code to the last working version. The software intelligence platform will take care of all the dependencies and do it correctly every time.The Room for Error

Imagine doing that just by us humans manually, the room for error is immense. Even if you've rolled back the code to the previous state, you probably forget one in 200 dependencies and you're in a world of hurt. But with a software intelligence platform, that can be all automated. In the federal government, that's kind of a key takeaway when you talk to federal customers.

I was in cyber and you were trying to sell them all these point products. They're like, "Look, I know it's great, but I don't have time for this. I have three guys doing all these things. We need something that provides automation." So automation, like you asked about before we started chatting, and what is the big takeaway from the federal government? I think automation is that journey to automate as much as possible.

Carolyn: Well, based on what you just said, this is not a founded fear. But there is a fear when you talk about AI and when you talk about automation. That means loss of jobs, because the robots are going to take over.

Andrey: Oh, yes.

Carolyn: But what I heard you say is, and I've heard this from other people too, like Willie Hicks, our federal CTO.

Andrey: It empowers workers and enables them to perform at their best. Yes. I mean, the stats are out there. There's simply not enough people to do this. Data keeps growing at exponential rates. Don't quote me on this, but in the last year, we've generated more data than in all of humanity, since World War II or something like that.Knowing the Unknown Real Issue

Mark: I wonder if you're seeing that fear of losing jobs is really not the issue. Because within a cloud-first mandated world, particularly in the federal space, it really allows organizations to take their smart people and re-allocate them. Have them do things that they really intended to do in the first place, as opposed to triage all the time.

Andrey: So you have the issue, with all these disciplines growing, where there's the broad knowledge and the deep knowledge. Unfortunately, a lot of the smart people are now spread thin having to be experts in multiple areas. But there's only so much CPU. Even a smart person can allocate all this. So I feel like a software intelligence platform tool can help go deep and take care of all those nuances. Even if you look at the field of where networking used to be, I'd say, with networks we would configure everything box by box.

Even to this day, most of the legacy networks and data centers in the federal government and even in commercial space are configured box by box. This is why we have CCAs making so much money back in the day. The CISCO certified is now the experts for those not familiar. But now there's a move to...