
Sign up to save your podcasts
Or


This week on the Acima Development Podcast, Mike starts with the fall of the Roman Empire. Rome guarded its borders with largely autonomous armies. When attacks came from too many fronts at once, the empire couldn't hold them all off. Mike compares that to where software security stands today, now that AI puts world-class vulnerability tools in reach of attackers well beyond nation-states. Joining him are Justin Ellis, a longtime security engineer who now works in OT, and Tim Chaffin from the Acima security team. Will Archer brings experience from several large corporations, and Vivian Moore offers the view of someone who just finished school.
The panel spends the first half on how fast the timeline has collapsed. Justin says attackers used to sit inside a network for 150 days or more before acting. Now an AI-assisted breach can go from first crack to ransomware in minutes. Tim shares numbers from Mandiant during a recent incident. Big telecoms take about 43 days to patch a critical vulnerability, but a proof of concept posted on a dark web forum gets exploited in about five. He also describes DEF CON capture-the-flag teams who handed their recon work to local models, and a mobile app his team decompiled with AI in about 10 minutes. Will notes that AI makes finding exploits easy, while writing a kernel patch remains beyond most developers. Vivian pushes back on Will's idea of cutting hostile countries off the internet. Cheap streaming sticks and TV boxes sell their owners' connections as residential proxies, so blocking by geography won't hold.
Mike then lays out what teams should do. His first step is constant observation, with automated watchers that block abnormal behavior without waking anyone up. His second is patching critical vulnerabilities within a business week, which Tim says only works with automated testing in the pipeline. Mike also wants teams to attack their own systems, the way Netflix's Chaos Monkey shut off infrastructure on purpose. Tim adds countermeasures for specific AI threats, such as FIDO2 passkeys against AI-written phishing. His team is also building scripted proof-of-concept tests so patched vulnerabilities can't quietly return in a new release. The group closes on people as the weakest link. Tim describes a phishing email that spoofed his dad's name, and Justin raises deepfake scams aimed at older parents. Mike ends with a call to act today, because the barbarians are already at the gates.
Transcript:
MIKE: Hello and welcome to another episode of the Acima Development Podcast. I'm Mike, and I am hosting again today. With me, I have Justin Ellis, Will Archer, Tim Chaffin, and Vivian Moore.
And I'm going to start with a topic that I usually avoid. I'm going to talk about the Roman Empire. I avoid it for a few reasons [laughs]. You know, there was all the, what was it, 2005 or so, they...there was all the social media stuff about adult men thinking about the Roman Empire multiple times a week. If you haven't seen that, it's a thing out there that have girlfriends ask their boyfriend, "How often do you think of the Roman Empire?" And they're like, "Oh, I don't know, a few times a week." And they're like, "What? Who think..." [laughter] There's this weird gender divide.
I don't follow social media much, but it was enough that, you know, it made a thing, like, made the news. It can be kinda trite, and also, I think, people sometimes portray it as like, "Oh, well, the Roman Empire fell. What a tragedy," and well, in some ways, yeah. But, I mean, it was this brutal, violent entity that destroyed the lives of many people for what cause? So, you know, you can make arguments for and against. My point is that it's a complex, morally and ethically ambiguous topic to talk about the fall of the Roman Empire.
So, I'm not going to dig into all of that. But I am going to say [laughs] that there was this entity that is commonly read about in history where there was a large, complex system that existed, you know, along the coast of the Mediterranean and Southern Europe for hundreds of years. And they were able to maintain that for a long time, right?
And eventually...and they managed the borders of the empire by having armies that were largely autonomous, right [chuckles]? Somebody would build their own army and go out and man the frontier, you know, walk along the frontier and expand it and take spoils as they chose [laughs]. And there eventually were so many different attacks coming from, particularly from the north, that they couldn't wage, you know, defenses against all of those fronts. And they had not kept their defenses up, and they'd been kind of rot from inside for a long time. And, eventually, those attackers made it all the way to Rome, and, you know, Rome was sacked. And there's still holdouts of the Roman Empire today. I mean, the city still exists. There is a Rome.
But after the Roman Empire broke up, you had a series of smaller city-states, right? And you had what we, you know, the medieval era, where you had easier-to-defend smaller units that could defend themselves more or less autonomously. Probably couldn't defend themselves against a big attack, but you moved from, like, a monolith to microservices kind of idea [laughter]. And, you know, there had to be that fundamental change.
I actually read an extensive article about this once. It said that the...you know, it's not definitive, right? But that the complexity of the Roman Empire was too much to sustain. They didn't have a system for managing that complexity, didn't manage their defenses, and eventually everything fell apart, and fell into something that could be managed because each of those units was less complex and could be managed by the systems they had at the time. Fascinating idea. Where am I going with this [laughter]?
TIM: Yeah, where are you going with this [laughs]?
MIKE: The barbarians are at the gates [chuckles]. We have a rise of AI now, and the tools that are available to exploit vulnerabilities are now potentially world-class, available to not just nation-states, but...you know, and maybe not quite the script kiddies, you know, somebody in the basement thinks, "Hey, what can I do here?" But getting close to that level. Some of the main things that are preventing it right now are cost. And some of the large frontier models are, you know, big companies publishing frontier models are holding back the strongest functionality for vulnerability detection because they know they can cause so much trouble.
There are a couple of projects out there, Project Glasswing from Anthropic and Project Daybreak from OpenAI, where they're partnering with large government and business entities to try to test their systems, test libraries, and get things hardened because the clock is ticking. In some ways, we've already reached the point where it's starting to tick up.
We're already seeing news items all over the place. If you look at your logs in your servers, you're probably seeing attacks. You've always seen the kinda script kiddie kind of attacks. Low-hanging fruit have been out there for a long time, but it's ramping up, and I'd like to talk today about what we can do about it.
I've got some thoughts, but with...we've got Justin Ellis here. He's a long-standing security engineer who has worked in Acima in the past, but is not currently. Tim comes from our security team. Will has worked for a variety of large corporations [laughs] and can speak from that perspective. And, you know, so I think we've got a good crew here. And Vivian's recently come out of school, so she can talk about [laughs] things from, you know, the perspective of somebody coming to the industry today, because it's very different than the industry looked five years ago. You know, we're in a different world.
I do have some thoughts on here about some tactics we should be doing, but I'd like to stop talking myself and get some initial thoughts from y'all.
JUSTIN: So, just one thing: I'd like to, like, share the current status, and then, you know, other people can share. It used to be where somebody would crack a network or crack an organization, and then they would sit in that organization, you know, hidden inside their network, trying to scout out other things that are vulnerable inside and slowly go through that and spider web, and then do something with it, whether it's a ransomware, or something like that. But it was, like, the average time that they were in that network was, like, measured in, like, 150 days, so it was measured in, like, 100-plus days.
Now, if your outer defenses get cracked, you can lose your inner devices almost instantaneously because of the AI assistance that is part of that attack. And they can go out and, you know, crack all of the vulnerable devices inside your network very, very rapidly and then decide to do something with it almost autonomously. I mean, that's the whole point to the AI, right?
And it's making it such that the, you know, crack to ransomware or, you know, compromise of your entire system is measured in minutes rather than, you know, days, where it used to be, you know, hundreds of days. And so, it's just a very different landscape for cybersecurity than it used to. So, I'll just leave that there to start off with.
TIM: Yeah, we had some...I recently was working with Mandiant on an incident, and some things that they said which were interesting to me was that the median time for groups like Verizon or big telecommunications groups to patch a zero-day or something that, like a CVE 10, 9, somewhere in there, something that's pretty critical, is about 43 days.
And with the advent of AI, the average time from exploit PoC publication on a darknet or a dark web community to demonstration of that exploit being exercised is about five days now. So, someone will find a zero-day, put it on their forum, and then people with enough GPUs to run Kimi 3 on their local rig at home will then use that PoC to then roll out a campaign against, you know, against whatever their intended target is. And they'll pop a target faster than that target will ever be able to roll out a rolling update or some kind of change to fortify against that. So, at this point, every security engineer right now is like, "Shit," you know [laughter]?
JUSTIN: Yeah. You get me [laughter].
TIM: Like, yeah, it's so hard because, like, I try to explain to my leaders and everyone like, "Hey, we need to get...we've been trying to upgrade the standard that we have as a business. When we identify something that's critical, we need to get it done within, like, a business week. That's the target we should be aiming for." Whereas the standard up to date has been, like, 30 days, somewhere in there. And that's really hard to do between application development and then, like, your server-side administrators that are, you know, protecting, you know, an Oracle whatever server on an old rack in the corner of a data center. You know, it's a lot to take in.
WILL: Well, I mean, I'm curious, right? So, like, a couple of iterations past in my career, like, I was doing a lot of sort of, like, validation work on, like, a OS level, like, really deep nitty-gritty, like, on the sort of, like, the surface layer where these exploits would pop up. Like, where you're exploiting, like, you know, the things that were written, and they were written, generally speaking, like, pretty well, but, like, real nitty-gritty threading, memory management, like, ways of breaking the...I mean, effectively you're breaking complicated programs, right, in predictable and exploitable ways.
And I can see very clearly how you could take AI, you know, frontier model tools to, you know, effectively write, like, a next-generation fuzzer, right, to interface with these APIs in, like, complicated ways to try and break them. Like, that is a pretty clear A-to-B sort of a thing. But creating a patch for those things is breathtakingly hard, well outside the capability of most models.
I mean, because this is stuff that I have some familiarity with and, like, you can't just...like, not everybody is just going to do that stuff. Writing a kernel patch is well beyond the capacity of anybody who...like, very few developers could do that. But now we're, I think, we're faced at a situation where, like, the average, let's say, North Korean [laughs], you know, North Korean exploit writer could create these exploits relatively efficiently, let's say.
You know, if you say like, "I've got a guy full-time, you know, I'm going to get him a bunch of GPUs. If he gets one home run a year, that's going to pay for this whole thing." Whereas, like, on the other side, it's just, like, you know, how many developers? Like, I don't know if you've got a person who could do that work at Acima, like, you know.
TIM: No one.
WILL: No one.
TIM: I don't know anyone at, like, a big shop that...we don't. No one has that capacity. Man, it's just...So, Chance, who's my peer, so I do the cloud security side of things, and Chance does the application security side of things. And we talked about this after going to DEF CON last summer. Because we were participating, planning, and doing, like, these capture-the-flags, and I went there with the intention to...my little hobby was to hack my hot tub. They have, like, an IoT, whatever they call them. They're, like, communities, but they're, like, little hacker huddle kinda things. And I like to go to the IoT community.
And I brought in my little hot tub controller, and we sniffed it and kinda reverse-engineered its API, and that was a lot of fun. But what has changed dramatically, which we noticed last summer, was these CTFs, these people who are competing, no one was pulling out, you know, the normal tools and doing the kind of the recon they usually do. They were spinning up local models on their machines or tapping into some external resource. And they were letting the models of their choice do all the busy work.
And it almost took the fun out of it. Like, I remember Chance was like, "Dude, this is really boring. Like, this isn't fun anymore [laughter]. Where's, like, the edge of competition?" You know, even, oh, what's that company that makes electric cars? Not Tesla, the one that made that truck.
JUSTIN: Rivian?
TIM: Yes, Rivian. They brought in a couple of their cars to DEF CON and said, "Go for it. Like, please try and hack into this thing." And it was really cool to see, you know, a dozen teams tearing apart a Rivian and pulling out the cards and everything, and they figure...but they were all doing this with augmented AI assistance with everything they did.
And after we came back from that, we talked to David, our SVP of cybersecurity. We're like, "It's changed now. We no longer can rely on our ability and our experience to effectively defend an organization. We have to be just as tied at the hip to AI as the attackers are. Otherwise, we will never repel a defense, you know, offer some kind of defense for our organization at a commensurate rate at which we are being attacked."
And this year at Black Hat, we're seeing that cybersecurity vendors are finally catching on to that. They're embedding agents into their tools. Tools like Orca and Wiz now have red team, blue team, green team agents. They have AI-assisted tools that are designed to do what would take us, you know, weeks to get done.
It's also changing the way CISA does things. Like, the annual pen test that everyone's required to do, it kinda doesn't mean anything anymore now. You do, like, a pen test once a year, and you're like, "Cool, you patched that. Did you know that there are now 10 more that you didn't even check in your estate of, you know, portfolio?"
This side of the industry is changing a lot. I think even just as much as the worry of a, you know, some senior engineer is given an AI plop of slop that he needs to run through a PR and see what was crap and what isn't crap, as much as that's changing how we form software, we're also seeing a massive change in how we keep it protected. It's a lot [laughs].
WILL: I mean, with the IoT thing, I don't know. I mean, like, decompilation, right? Decompilation used to be hard.
TIM: No, yeah [laughs].
WILL: And it's not. It's just, like, oh, I want...like, I'm sorry. This is...I, you know, I should move through this, like, moment quickly as, like, an old dev, but, like, there was so much...there was so much around, you know, a library versus your source code, and your source code was protected. Your source code was this precious thing, you know, that you wouldn't let out, and now that's gone, man. Like, an LLM is, you know, really good at...you know, and again, it's...we talked about this before, like, LLMs are good at things and bad at things and translation, right? Translation is something they're breathtakingly capable at, and especially translation of something generated by another machine, and is very predictable and easy to understand ways.
And so, God, it's so crazy. I mean, like, oh, I need a device driver for this video card, but they didn't write one for, you know, macOS this version. It's like, oh no, just pull it down. Write your own [laughter].
TIM: Being a little OPSEC-aware, we had an incident just like that three or four months ago where a mobile app that we were responsible for, we had reason to believe that it had compromising information in it, and we were like, "Okay, we have private models that we use in cybersecurity." We said, "Okay, go for it. I want you to rip this mobile app apart and find it." And within 5, 10 minutes, both Chance and I are, "Oh, there it is, like, we found it." And, like, holy...that would've taken me, like, a day. That would've taken me so long to decompile that and rip it out, but now it just blows right through it. It's staggering how uncomforting that is [laughs].
MIKE: Yes, and, you know, we're talking about our own context here, but this is everybody. This isn't Acima. This isn't name your company. Wherever you're working, it's you too. It's all of us. The world has changed, and it's going to change a lot more in the very near future because if you look at the curves of how these things are getting better, it's exponential. Even if it was linear, we can't keep up. We're already behind as an industry, right? We're already behind, and it's going to get worse quickly over the next, like...we're not talking about years. We're talking about months. Maybe even weeks. You know, we're talking about a short time horizon here. It does take a while for people to get familiar with the new tools, and there's costs and, you know, there is...it's harder, right?
But you can download some of these big Chinese models, run them on your local rig, like you say, get the right harness and the right target, you're good to go today. It's really scary. I'm not on the security team, right? I'm on the engineering side, and I think about this all the time because this is just central to what we're going to have to do to survive over the coming...it's hard to name your time horizon exactly, but I'll be surprised if, you know, we can go a year at status quo. And, again, when I say we, everybody.
JUSTIN: So, talking a little bit about the industry I'm in right now, which is the OT industry, it used to be that a lot of companies would roll their own network security. And they did all the right stuff, like the whitelists and the filtering and such. But now you almost have to have a subscription to a big network security firm in the OT space, like Dragos, or something like that, to be able to keep up with the attacks that are out there. And the comparison on your side would be Wiz Subscription, where Wiz is monitoring every single, you know, node in your network, in your cloud infrastructure, and it is alerting or blocking any abnormal behavior.
And you need that type of software that can understand normal behavior and that is authorized to block abnormal behavior the moment it's detected. So, it needs to be able to, like, make that decision without having to wake somebody up at night or anything like that. It just needs to do it, to be able to react instantly when it detects something that it suspects. And then you can come in and...in the morning, you can go in and see, okay, this stuff got blocked. It may have broken something, but at least we know, you know, we haven't been popped or, you know, something hasn't gone horizontal in the network.
MIKE: To your point before, if you have to wake somebody up, you're already too late.
JUSTIN: You're already too late. So, that's one of the things, like, I was chatting with a bunch of...I presented this last week to a bunch of people who are financing energy sites. And these are the guys that are deciding whether or not to give money to sites that are asking for money to build new sites. So, I presented to them and told them, "Okay, you guys aren't cybersecurity experts, but here are the questions that you should ask the site operators before you give them money to make sure that they have a plan to address, you know, this new landscape that we're in."
And the questions were basically: Do you have a network security software that is one of the big, you know, contenders in this space that is going to be turned on the moment you, you know, the moment you're building this? The next one is: Do you have normal traffic analysis and things like that, or monitoring that you can detect abnormal behavior? And then the last one is: Do you have a disaster recovery plan, and have you practiced it? Because even with all the preparation that you can do and all the defenses you can do, you need to have a plan for when you are popped.
TIM: The other one I see, too, is, just maybe a month ago, the U.S. started opening up cybersecurity contractors to act on behalf of the state. So, now you can technically work beside cybersecurity command and the NSA and others to do offensive attacks against other nation-states on behalf of the U.S. And, like, you have to have a...I think it was, like, a million-dollar bail for your own self before you can work with, you know, the government and stuff.
But it, like, the cybersecurity side of everything, you know, when you engage someone in your supply chain, you almost have to insist that they have a cybersecurity insurance policy for when they get popped so that your...if you get popped, how am I covered? You know, the intellectual property that you are privy to or storing in your systems, how am I going to be covered from that as well? And there's, like, this whole new industry now of cybersecurity tools, insurance, even some insurance companies that are working on behalf of the threat actors to parlay at ransomware conversations. It's insane, like, how much money is coming out of that whole industry.
WILL: I mean, I've long been of the mindset that, how do I put it? Like, IT, let's call it, as an industry, is long, long, long, long, long, long, one more long just in case, long overdue for any form of professional licensure or oversight or anything. I mean, right now, like, how much of our IT infrastructure is living in nations with, let's say, tenuous rule of law, money laundering, you know what I mean? Just sort of, like, security apparatuses. And I'm not saying anything about the quality of the developers, but I'm saying, like, where are they?
It's like, okay, we've got all these people...we have people who are developers, good developers doing great work in active war zones, like [chuckles], you know what I mean? And it's just like, that's...there's an exposure there that the industry, I think, has been, you know, for profit and motive reasons, very hesitant to really investigate at all because, like, you know, like, offshoring just wherever you can find a cheap keyboard. But this is a thing, guys. Like, just anybody...anybody could do it. That guy said he'd do it for $5 an hour cheaper, so let's roll [laughs].
TIM: That's a really good point. This came out today. The story was that as the U.S. was approaching against Iran in spring this year, and just before they were really getting to the throes of this war that we're in, a CIA analyst was presented a finding to leadership that there was a Chinese vessel that was transporting materials for nuclear weapons, and they were in the area of Iran. And that was, like, borderline World War III, you know, something that the military leader's like, "Yeah, we can't have that. We're going to do everything we can to shut that down."
And at the very last second, they did their due diligence and looked at, "Okay, where's your information? Where's your source?" Everything like that. And they dug into it. Like, the analyst had actually been using AI to analyze some imagery or something like that, and they were completely off. Like, they didn't...the article doesn't say what it was or anything like that. They kind of keep it a little capped in, but it adds light to this conversation that, yeah, you probably should be qualified before you make really big decisions off tools that you don't know a lot about. That kind of stuff could go wrong really bad [laughs].
MIKE: Well, you bring up an interesting one. Already, I was just reading about this in the last couple of days, at least twice, worldwide conflagration, like thermonuclear war destroying all of human civilization, has been avoided by one person who knew what they were doing saying, "Oh, you know what? This device is telling me that missiles are coming in, and I don't believe it."
TIM: Yeah. War games, right?
MIKE: What's that? [laughter] Yeah, it comes down to one person, I think, both times in the Soviet Union somewhere, who said, "You know what? It says that I'm supposed to launch the counterattack, but I don't trust this device because I know better." And if they had been, you know, trigger-happy, we wouldn't be talking here. That's scary [laughs].
TIM: Especially when...
MIKE: There's risks out there, real risks.
TIM: Yeah, especially when leaders talk about moving decision-making processes to automation, and they feel that pressure to do it because the only way they can react quickly enough is to let automation take over for it. But man, that's...yeah, like you said, if you don't have someone that has experience, I don't think that's a good idea [laughter].
WILL: I mean, like, no, I disagree with that. I disagree with that entirely. I mean, I wholesale reject that. It's a question of like, they don't want to spend the money. If we were operating, let's say, a diamond exchange, right, that was open for business 24/7, right? 24 hours a day, seven days a week, like our web servers are, we would have an armed guard on premises, 24 hours a day, 7 days a week, you know what I mean? Like, do you have a person who is, you know, monitoring your servers 24 hours a day, 7 days a week? No, you don't. You go to bed because you don't want to hire somebody to be your night watchman, right? Like, somebody with that skill set to be your night watchman. You don't want to hire that guy.
And we've gotten accustomed to a level of laxity that I think is maybe unrealistic and, like, slightly entitled, you know? It's like, no, if we were running a bank, you know what I mean, you'd have a security guard while the bank was open, right? It's like, "No, no, no. We've got one guy. We've got a guy. We have security guard. We have guard [laughter]. Guard works, but we can't...we have to let him sleep, so I guess we just won't have guard then."
MIKE: You know, that's maybe a good pivot. This has come up several times in the conversation already. So, we've talked about the risk we're in. I think we've clearly established we're in a different time, and the risk is elevated, and it's just going to get worse. So, what do we do?
Going back to, you know, pre-modern times, you got your Roman Empire in ye olden days of yore. You had somebody sitting on the hill, right? Or on the tower. You have that watch person, right? You've got the watcher. In fact, if you've got a large empire, you probably have watchers. So, you've got your Great Wall in China, right? And you've got watchtowers, you know, within sight distance of each other, with somebody staffing each of them because you need to be vigilant. I thought of this when Justin was talking about, you know, your automated system to catch when things happen. If you're not watching, you don't know. And once the invaders are inside the walls, they're already through your defenses [chuckles]. You're in a bad way.
So, it seems like, and I'm going to put this out there...I want feedback. Step one: observation. And it doesn't matter how good your walls are, you better be watching. You know, you talk about that bank, Will. Yeah, you've got a bank, and it's probably really well-defended, and it's got a really nice vault. And it's got lots of, you know, it's going to be difficult to guess the combination on that vault. Somebody walking in is not going to find it. But that guard's there anyway. That guard is there anyway, because they don't trust that the walls are sufficient. So, I'm going to suggest that the watchers are critical in a way maybe they never have been before. If they weren't before, they sure are now. Thoughts?
WILL: Well, I mean, I'll tell you another thing why I think we need to do away with, I think...and this needs to be done away with fairly permanently. And it's already happened, but it needs to...we need to finish the job. I feel like a lot of our really critical infrastructure layers, right, OS layer, device driver layer, et cetera, are operating on 'trust me, bro' security, right? And I don't think...I mean, we've all seen, like, the enterprise runs on Linux, really, right? But, like, you know, Windows, macOS, stuff like that, like, they're massive security surface layers that are operating on, like, 'trust me, bro' security. "You see this black turtleneck? Black turtleneck would never compromise your data." "But, well, can I audit that? Can I verify that?" "Trust me, bro. We're good."
And we know for a fact that many of the browser exploits are not getting fixed on time, you know? Like, we're finding them, and they're just sitting. And I don't think we could afford to operate like that anymore, especially since...you know what I mean? We have these, like, service layers. You can take, like, a Windows surface layer. I haven't been on Windows for a very long time, but I know historically the real serious attack vector for, like, a Windows workstation was through the device drivers, because you'd have some company wrote a third-party device driver that was operating at the kernel level. And they didn't have the resources of Microsoft to keep up with things, but it's still running as root.
And so, you crack that device driver from, I don't know, the headphone [laughter], you know what I mean, like, your headphone amp that you got from the cheapest supplier on Amazon, and you're off to the races. Obviously, that's, like, I think, a little bit of a contrived example. But we'll definitely go for the lowest bidder for a keyboard or something, and we have to get a little bit more serious about this stuff than we have been.
MIKE: Just to do some data backing that up, Anthropic came out with their Mythos model a couple of months ago. And when they published the initial stats, they had done some vulnerability testing, and they were able to find significant vulnerabilities in every major browser and every major operating system, including things like [inaudible 32:25] security operating system, the BSD. It was one of the BSD distributions. Like, things that have been hardened for decades, they found vulnerabilities in. It's out there. Those vulnerabilities are there.
TIM: Just to add on a little bit to that, that was what, four or five months ago now? The open-source models that are coming out, there was a recent analysis done, I think, last week, saying that it used to be that the open-source models were six months behind the frontier models, but now they are four and a half months behind the frontier models.
So, it's like, you know, the stuff that Mythos was doing back in the day, back in the olden day of six months ago [laughter], you can have that same capability running on your pretty well-specced-out MacBook Pro, and you don't even have to be doing it maliciously. You could be saying something to the effect of, "Hey, my model, I give you control over my laptop to go plan me a really cool vacation and make all the things," and your laptop decides to go take over a power site because it wants you to have a good vacation [laughter]. And that sort of thing is going to be entirely possible. I say that kind of joking now, but think of what happens five years from now.
MIKE: I read an anecdote just in the last week or two about somebody who was using one of those agents that went and hacked the travel website because the tickets weren't available so that it could get them the tickets [laughs].
TIM: So, it could get them the tickets.
MIKE: Yes.
JUSTin: So, it's going to be interesting because, basically, I think, every single business out there, every single mom-and-pop is going to have to have a subscription to Wiz that is monitoring their network and that has the ability to shut penetration attempts down. And it's just going to be a business expense that everybody's going to have to pay, and Wiz is going to make gajillions of dollars of money. But that is the thing, because they aren't going to be able to keep up on the internal defenses, all their devices that are running inside, there's no way they can keep all of those individual devices up to date, like IoT stuff, everything.
So, you really have to have a really strong outer defense, and that is not even going to be sufficient long-term because, you know, what if that one gets popped? But at the same time, that's almost, like, going to be a for-sure expense that you have to have, that you have to spend.
WILL: Like, I almost wonder, like, just the scope of, like, the vulnerability and, like, the difference in sort of regulatory regimes across countries, whether, like, the internet is just not going to be for everyone anymore, you know? Where it's just like, "Hey, yeah, North Korea, we're not going to talk with you anymore. You guys don't get the internet anymore." And, obviously, that's a little bit more complicated than just saying, like, "We're going to kick the plug out," right? Because, like, the Chinese will still...The Chinese and the Russians will still talk to North Korea. And if we still talk with the Chinese and the Russians, then we're talking to North Korea by proxy. But, like, I wonder, to a large degree, whether with the political, legal, regulatory framework being what it is, whether, like, the internet as a whole can truly be worldwide anymore.
TIM: Maybe.
VIVIAN: I'm going to push back on that actually. There is a huge market nowadays for essentially residential proxies. Every time you buy a projector, or a Roku Stick, or an Amazon Fire TV Stick, or whatever, part of what they are providing is they sell your internet as a proxy to other people so that they can use what appears to be a trusted IP address to do any form of malicious activity or very legitimate activity. But regardless, that is a huge component of the sale of TVs nowadays. Not only are they monitoring your content, but they're actively using your network in order to disguise your traffic and make it appear legitimate.
So, as much as I would like to say that we could geo-block or re-architect things in a way that makes it so that individual organizations, like, manage more of the infrastructure of the internet to help provide a layer of, like, security and authentication there, there's already so much infrastructure out there that relies on these systems of residential proxies that kind of eliminate that possibility unless the entire internet system is re-architected from kind of a fundamental level.
WILL: Yes. Yes, that's right.
TIM: My favorite thing is when we get a cybersecurity issue, and the first thing I'm told to do is, "Well, just block the IP." It's like, that doesn't do anything, man. You need to, like, not think about security that way [laughter]. "Just block the IP. Like, you guys know they're, like, vended botnets where, like Vivian just said, like, people hijack IoT devices everywhere.
So, Mike, your question was: how do we mitigate or try to defend it? First thing is you just have to change the paradigm of your organization, engineers, and everyone to stop looking at AI as an adversary. And you just have to...if you can't beat them, join them kind of thing. You have to have that situation, that acceptance, like, AI is not going anywhere. We can bomb as many data centers as we possibly can. Engineers will just find a way to make AI run on less hardware, you know? We've seen that happen before with DeepSeek and frontier models, like, keep getting distilled so they can run on anyone's iPhone or whatever they want, you know? It's just going to keep getting there.
So, we've been tracking a couple of threat categories, and a threat category that originates from AI and how you're supposed to defend against it. So, the first one, AI-generated phishing, right? Phish attacks. That happens so much. And it's not that hard to prevent it; it's just a pain in the ass for everyone who's on the other side.
You have to require MFA. You have to require FIDO2. You have to require passkeys. No matter how good your phishing attack is, they're not going to break a FIDO2 passkey, yet. Now, we put a yet there because there is a threat that we're tracking where the lines of quantum cryptography and quantum computing converge with our LLMs, and we're a little worried about that. That's when I start to get a little freaked out because now an LLM has the compute capacity to short-circuit a model or, you know, an RSA cryptographic signature instantly.
Password attacks, move to passwordless. Conditional access, MFA, that's how you keep them from doing that. Malware generation that, you know, script kiddies give a Kimi K3 task to write a new malware thing. Well, you have to do application allowlisting. You have to do behavior-based detection. You know, like John was saying, put, you know, some kind of UBA mechanism on devices to make sure you're not getting attacked.
Vulnerability discovery and detection, we've lived in this world for three or four years now, where everything has to shift left. You have to have the SAST, the SCA, the IaC; all of those tests have to come before someone even does a pull request. We want those tests to happen before it even leaves a developer's machine when code is committed.
Deepfakes: Apple just pushed this out, this week, Apple Reference Images. Google was the one that actually came up with the technology. You see something that looks questionable, you're like, "Cool, show me your reference image. Show me the signed cryptographic, you know, signature to this image that shows it wasn't modified." That's already out there.
There are a lot of things, strategies that are out there in the industry to combat the risks that AI presents. The problem is a lot of businesses are not willing to swallow the heartburn that's going to cause on engineers in the day-to-day. And sometimes people, organizations overreact to some of these threats, and they might make things harder than they need to be. You know, like rotating a password every four minutes when really you should just put MFA on there, you know, that's the kind of mistakes we see a lot of organizations make.
WILL: Yeah, but how is this going to affect our scheduled day offs [laughter]?
JUSTIN: We aren't even going to talk about, like, how AI is going to replace all of our jobs [laughter].
WILL: You know, I'm very much of the mindset that, you know, if you can't beat them, join them, and that's just fine. But, like, having been on the surviving end of several, let's see, let's say, like, productivity revolutions in software development, you know, of which this is one, this is certainly one. I love it. Productivity revolution, like, great. Great. Super great. I love it. But, man, my ticket queue hasn't gotten any easier. Has it [laughter]?
And, like, I wouldn't say that, like, I'm not the best around. I'm not the best AI cat around. I've got a couple of friends, you know, who are real hotshots on it, and that's...God bless them. But I'm by no means a Luddite, and yeah. Yeah, I'm going to be working hard right up until the final bell on a Friday. Count on it [laughter].
MIKE: So, we talked about...So, joining them. So, you got to have your observation layer, your watchers, right, watching this. And nobody can afford to have somebody on every watchtower, right? So, you have to automate this. You have to have the AI engaged to be doing something with the AI.
You bring up something else. I think it was mentioned earlier that your average big org takes a month and a half to patch a serious vulnerability. Your vulnerability policy that you had from last year isn't going to work anymore. So, there's my second suggestion. You got to start caring about those vulnerabilities and patching them now. You thought that you could wait till next month when you had a little more time; I don't think that's true anymore.
TIM: So, corollary, I mean, downstream effect from that is that you have to have a way to test that your patches are, you know, good. And if you have a QA department that's going to take you a couple of weeks to make it through that, that's not going to be good. So, you got to automate all your tests and, you know, get your pipeline up and running to where you can do that very rapidly. Now, a lot of people already have this, but a lot of people don't.
WILL: I feel like the automation of QA, I've always had a very soft spot in my heart for QA just because I've seen so many difficult things to diagnose, and troubleshoot, and validate, and, like, a lot of every organization has a lot of secret sauce. And I feel like...I do feel like the pendulum, it's about time for the pendulum to start swinging back the other way with QA because those guys have been...It's been a bloodbath in QA for the past several years, and, I think, we cut too deep industry-wide. I'll say it, 10 toes down [laughter].
JUSTIN: So, I almost think that, I mean, just as an aside, kind of a response to that, I think, QA engineers who were click QA engineers who didn't really do that much beyond clicking and such, I think that position is gone. But I think QA, true QA engineers who understand end-to-end and who are automating things and who are dedicated to their craft, I think, that position is more needed than ever.
WILL: Exactly. Exactly. And when I say the QA engineer, like, what I'm talking about is somebody who knows how everything works end to end, flow to flow, like, all of the whole process of this thing going back and forth, and all the things that have exploded, and all the things that are so easy to fall through the cracks between engineering teams, where it's just like, "Oh, well, I work on the checkout cart, so that's my problem." "And I work on the front end; that's my problem." "And I work on the lease, you know, information underwriting data collection part, so that's my problem."
I've known so many people in QA who were the final UX authority from, like, this entire flow from soup to nuts, and I don't see a replacement for that, and I don't see that value becoming less. But you're right; I wholly agree that if you are just a button monkey, you're probably going to need to tool up, or you're going to need to retrain one way or the other [chuckles].
TIM: We've looked at this idea of if I, the cybersecurity team, find a vulnerability in your software, I'm going to verify it, confirm it, give you as much information as I can to an engineer or a leader so that they know, "Yep, this is a vulnerability. This is where it needs to be fixed." But then on our side, we're also going to write kind of like what you consider a unit test in software engineering, but it's a cybersecurity test. It's a scripted PoC designed to test that your vulnerability does exist, and then we would automate that.
There are a lot of tools out there, like, Nuclei and other frameworks where I can write a security exploit that's to the PoC level only. And what we want to do is, every time you do a new version out to prod, that server is going to go through and do a regression test to make sure that your new feature didn't disable some kind of security vulnerability that we patched last year. And that idea of QA from a cybersecurity perspective is kind of a little bit new in that we're designing an automated hacker to make sure the things that you patched don't come back up, or that you actually did fix the vulnerability. It's kind of a fun idea, and we're still trying to brainstorm through it, but yeah.
WILL: And so, I'm curious, like, what are you guys seeing in the wild, to the degree that you can, you know, disclose? Obviously, that's not, like...Obviously, that's a privilege to a degree, but, like, just on a broad, qualitative, you know, sense, what are you seeing, like, dealing with stuff out in the wild?
TIM: The OPSEC side of that question...[laughs] I will say a lot of the attacks that I am seeing are not terribly sophisticated. What's happening is the volume at which they're coming at you as an organization is overwhelming. So, I'm seeing a lot of vended botnets because, like Vivian said, you know, Roku's...Oh man, there was even a free TV box (I'll have to pull it up. It was on a Darknet Diary podcast episode.) that gave people free TV, but it was a harvester in everyone's homes for a botnet to use, and man, they had such a huge botnet.
So, we're seeing a lot of volume where threat actors are just doing everything they can to get past the credential layer of your application. We're not seeing a lot of RCEs. We're not seeing a lot of memory escapes. We're not seeing a lot of the higher computer science PhD-level of exploits. We're seeing a lot of the threat actors relying on engineers not to do the basics. Like, if you have a basic username and password, you will be attacked. If you don't have MFA or something like that enabled on your login profile, you will be attacked. So, they're just looking for the low-hanging fruit that you have not done the minimum to step up to.
WILL: I'll say...I think I can be vague enough about this. Something that I have seen, and I know for a fact this is AI-enabled, I have heard about a large telecom company who was getting attempts made on them from a reverse engineer of their mobile app, whereby they are trying to exploit...We're talking about the decompilation, right? And they're reverse engineering their mobile app to go out and run exploits on non-published APIs. So, there's an API that's still in the oven. It's released behind a feature flag, right? Because, like, you can't roll a mobile app the same way you can roll a web app, right? You've got to, like, put it out there, and then when the back end is ready or, like, your release channel [inaudible 50:16], you turn it on.
Well, I have heard of people decompiling the mobile app, right, and just, like, filtering through and just seeing what they can see, taking this thing apart and, like, finding non-published APIs and trying to exploit those. Now, the people that I'm talking about are good boys and girls, and they, you know, treat the mobile application as wholly untrustworthy [laughs].
TIM: Zero trust.
WILL: Code that was written by us kind of, and is running on somebody else's machine and doing God knows what, you know, because a mobile app is wholly untrustworthy. But I believe that is a direct line to an AI-enabled attack, just because applications are under active development, and they get released very quickly. And the chore of decompiling the apps is relatively burdensome, you know, to just go and filter through it and see if there's anything you can see. So, it wasn't a problem, but it was just like, "Oh, wow, that's a funny story that this person told me [laughter]."
MIKE: Well, again, I think we can be...We have seen requests coming in from an emulator as the agent [laughs] the mobile app is running on. Wait, what? You should not be running this in an emulator. That is not right. When we did a new release of the mobile app, you know, there's clearly something untoward going on there. There is no reason one of our customers, a week after we released the mobile app, should be running on an emulator [laughs] and not on an actual device. You know --
TIM: That's just poor...I'd call that poor cyber-attacking.
MIKE: I agree [laughter].
TIM: They control those headers. They should not have...
MIKE: Exactly.
TIM: They shouldn't have had it be, like, an emulator header.
MIKE: That's what I thought [laughter].
WILL: Like, just do me the courtesy [laughter], you know what I mean? Like, at least put it through Opus for crying out loud. That's some haiku level shit [laughter].
MIKE: So, one more thing I'd like to propose is that we should be proactively attacking our own things.
TIM: Yes.
MIKE: You know, we've talked about the defenses, the watchers, but it doesn't matter how good you think your wall is; if you haven't tried to knock it down, you really don't know. So, you know, Netflix famously built their Chaos Monkey.
TIM: Monkey, yeah.
MIKE: So, they would just turn off important parts of their infrastructure at random. And that enabled them to have the infrastructure that it could handle things that, you know, handle when things went down by accident. They did it on purpose. If we are not taking a similar approach of actively attacking our code, our infrastructure, everything about our stuff, I don't think that we are going to be prepared for the coming onslaught. Thoughts?
TIM: That is a requirement today. Just getting really nice tools that the bad guys are going to have and using those to attack your own stuff, that's a big deal.
I love this podcast called Darknet Diaries. The first of this month, they released an episode called The Courthouse, where it was all...these were physical pen testers. Their mission was to break into a courthouse and to see how far they can get into the courthouse, what records they can get to. And this is a small town, and the small-town sheriff, they did end up tripping an alarm, so, like, okay, you know, they walk out, and they have a signed article from the mayor's office, whatever, saying, "Hey, we were authorized to do this penetration test."
And I guess the sheriff just had a bone to pick with anyone who would try to get into one of his federal buildings, and he sent them to jail no matter what. He just had this prejudice against, you know, "We would never do a penetration test. That's...You don't do that. Why would you do that?" And they did, like, these interviews of people in the community, like, "That makes no sense. Why would you do that?"
And the mentality is absent, even in organizations where we think, you know, a routine scan or an audit is sufficient. We actually need to be way more involved and way more aggressive in how we test our systems. We need to get really ugly and dirty because the other guys have no qualms of crossing those boundaries [laughter].
JUSTIN: Yeah. And just leave your pride at the door because you are going to be, you know, hit every which way, and they're going to find some really ugly stuff; at least, hopefully, you find it yourself. But...
TIM: Yeah. I had a threat actor hit me personally. I've got some public infrastructure for my own stuff that I've got out there.
This guy did some open-source intelligence on me, and he figured out my dad's name. And so, he started sending me phishing emails where the...when you send an email, you can display the name of the sender email, and it can be different than what the actual originator email is. And so, it showed my dad's name as the sender, and then it made a reference to some vacation pictures.
First of all, we don't go on vacations as a family. That was, like, my first yellow flag. I was like, what the...[laughter] We are homebody librarian people. We don't share photos. We don't care [laughter]. Like, it just threw me so quickly [laughter]. I was like, "What is this?" But it was coming from my dad, and I was like, "Wait." So, then I pull the source; I look at it. I was like, "This is one of the best phishing attempts I've ever seen." It was a curated phishing attempt to me based off the information around my OSINT out there, and I was like, "Damn, dude, you earned it. That's really good." Like [laughter], that is incredibly good.
And I was like, well, I guarantee a business would fall for this. Like, you see someone from a real email that you trust, from a name that you trust with a personal reference, that could be just as simple as, "Hey, I've got a meeting at this time. Here's the Webex link." Cool. It pops up right there, boom. You own them.
JUSTIN: You bring up a really good point that a lot of us have parents who are older, and, in fact, I need to go chat with my dad later on today because his phone got owned, and I got to walk him through a factory reset, but that is a whole different subject for another day. But, like, older folks who can get deepfaked, you know, and, all of a sudden, all of their retirement money is gone, and that is just a scary thing. And they're being deepfaked from a photo of their kid who, you know, a professional photo, or something like that, and, you know, they have some details from a vacation or something like that, and it's gone.
WILL: Man, all I'll say is, like, we as an industry run on the backs of lonely, overworked single men a very long way from home, and, I think, we should be more...I think we should be aware of more classical espionage honeypot scams because, "Oh, sweetie, I'm your long-distance Canadian girlfriend, and man, I've been trying to get this program working on my computer. Honey, you're so smart. Can you fix it for me [laughter]?"
TIM: Yeah. Romance scams and, what's it, pig fishing or...
JUSTIN: Spear phishing?
WILL: Spear phishing. Yeah. Yeah. I don't know. I've been working with these guys for a long time. Some of them are a little pent up [laughter] making bad decisions.
MIKE: So, final point there is that humans are often the weak link. It doesn't matter how good your external security is, just takes one of said people opening one email in desperation.
WILL: I think the industry has been going through a contraction and running a skeleton crew extremely hot, and it's been this way for several years. This has been a rough business cycle for engineering, and we've got all of this sort of...We could see the wave gathering in front of us. And, you know, the cynic in me, having survived many of these business cycles in my lifetime, is some people are going to get swamped before we start making changes because, like, they are systemic.
There are systemic aspects to the industry that this is going to...The tide is coming out, and there have been a lot of people who have been cutting a lot of corners for a long time. And me saying it and everybody nodding their heads and saying like, "Yep, you're right," is not going to make any kind of difference. Like, people are going to have to get got. And, I don't know, I guess I did my job, and I said, "Hey, guys, I don't know about this [laughter]."
MIKE: We could probably go on for hours here. I think that we've covered some key tactics you can take, but if we don't do it now, it's going to be too late.
WILL: Yeah, that's right. We have watchmen. We have watchmen. But he sleeps [laughter]. So, no watchmen while he sleep. What could we do? Only have watchmen, must sleep [laughter].
MIKE: I know that we are being proactive, and, you know, we've recently staffed up, and I'm optimistic that we're going to be making some right choices. I also am fully confident that in the reality of software, there are going to be vulnerabilities somewhere. There will be a library somewhere that [chuckles], even if our software is perfect, there will be a library somewhere. There'll be a vulnerability found, and as we said before, they're minutes once that's exploited.
So, we have to, as ourselves at Upbound, and as an industry, change our posture here. So, here's the voice of warning [laughter]. Act now; act today. The barbarians are at the gates. Sometimes they're already in the gates, and there's no time left to wait.
TIM: And on that happy note...[laughter]
MIKE: Exactly. And I'm leaving it bleak here. This is a call to action. I'll leave [chuckles] it there. Till next time on the Acima Development Podcast.
This time on the Acima Development Podcast, Mike opens with Nitinol, the nickel-titanium alloy developed at the Naval Ordnance Laboratory that can be bent far out of shape and spring back more or less indefinitely. NASA has prototyped woven Nitinol mesh wheels for future rovers, which would spare them the battered look of Curiosity's wheels after a decade on Martian rock. Mike uses the alloy as the frame for the episode's real subject, which is pressure. Pressure at work is not optional, and in the right amount it is what keeps people from getting bored, so the question is how to bend under it and come back to shape.
Will Archer argues that anything worth shipping costs you something, and he quotes bodybuilder Ronnie Coleman on the gap between wanting to get big and wanting to lift heavy weights. He has stopped punishing himself for needing a hard push at the end of a project, and he describes recording studio days that come in 12-hour blocks as the same bargain in a different industry. Mike adds that the author behind "follow your passion" meant something closer to finding work you care enough about to survive the misery of doing it. Tim Chaffin brings in what cybersecurity calls ethical pressure, the practice of rehearsing failure through penetration tests and chaos monkey runs so that a real incident calls for calm execution instead of improvisation. He compares it to the expansion joints in a Florida causeway, which are built to split so the rest of the road survives. That connects to the "Antifragile" idea Mike read about, where a muscle under load gets stronger and an unused one withers away, and to a study finding that people in hunter-gatherer societies burn about the same daily calories as an office worker while spending them very differently.
The back half is about the other end of the bell curve. Tim points out that chronic urgency produces burnout, turnover, and lost tribal knowledge, and that on-call rotations need real recovery built in. Will describes the drop that follows a launch, the sprint or two where nobody is any good, and the doom loop that follows when leaders crunch a team on promises the capacity was never there to keep. Dave Brady brings up Dan Pink's "Drive" and its three motivators of autonomy, purpose, and mastery, then contrasts a boss whose reward for a 90-hour push was "you get to keep your job" with an employer who handed out comp days. He also tells the story of an engineer who lost the company a lot of money and was sent back to his desk with the line that the company had just paid $25 million to train him. Mike closes with Balaji's habit of handing a promising person a hard project, checking in weekly without micromanaging, and thanking them specifically when it lands. His last image is riding in front on long bike rides with his son, setting a pace the younger rider can hold instead of letting him burn hot and crash.
Transcript:
MIKE: Hello and welcome to another episode of the Acima Development Podcast. I'm Mike, and I'm hosting again today. With me today, I've got Thomas Wilcox, Will Archer, Vivian Moore, and Tim...It is Chaffin, right? You know, I've heard people say Chaffin all the time for your name. It's Chaffin, right?
TIM: It's Chaffin, but, you know, elementary school just damages us all, and I don't really know [laughter] what it is, so...
MIKE: [laughs] Yeah. As usual, I'm going to start with a lead-in. I'm going to go to aerospace. There is a cool alloy I learned about a few years ago. I've looked it up in advance of the call to make sure I knew what I was talking about. They call it Nitinol, which sounds like it would be some sort of alcohol, but it's not. They just stuck some words together for nickel, titanium, and NOL comes from the Naval Ordnance Laboratory, where they developed it. So, it sounds like it's an alcohol, but it's not. It's just nickel and titanium in equal parts. That's really it. And they just did some very slight variation, like a few percent between equal parts, but basically, it's equal parts nickel and titanium.
And it's got a few interesting features, but the one that I want to talk about is that it can be deformed significantly and spring back to its initial shape pretty much indefinitely. You know, normally, you know, you think, like, steel. Steel's pretty good at having some elasticity, right? But, you know, you bend some steel bar, you know, steel bar enough times, it's probably going to lose something. To some degree, it'll do well. You know, if you have some springs, you know, they'll spring back a lot of times. This nickel-titanium alloy is even better. It's just basically forever, you know, it'll always spring back, and it's relatively light.
And they've made wire out of it and woven the wire into a mesh, and they're using that woven mesh. It looks kind of like chain mail-like, so it's not...It kind of retains its shape. And they plan to use it for wheels, and they've got prototypes of this for future rovers for NASA. So, instead of having a solid metal wheel like they have...And have you ever seen, like, the selfies that the Curiosity Rover takes on Mars? Those wheels do not look good. They've been [laughs] riding around on rocks for, like, over a decade now, and they do not look good [laughs], all kinds of big holes in them. Because they kind of fly the...you know, send the thing up on a rocket, so they make them super light.
These nickel-titanium alloy wheels are basically just a...they're even lighter because there's...it's not like foil; it's just this wire mesh, right? Just this really thin wire. But they can put a vehicle on it, and it'll behave like a rubber tire because it'll spring back as many times as you want. Super light. So, it takes the pressure from the vehicle above, and as it goes, it just springs back.
There was a Kickstarter a couple of years back where they were selling bicycle tires made out of this stuff, and I didn't hear about it till after. " Oh, that would be cool [laughs]." Because if you put that on your bike, you'll never get a flat again, ever, and it lasts for, like, decades. So, it's the last tires you ever have to buy. Just, it's a really cool idea. I guess it's an expensive alloy to make; that's why we haven't seen it widespread. But they use it...I guess there's quite a few usages of it in aerospace and other places where they really need something that's resilient. So, cool stuff. Worth looking up if you haven't looked it up before, and just, like, watching videos of it. It's pretty cool.
It has some other cool properties, too, that are a little off-topic, but I guess you can put it to a lower temperature, bend it, and it'll hold that shape. And then you raise it up to the original temperature, it'll spring back to its original shape. Interesting stuff.
And there is a tie-in here because today we want to talk about pressure and resilience under pressure. We often talk about pressure as a bad thing, but pressure is reality [laughs]. We're all going to be under pressure of some sort. We are paid to go do a job. You could say, "Oh, it just doesn't matter," and just wander off, and that's not going to work very well. There are demands. There are demands in life, and they're not going to go away.
And, in many cases, pressure can even be motivating. Like, you know, if you don't have any demands, you can get bored. People actively seek out more interesting and demanding work and activities. People who are retired still do something interesting because they want to have a little bit of pressure. We don't do well without it. Then again, people break, right? Burnout is a real thing, and it's something we got to deal with.
So, there's the topic for today. How do we get ourselves in a position so we're like that nickel-titanium alloy, you know? Pressure...under pressure, we bend, we spring back; we're good, and keep going. How do we get there?
WILL: We all know about the bad parts of pressure. Those are pretty well-established. We've sort of, like, gotten over them, like, many times. Foundationally, I keep on going back to the two things that I think most people want, right? I mean, people want excellence, right? I think that's a foundational thing that, like, most human beings, and certainly most engineers, are driven towards excellence. And people are also driven towards growth. And both of those things, I believe, are uncomfortable. They're painful. They're going to hurt, you know?
One of my favorite quotes is from a guy named Ronnie Coleman, absolute monstrous freak of a man, one of the legendary bodybuilders of the world, and quite the wordsmith. And I quote, "Everybody wanna get big, but ain't nobody want to lift no heavy-ass weights." [laughter] You know, it sums up so many things in life. Getting the opportunity in the year of our Lord 20-odd-26 to work as a professional engineer is hard yards. People struggled for years. Even me, who grandfathered in in Gen X when it was, quote-unquote, "easy," I saw the wrong side of midnight a whole lot. I spent a lot of nights and weekends studying to, like, get an opportunity to do this thing.
And if you can't deal with pressure, if you can't push yourself to an uncomfortable level of effort, it's hard to imagine people being successful or people being very good at this stuff. For a long time, I have held as my, like, sort of platonic ideal, like, a good marathon runner pace as what I aspire to in my professional career, right, like, where I'm running hard, and I'm feeling it, and I'm hurting. If you run for a long distance, it doesn't feel good, you know? But at the same time, like, you are in a flow state. You're rolling. You're in this pace that works.
But as I've sort of gotten older, I've maybe come to a level of acceptance of, like, how I work and, like, kind of how everybody works forever. It's very rare, very much the exception rather than the rule, that if you're going to do anything, if you're going to accomplish anything, if you're going to ship anything, be it writing a novel, or releasing a stage play, or shipping a new version of your app or whatever, you're going to hurt. Like, pushing yourself over the hump is just going to hurt. It's going to have this spiky burst of work.
And I personally have, like, raked myself over the coals for, like, decades over my inability to, like, just glide my way through these projects. Because I know I could just glide, and if I was a perfect person, I could just glide, but I don't. And very few people I know do; like, there are very few people I know accomplish meaningful things without, like, just some level of pressure and suck. And I've given myself a little bit of a hall pass to just be like, "Hey, that's what you're going to do."
If you want to make a big release, you know, if you want to get one of those bullet points on your resume that you're so proud of, yeah, you're going to see a couple of weekends; you're going to see some nights. It's just how it's going to be. And, I don't know, I mean, that's just sort of, like, my, like, maybe more nuanced relationship with pressure because it's just the same old thing all the time. You got to have some of it to do something really cool that you can really be proud of, you know?
MIKE: I read something from...I believe it was the person who coined the phrase, and I might have the phrase slightly off, but follow your passion, or something similar to that. It was this big idea, you know, follow your passion; do what you love doing. And then a lot of times people fill in the blanks after that, right? And then you'll never work another day in your life because it'll always just be joyful.
And I read an interview with the person who'd...they wrote a book about it or coined the phrase. I don't remember the exact details, so you'd have to look it up. But I read an interview with them, and they said, "You know, that keeps on getting misinterpreted. What I meant was: follow the thing you care about, and then you'll be able to deal with all the misery that you're inevitably going to go through to get there."
WILL: Oh, man. Yeah, so, you know, in an alternate, like, reality, right, like, I've been a working musician, right, for a long, long time, like, since I was a teenager, right? Do you know what a recording studio day is if you buy, like, a day at a recording studio? That's 12 hours. That's a 12-hour day. Like, I bought a 12-hour block. That is the block that it comes in. It's not like 9 to 5 with a half-hour lunch. None of that.
It's a 12-hour day. You will buy as many of them as you can afford. You will be working from start to finish the entire time. And, you know, as much fun as it is to have, like, you know, to make music and to see, like, your finished artistic process, you know, realized, it is a brutal, brutal grind. If you have ever listened to different snare drums for three hours continuously, let me tell you that the new penny shine does wear off quickly [laughter], you know? And, I don’t know, I could give you so many stories about, like, people who have pursued a life in the arts and the absolutely savage hours that they undertake for the privilege.
MIKE: And not everybody can do it because they...No. They didn't love it enough, they say [laughs], but there's people who do. Many of us who have things that we love doing, and we're willing to suffer through it because, at some point, it is inevitable; it is innate, I think. It's inseparable. Accomplishing anything meaningful is going to be tied to some degree of suffering, some degree of pressure, some degree of...This is not fun, you know, type 2 fun, as they call it in outdoor activities. Like, type 1, it's just, you know, everything's great. And then type 2 fun is, yeah, that was hard, and that hurt, but I'm glad I did it.
WILL: Yeah. So, I guess maybe, like, maybe, like, operationally, right? So, okay, when is it good? When is it bad? When is it necessary? And when is it too much, right? Like, how do you draw those subtle distinctions, right? Because, like, because, again, if you're even listening to this podcast, you're so far down the rabbit hole of dealing with all of this. You've done it all. You've done all of it.
Even Vivian, who's just starting, right? Tell me what finals week is like, you know what I mean? Like, and you're just getting started, but you know what I'm talking about. What is final group project week like? You've got the deadline; you've got to ship it. And then, you know, if you haven't figured out that it is not going to change any [chuckles], let me break the news gently as I can. Like, you're going to be steady rolling on this forever. So, how do you draw that line? When are you doing a good job, and when are you doing a bad job, you know?
MIKE: I was thinking about this ahead of time, and I'm going to...I was going to touch on one aspect of this, because I don't think it's the only one. But one thing that makes a difference is the degree of autonomy that you have in dealing with this. And I'm going to define that a little bit more broadly. You think about an open-book test. They can be harder because the professor's going to make that hard, right, than a test where you're just testing the knowledge that they've acquired, and you don't have that...
But people still often prefer it because there's no arbitrary rules, right? You can go back. You can reference it. That world has probably changed. Now we've got AI, you know, the whole world's turned upside down. But assume [laughs] that time that you'd have an actual physical book sitting there in the test that you can reference, and you have, you know, like, a time limit. So, you know, you couldn't possibly just start reading the book at the beginning of the test and be able to answer all the questions. So, you have to learn the material.
But if you have that reference material so that, you know, the arbitrary slip in your brain, "Oh, I can't remember that word," and you can look it up, and you know where to go look it up, makes you feel like, "Oh yeah, I can do this." And I think that naturally takes the pressure down because you have the tools that you need. When you're given the tools that you need, then I think that...and, you know, that can be training, that can be the ability to search, whatever the case may be, you know, somebody to reference, "Hey, how does this API work?" I think that changes things dramatically because you don't hit...The times when you hit a roadblock, and you can't do anything, right? You're actually blocked.
One of the things that is most frustrating is, "Oh man, I'm ready to ship, but the infrastructure's not ready." Or, you know, "I've done my part of the assignment, and my partner didn't." The things that you can't control because you literally can't do anything else, I think, are what's most frustrating. Those are the things that take it from good to bad really quick.
TIM: You touched on something. This is the reason I was interested in this session, because pressure in terms of cybersecurity is such an interesting topic. And it's something that we look for when, like, interviewing someone into the job and trying to get a gauge of how they perceive pressure because cybersecurity work is adversarial by nature. And there's this whole concept within industry called ethical pressure, which is so cool.
But on the topic you were referring to, I like the concept of being prepared to fail. Like, you ever gone to Florida or somewhere in the down south where we've got, you know, these huge causeways that are going across large bodies of water? You start going across them, and you hear that da-da, da-da, da-da, and then you, like, take a sec, and you look at what's going on there. They've got this huge bridge expansion joint right there. So, when you have a natural event, your bridge doesn't destroy. It's designed to split at that point so the whole causeway doesn't get torn out of the ocean floor. It's actually designed: "Oh, well, we're just going to adjust. We're going to fail for a brief period of time so that we can still maintain the overall concept, which is a road."
Same thing; we see that in cybersecurity all the time. It's the whole concept behind a honeypot. We know that your front door is going to fail, and we can't protect it 100% of the time. So, when we do get to that point, someone's going to eventually pop your password or your account, whatever it is; we flip a rule, route that traffic to a honeypot, let them win, so to say, so that we can see what their goal is, stuff like that.
Like, the whole point, I think, of good pressure is to learn how and where to fail so you can be prepared for that. Like, calm execution matters so much more than heroic or dramatic improvisation. You're not going to be good at that point to be calm, prepared, and be able to cope with that pressure unless you do what we call ethical pressure, which is, like, you know, you do penetration tests, you do...And site reliability was, what is it? Chaos monkey tests.
DAVE: Chaos monkey.
TIM: Yeah. Every kind of discipline within software engineering has their own flavor of we need to inundate the team with failure so that we can learn how to handle this routinely when it comes around.
WILL: So, when you're talking about pressure, I feel like you're talking about, like, sort of, like, pressure testing systems, like, on an engineering level where you're, like, sort of, like, putting these systems under pressure to, like, just, like, we're talking, like, a stress test or, like, a failure mode for penetration or something like that. Okay. All right.
TIM: Yeah [laughs]. That's a lot different from the business side of it, which is hit a deadline, right? This is like I want to pressure the infrastructure, the actual functionality of the whole thing, yeah.
MIKE: Is it so different?
TIM: No. That's the part, right? It really isn't [laughs].
MIKE: Well, somebody who's been in a few roles here, Justin, a number of years ago, sent me a book that he had read, "Antifragile." Has anybody read it or familiar with the concept?
DAVE: I've read of it.
MIKE: I read through a lot of it [chuckles]. I don't think I quite finished it, but I read through a good portion of it so I can speak to the idea. The idea is that there are some systems that, when pushed, do better, to a point. It only works within a band, right? You push it beyond a certain point, then it has failure.
But a good example, kind of a classic example that refers to what Will is talking about, is a muscle, right? Under stress, a muscle will become stronger. It's designed to actually do better under stress, and if you don't put it under stress, it'll atrophy and eventually just go away. My dad had a tendon, I think, in his shoulder snap. And he mentioned that the muscle it was attached to was just essentially after...He actually didn't know that this had happened [laughs]. He'd had surgery on it at one point, and eventually it snapped, and he wasn't aware of it.
But the muscle associated with it had just withered away to nothing. It's essentially not there anymore. Because it wasn't pulling on anything, your body just absorbs it more or less. It disappears. But if you use it, right, if you use it, you can get all big [laughs], if that's what you're after, or at least strong, depending on your goals. But some systems do better under stress, and that's a really interesting idea: that any of those systems, you want to keep them under stress because that will lead to improved performance. That's the idea behind the chaos monkey, right?
If you're always breaking your system all over, then it improves that system because, over time, you become more and more resilient to where you anticipate it before it happens. And then when real bad things happen, you're okay because [chuckles] you're already way stronger. And on a personal level, and I haven't really explored that with, but, I mean, you develop skills by putting some pressure on them, right? You work it. You develop it. You push it beyond comfort and find out where the gaps are and then start filling those gaps. I don't know what your all's thoughts are.
WILL: No, I mean, I think, that's exactly right. I mean, I think most of the people...I guess I'd say two things. I think, like, most people I know who are successful are sort of [laughs]...if they don't have a problem, they'll invent one just to keep things easy, you know? Like, they will get bored. And maybe on a slightly darker note, I've found that my baseline level of anxiety is pretty much independent of my actual physical or mental circumstances. Like, if I don't have some drama, then I'll invent it, and so I might as well invent productive things for me to be wound up about so that...you know what I mean? My nervous system's baseline level of cortisol is inadvertently directed to something productive.
DAVE: Pick a good LMO.
WILL: Yeah. I'm not going to stop, so I might as well harness that energy towards something useful. Yeah, I've spent a lot of time, like, chasing inner peace, and it's just like, this isn't in me, man. Maybe in the next life. Like, I'm just going to be productive and mostly happy. And I don't know, but it's like...
MIKE: You remind me of another reference I read about somewhat recently. They've done studies of hunter-gatherer societies, people who live in hunter-gatherer societies. There aren't very many of them left, right? Most of the world is pretty modernized. And even those that exist usually have some connection to the outside world, but there are some societies where, you know, the people pretty much...I mean, they're getting their food through hunting and gathering, pretty much how they live.
And they did a study a few years back, and I'd have to go look it up. But they looked at the calorie expenditure of people in this hunter-gatherer society, and everybody thinks, "Oh, wow, they must be burning, like, 5,000 calories a day," right? They just got to be these superheroes because they're running around all day looking [chuckles] for their food, digging up their food. And what they found was not that at all.
They found that their daily caloric expenditure is almost exactly the same as some guy sitting in an office. It's the same. And you're like, "But how is that even possible?" It's surprising. But it turns out that when your body is under that stress, it uses...You know, because they actually are active, right? That your body will start shutting down systems that are less important, such as inflammatory processes.
So, if you're sitting in that desk and not running around chasing a gazelle, you know [chuckles], or digging up roots somewhere, you are not stressing that system. And so, your body's like, "Okay, I'm going to take care of all the background processes." And so, all that background process handling can go overboard, and you end up with inflammation. You end up with, you know, with your body actually going too heavy on the, like, the immune and take care of itself response, which can be harmful.
So, it's fascinating. Like, even though the total energy used is the same, the outcomes are different, and the only way to keep your body healthy is to stress it some so that it doesn't overdo it on those background processes. I thought that was really fascinating.
TIM: Yeah, I remember I even had a therapist tell me about that applies to, like, mental stuff, too. The body is really good at adapting itself to do whatever you are doing in the most effective way possible. And it's even more traumatic from a, you know, a psychological or even a physical perspective when you go from...if you take the desk, the office worker, and then overnight put them into, you know, an Aboriginal tribe in Australia and be like, "Okay, figure it out," like, the body will go through shock, same thing with, like, relationships, same thing with your career.
You're really good at being optimized at what you're doing. Calories is everything, yeah. And that pressure and stress forces you to stay out of the optimal pattern, so hopefully you're progressing or growing as an individual. Otherwise, we're really good at coasting as humans [laughter].
WILL: Well, I mean, I'll say...
DAVE: That is interesting because a weird parallel to that is mentoring is one of the best ways to get a step change in somebody's performance. So, you're bringing them in, and I'm going to show you the entire process.
Worked at a call center years ago, and they put me with Christie. And everyone in the call center was knocking down 8 calls a day; really difficult tech support. So, I mean, like, 8 calls a day, like, a call an hour was reliable. Christie knocked down 30 calls a day like clockwork, and she was my mentor. And when I was done, I was knocking down 20 to 25 calls a day. I was the new guy. I had to go to Christie for some of my calls.
But it's like, how is this? And I'm like, well, the first couple of days with her were really, really stressful because she hangs up on a customer, immediately answers the next customer, and is typing up her notes on the previous call while asking the next customer, "What is the problem?" and da da da. And nobody else was interleaving, and that's...I realize it's a kind of a weird thing, but I picked it up because of the mentoring. I guess if we drop you in a tribe of Aborigines, that's going to be extreme mentoring, but it's the same kind of idea. You pick up all of the side habits that coexist and support it.
WILL: Well, I mean, like, I'll tell you one thing about, like, working out, right? Like, so this is devilish, and this has nothing to do with programming, well, maybe only tangentially. But a lot of people, they work out, and they have this misunderstanding about how calories actually work, right? And so, you'll go out; you'll go for running. "I'll go run for a mile, and I'll burn, I don't know, some 100 calories," some pathetic amount of calories, right? And so, "I'll burn 500 calories, and then I can get a Snickers bar," right, "at the end, and it'll even out." No, you cannot, because what will happen is your body will try to achieve homeostasis. And so, like, you will burn off 500 calories, right?
And, like, the first thing it's going to try and do is make you ravenously hungry, right? Because if you owe your body 500 calories, you're going to have to pay up. But the second thing it will do, you're just like, "No, I'm going to white-knuckle it, and I'm not going to eat," right? Or, "I'll eat some broccoli," or whatever.
The second thing it'll do, it will throttle down your metabolism, right? It'll lower your heart rate. It'll lower your energy expenditure. And it will also lower all that energy that you were spending stressing out, which is really good [laughter].
But, you know, so if you find yourself with all this anxious energy that is making your life, you know, less pleasant, and you'd be like, "God, I wish I could just chill out," burn off 500 calories right at the beginning of your day. And, like, your body will...anytime you find yourself trying to freak out, your body's like, "No. No, you owe me. You still owe me 500 calories, and you want to spend all your anxiety on this? No, go take a nap, or better yet, give me a Snickers bar."
DAVE: Your body starts to mortgage, right? It's like, oh, this is just a temporary famine. I'm going to shut down in order to bank to get that Snickers where I'm not running healthy, right? All the intermittent fasting guys are really up on this. It's like, you've got to go for 18 hours because you're still burning sugar at that point. It's not until about the 18th hour that your body finally goes, "Okay, fine. I will operate in a sustained mode like this," and you switch over to ketones and yada yada.
And it's that same thing, right? It's like, if you just do short stresses, your body just increases its kurtosis. How far away from average it's willing to go and still believe nothing has changed without moving the center point. [inaudible 27:53], "Oh, he's just starving us for a minute? Fine. Shut him down. He'll give us a Snickers later." And then, boy, you're going to get into insulin and bank that fat.
MIKE: This is still kind of tangential, but I've found the same thing. If I can beat down my body a little bit in the morning, right [laughs], that workout, then my stress levels are way lower during the day. I don't know, it's hard to measure, right? But there's, like, this well of calm I can draw from because my body is like, "Okay, you've already taken care of the stress today. You don't have to do that anymore [laughs]." And it helps.
I don't think it's the only thing that helps, because there's a variety of other techniques that, you know, you have to do to be able to deal with it, but it helps to stress...you think, "Okay, I'm in a stressful situation. What I need to do is go to bed." No. What you need to do is get up early and do something additional, and then your day will be less stressful, which seems totally counterintuitive. But it works.
WILL: Yeah. Like, it works great.
MIKE: So, for software, what does that mean? How do you develop, you know, that, you know, what's your chaos monkey, right [laughs]? What's this anti-fragile thing? What do you do to stress the muscle, you know, stress the system so that it does better?
TIM: So, I think the first thing that we need to call out is that sustainable performance matters more than the stress. It is important to have stress and pressure so that we know where the faults lie and where we can improve performance. But just like working out a muscle to complete absolute failure seven times a week, chronic urgency just creates burnout, and eventually, from my perspective, it increases security risks; healthy on-call rotations fail. You actually need physical recovery time as an employee, as an engineer, you know, just getting away from your desk, and then staffing becomes a problem because then you have high burnout, high turnover, and then you start losing tribal knowledge and things start to compound.
So, there's, like, a healthy...it's a bell curve of healthy pressure, and up at the front end, you're losing efficiency because you don't have healthy pressure. But on the back end of the bell curve, you have...The contributing factors are just spiraling after that point. So, you really have to find that healthy point in software engineering. Where is healthy pressure, and when is it to call it quits and let people be people [laughs]?
WILL: Well, I mean, yeah, I mean, I love a good...not love, I accept, as necessary, a good, hard burn when I'm trying to kick a project out the door. I've done --
TIM: A hackathon, yeah.
WILL: No [laughter]. [inaudible 30:39] a hackathon [laughter], but we'll leave that one alone. I think that's more, like, management failure, leadership by pizza party [laughter], adding responsibility. But no, I mean, like, a good hard crunch at the end of a project where you're getting it over the hump, out the door, you are launching it. You are giving birth to this beautiful baby, and there'll be blood, sweat, and tears. And then you're going to have a hangover, you know? You're going to have a hangover. You're going to have a sprint or two where you're just going to suck. You're going to be bad. Just own it, where it's like, "Oh yeah, I'm going to get this thing out, and it's going to be a glorious, shining moment."
And I'm like, I've found, like, as I get older and maybe a little bit more in tune with, like, you know, my own internal life, when I ship a project, I bum out the next day, you know? Like, I bum out.
DAVE: Crash.
WILL: Yeah. Like, I'm sort of bummed, like, the day after. I ship it out, and, like, that weekend I'm just like, "Woo, woo," just a big mope, you know? Because I came down. And I'm going to suck for a sprint or two, you know, no zero days. But, like, I'm not good. And if you're sort of, like, managing your team and their stress, like, you're factoring in that. And when you're not managing pressure, then you'll just be like, "Okay, we did crunch, so hey, appreciate you guys. Back to just a regular sprint," you know? Like, it ain't going to be that way. It's not going to be that way, and it'll...You can't push it.
And you can get into, like, really dark anti-patterns where, like, you'll overpromise. You'll get in a burn, right? But you're not going to make your goals because you can't manufacture capacity that you didn't have in the first place, and so you'll do your crunch, right? And then you won't have delivered on the promises you made, but you crunched your team, and you just squeezed them out as much as it'll be. And then you get into this sort of, like, suck-doom loop where everybody's burned out. They're not particularly productive. They're not doing much. But everybody's working 60s because you have this sort of cascade failure of, you know, broken promises.
DAVE: Did we talk about meaning? Dan Pink wrote a book called "Drive." There's a fantastic video on YouTube if you...bunch of them out there. The RSA videos where they do the whiteboard drawing of the guy giving the talk.
But autonomy, purpose, and mastery are the three things that you need to really just, like, be driven to do things. And autonomy is kind of the, we trust you and value you enough to let you do your job the way you want to. Mastery is, is this something you can get better at? In other words, is it worth getting better at? And then, yeah, the purpose is: is it tied to, like, a valuable mission?
And the weirdest thing is, like, when you put in 70, 80, 90 hours, and you hit the deadline, and everybody's happy. And your boss comes in the next day and says, "Congratulations. Your reward is you get to keep your job [laughter]." I've seen that happen. Half the team resigned. And he was mad that we, I was one of them, that we walked out on him. It's like, yeah, you deserve that.
I had another employer that gave us comp days. It's like, "We love you. Go home. Don't come back till Wednesday." And man, we would crawl over glass for that guy. It was fantastic. Yeah.
MIKE: We talked about --
TIM: Yeah, my last SRE team, we loved comp days. You could get a bonus for being on call. And, eventually, we're just like, you know that... just forget about the money. But if you give me a four-day weekend every three weeks for doing a rotation, I love it because I spend more time with my family doing things I love doing, things that money can't pay for. I want that time.
MIKE: Absolutely.
DAVE: The other side of meaning, and somebody said death slog a minute ago, and that's what keyed me off on this, is that...I was just talking with T1, Travis, my team lead, and I sat him down. I said, "Let me tell you how to manipulate Dave Brady. If you tell me, 'Boss wants butts in seats. Everybody here until 6:30,' I will punch out at 5:00 so loud you can hear it from two buildings away. If you come by my desk at 3:00 o'clock on a Friday and say, 'There's a rabbit in the field that needs caught, and we need it. We've got to get this taken care of,' I will be at my desk Monday morning wearing the clothes I was in on Friday and smelling like I haven't been home for two days because I got to catch the rabbit."
Give me meaning, and I will burn the candle at, you know, both ends and the middle, and then I will crash, right? Drop is a real thing. Like, post-intensity drop is a very, very real thing, and how you manage that is key.
Mike and I had some really good heart-to-heart conversations in the middle of a really big tech push a couple years ago to integrate a really, really big thing for Acima that...I don't think I can name that project, so I'll just say it was a really big tech thing that we were trying to bring together, big integration, and it all came together at the very last minute. So, QA was testing everything end to end, and we were all...It was 3:00 o'clock in the morning on a Saturday, right, Mike? It's like we were --
MIKE: [inaudible 36:12]
DAVE: And we were on Slack calls typing code, and, "Oh, this piece doesn't work. Okay, let me go fix the onboarder." And we were turning and burning, and we got it done. It was fantastic, and we were so tired for weeks afterwards. I don't know if the project was worth it, but having that experience in the belt was worth it.
MIKE: We talked earlier about autonomy and excellence. It's kind of a couple of those aspects, but not necessarily the purpose. That was number three, right? Having that autonomy is what we started with, knowing that that was not everything and that need we have for excellence. You know, we want to do stuff, but we hadn't talked about the purpose yet. It's working on something meaningful. Like, even if it's just you finish the feature your company wanted, you know [laughs], and you got it out there, seeing that thing in production is nice. You know, there's something to be said for that [laughs].
WILL: Yeah. Well, I mean, like, and I...It's funny how, I mean, because on one hand, right, like, there is some...hmm, how do I put it? I hesitate to overstate the whole, like, notion of, like, you know, this burning, purposeful desire to sell, you know, leases or mobile phones or whatever, right? Like, I mean, there is some, like, there's a bit there, but, like, it's just people are social animals. We have a social mission at work. We're all part of this team, this tribe, and we do this thing, you know? Like, we want to do it, and we want to do well at it.
I've run into a lot of, like, the most burned-out, toxic teams that I've struggled with, like, morale-wise, have been when there was this death of hope, I guess you'd call it, right? Where they were going to keep on getting challenged and challenged and challenged and challenged, right? Push hard, push hard, push hard, but it's like this isn't...You know, the top of the hill is just another hill. There's no, you know, like, people need to believe in some sort of, like, progression or advancement and not just this endless cycle of slog, you know?
And it can be difficult because I've worked for several companies, big companies, that are going to be...they'll bury us all, but, like, they were going through really hard times, and there weren't a lot of growth opportunities. There was this guy...I remember I was...It was on Threads, so it wasn't a tweet; it was a Thread. But it was this guy, and he was talking about how he made it to, like, senior director in some, like, ridiculous period of time. He had a bullet-point list, you know, of the thing, right?
And it was a whole long list of Threads, right? Like, he was talking about all the things that he did and, like, the projects that he took on and, like, the leadership opportunities that he seized and the initiatives where he just did extra. But, like, the first bullet point was like, "I was in a company where we were 10Xing," you know, like, "We were on this massive growth trajectory, and so I just happened to be there," and then all this stuff, you know what I mean?
And if you're working for a mature enterprise, you know, like, it mitigates the level of sort of, like, you know, pressure and purpose, right? Because, like, on one hand, it's like, hey, you know, with this company, right, whatever it might be, right? I'm not calling out, you know, certainly not calling out Acima or Upbound because there's dozens of these. They're almost the rule, not the exception, right, where, like, the company is just going to do what the company does, and it might go up, and it might go down, you know?
Walmart is not going to 10X, ever. It's as big as it could possibly be. They could do a little bit better, or they could do a little bit worse, you know? And you yourself can do a little bit better or a little bit worse, but it does, like, sort of, like, affect the perception. Like, I've never worked in my life harder than when I had my own startup. It was me, you know, my name on the door. I was co-founder of the thing, and I could work tremendously and get, you know, potentially unlimited rewards, right? Like, sky's the limit. And I worked, like, tremendous amounts.
But if you're working for your enterprise, limited trajectory, like, you know, very stable but, like, you know, neither too hot nor too cold, like, it changes the perception. And when you start applying this massive amount of pressure on somebody who knows that the pot of gold at the end of this rainbow is a 3% cost-of-living raise, you hope [laughter], changes the dynamics rather abruptly.
MIKE: It feels like many of the most successful engineers I work with have been through a startup or several because that experience is really hard to get another way. You've got to swim the deep end a little bit [chuckles]. It's a bit like that being dropped off in a tribe, "Hey, you're a hunter-gatherer now." Having the tribe there with you helps a lot if they can give you at least something. You know like, "Oh yeah, don't eat that one. It'll kill you." But that experience, you know, being stressed that way in a way where you can grow makes a huge difference, and if you go through a series of those, you just come out stronger.
WILL: It's true, you know? Like, it'll give you an extra gear. You know, it's hard to imagine you can't get that working for a sane and reasonable workplace [laughter], but you can never really know what you're capable of. You know, like, I listen to a podcast or two, and they'll have, like, some Navy SEAL, you know, guy out there who's, like, a motivational, you know, big deal business.
And they'll make points that, like, they're easy to mock, and they're easy to make fun of. But they really do resonate with my lived experience in that, like, you are capable of so much more. You're capable of so much more than you realize. And if you have the motivation and you drive yourself past your limits, you could do more. And I know I have, like, an extra gear that I can access because I've gone there and I've done, like, some intense stuff over the years. And that's a real benefit of pressure. Like, I'm not superhuman, and I will definitely crash, and I will bum out after I finish my project and all those things, but it'll get done on time.
MIKE: Well, I think there's a few practical lessons. Tim asked, you know, how do you keep from going on the wrong side of the bell curve, right, or on either wrong side? Like, we've talked a lot about, yeah, well, you need to hit the top of that, but how do you avoid going on the other side?
I think you have to recognize when you have gone that, like, and take the crash, right? Put yourself in a position where you get that break. If you're going full steam, eventually you're going to run out, and you're going to break down. You've got to put yourself in a situation where you're going to get that break.
On the flip side, and we talked about the startups being so valuable, I think there's something to be said for, you know, create an internship if you're doing interns...where people have to build something real and [chuckles] maybe their hands aren't held the whole time. Now, this is going into production; go ahead and work on that and, you know, we've got a real deadline. You know, giving that chance to actually have, "Oh, this is something I've got to do, and there's a deadline, and that's kind of stressful."
You know, give people a chance to shine so they can stretch. You know, don't try to keep every task so easy that people don't get the chance to grow. Give people legitimately hard tasks, and let them run with it. Let them struggle with it because that's how people will thrive. I think that we have to create that kind of environment and watch for that other side. Give people the chance to, oh yeah, you got an accomplishment? Take that comp day, right? Give people the chance. I think that those are both critical. You need to give people a chance to be under the pressure, and give the people a chance to decompress.
TIM: We've noticed in our cybersecurity team we need several mechanisms to make this work. We've even incorporated it into our weekly team meeting. We need a time-capped portion of a team sync where we just complain, and we vent, and our leader has to just listen to it. And he knows he can't do anything about half of it, you know? But the team needs to complain. It's a tough job, and we just need to vent as a team, "I dealt with this, this, and this, and it sucked," you know?
But the other parts of this that keep you from teetering from one side of the bell curve to the other—blameless learning is critical. No one should be at fault. We're all learning. We're all trying to do our job. Post-incident reviews are mandatory. We need to know: did we fail because we're burned out? Did we fail because we didn't have the right tools? We need to have an educated response to a situation so that we don't do it again. And we need to improve those systems and processes. And it just doesn't make any sense to punish individuals because then it just instantiates a culture of fear, and fear in any corporate culture is toxic, and people get out of there super fast.
DAVE: I've seen an executive get fired for losing the company money because it was that executive's job to know the company was losing money, and the company didn't know they were...The executive didn't know they were losing money. The executive got fired, and it seemed like the right call to me.
I've also seen an engineer lose the company a lot of money and just walk in ready for the pink slip, and his boss go, "What are you talking about? I just paid $25 million to train you. Get back to your desk." That is somebody who's like, "I love this company. I'm loyal. I'm sticking this out." Something else goes wrong...and you end up with somebody who when the next time the servers are on fire, can go, "It's okay. We got this. Trust me, we've been through worse. We're going to get through this."
Yeah, it's literally a...for the bosses listening to this, catch somebody failing and create psychological safety around that. It will pay off 10X. We used that before. It will pay off huge.
TIM: Yeah, we see that in cybersecurity so much. As soon as I find a vulnerability, or you put your password in plain text, or something goofy, wrong, if we do that in the wrong way, the next time you find a vulnerability, that person, engineer, business unit, they will do everything they can to hide [laughs] the fact that they did something wrong. And it's like, I don't want you to feel like you have to hide under a shell. I want you to know it's okay. Let's fix it together. Just be safe. You know, stuff like that.
DAVE: Yeah. It's the why are you hitting me with the stick? Oh, I need to stop doing that thing, versus why are you...stupid stick [laughter].
WILL: So, I want to...You know, I think this is an interesting question. So, if you're a boss, right, like, okay, you're a boss. You're a people leader. You're managing people, and you're in charge of, to some degree at least, the pressure that you put on people. How do you manage that? How do you gauge how much is applied? Like, how do you manage it?
I'm not going to lie to you. Like, my only real, like, conscious rule of thumb is don't. Because I will generate, like, just in my, like, living and breathing, a field of free-floating anxiety that I can't really resist [laughs] existing. So, like, all of my conscious action is like, "It's okay, everybody. Everybody, it's cool. Don't worry. It's all right. Everything's going to be fine. This is going to be cool," you know? Like, if you sat in a room, and you didn't even speak English, and your desk was next to mine and you only spoke Japanese, you'd be like, "I better get these tickets out." Like, you just pick it up [laughter]. And I have to combat that because it's not healthy.
MIKE: I think that I'll say, actually, something I've picked up from our CTO, Balaji, that I admire, that I think he does really well, is he'll see somebody he thinks has potential, and he'll give them a hard project. Like, okay, take this, run with it, and he won't even follow up that much. He's just like, "Okay, here you go." And then they come back, and it's not going to be perfect, right? But something will be learned. That person will learn what they're capable of. They'll learn a lot from that project, and now, you know, kind of they know where the limits are. And, oh, was that too much? Okay, well then you assign somebody else to help them.
That doesn't mean that it's a...I mean, in some ways, you could consider that a failure, right? They failed in it because you gave them too much. But that's okay. That's a learning experience. Give them something, you know, have somebody mitigate that, and then that's fine, you know [laughs]? Have somebody else help with that and solve it. And if they do accomplish it, like, there was a growth experience. Now we know what the limits are. Okay, here's another hard project.
I love that approach because, first of all, it gives...There's lots of autonomy there, right? Lots of autonomy, lots of purpose, lots of opportunity for excellency and all of those motivating pieces. And trust as well, and not the kind of judgment that we're talking about, right? It's a, "Hey, I'm not going to micromanage you. I'm going to let you do your thing and see if it works out."
And, of course, there's always going to be situations where it does...There are occasionally going to be people who shouldn't be in that situation. You can get them out. But most people, I think, given the opportunity, as we've been talking about, they'll want to run with it. And if you've been watching people, and you probably know approximately if they're able to do it, give them the task. Give people an endless series of meaningful things to work on. And that's not too bad of a thing to ask for. I mean, that's a pretty good life. Have a whole series of meaningful, challenging projects to work on. I don't know, thoughts?
WILL: I don't know. It wouldn't work on me. Like, I would just be like...because I wouldn't know if you cared, you know? Like, I would just be like, "Okay. All right," you know? I mean, like...
MIKE: There's something coupled with that, though. He's also...So, in this specific situation, Balaji's also very good at genuine, sincere appreciation. Like, "Oh, wow. You did that really well. Thank you [laughs]. You accomplished this for me." And following up periodically, "How is that project going?" So, he doesn't follow up every day, right? There's no micromanaging, but once a week, you're going to hear, "How's that project going?" Like, oh yeah, he cares. He wonders how it's going, and he wants a status report.
So, there are signs, regular signs, that there actually is caring, right? That it's not just send you off, never going to talk to you again. There's genuine appreciation, sincere, like, genuine, like, "Hey..." Because you can tell when somebody's sincere because they've really noticed, right? They notice the little pieces of it, and you can tell when somebody cares, and definitely get that. I think that, you know, we talk about psychological safety. Having somebody who listens and cares, that's hard to beat.
So, there's a pitch [laughs] for one way to approach it. Deliberately giving people challenging tasks, right? Giving them a chance to stretch. Thinking about that is a key part of how you're planning the work. "Oh, I think this will stretch somebody. Let's give them a chance."
DAVE: Well said.
WILL: How do you know when somebody's, like, under, like, too much pressure? I mean, like, there's obvious, like, there's like, oh, you worked all weekend, right? How can you tell when somebody's gassed out?
TIM: How do you know when someone's gassed out?
WILL: Yeah. You could just be like, oh, you know what I mean? Like, your perception of the workload is, like, you know, usually, hopefully good, right? Usually, hopefully, you know what you're asking of somebody, but sometimes things just don't go according to plan. Like, how can you read somebody who's under you and you're like, oh, they're not doing so good?
TIM: So, this is something I always look at on the other side of the interview table. A good employee is really good at knowing their boundaries and communicating their boundaries to their manager. That's the sign of a senior engineer or a senior employee, someone who knows, "If I do X, Y, and Z, I will not be able to do A, B, and C."
So, with my manager, I'm very transparent. I tell him, "Look, I can hit projects A, B, and C, but you just asked me for another project. Which project do you want me to cut? Which plate do you want to drop?" So, good engineers are really good at being transparent about that.
Junior engineers have this pitfall usually of they want to be the hero. They want to do the amazing things. You get the same amount of paycheck at the end of the day, and they haven't quite learned that yet.
So, you, as a manager, need to learn how to ask. You can't read that. You have to be able to appropriately confront your employee in a private setting and say, "Hey, where are you at?" Usually, that's like a weekly one-on-one or some kind of ceremony that you have, but I don't think you should read it from your team member. You should try to elicit it from your employee so that they feel safe to be able to tell you, "Hey, these are my boundaries." Otherwise, yeah, if you don't have that cycle going on, it's never...No one's a mind reader, you know?
WILL: I don't know. I mean, like, I haven't...Like, I don't always...how do I put it? Like, I try to have a plan B other than, like, you know, perfect. Like, if my management style and assumptions of private communication and psychological safety always, like, I feel like I'm maybe, like, letting myself down, you know? Like, "Oh, they'll just tell you." And I'm like, "Maybe."
MIKE: Yeah. They might not.
WILL: I mean, in the end, they wouldn't be over-pressured and burned out if, you know what I mean, like, you as a leader are managing their workload correctly, right? So, like, already we have one failure and then, you know, assuming that, like, "Okay, but they'll just tell me if they're burned out," you know? Like, something already went wrong somewhere up the chain of communication for you to get into this state.
MIKE: So, you're going to have to eat some crow [laughs] and be willing to say, "Hey, it seems like you're not getting as much done lately. Are you okay? You know, do you need a break? I may have put too much on you." I think, like, you know, you're implying that there needs to be some vulnerability and accountability from the manager saying, "Hey, I may have misstepped here. I'm sorry. You're not okay."
TIM: Yeah, that's a good manager if they can approach that conversation in a wholesome way. Like, someone that asks those questions is a good manager for sure.
MIKE: I think we've covered a lot of material here. We've talked about how to approach this from a manager perspective. We've talked about it personally. You know, we've talked about some of the things that you can do to stay motivated so you don't burn out, looking for some signs, you know, of burnout, managing that. I think I want to take this to a close.
I've got one more image that keeps popping in my mind as we're talking about this. I've done a lot of long bike rides with my oldest son. He loved bike riding, and so I got into it, and now I still love it, and I do it without him. But he's grown.
But when I go with him on a long bike ride, and we, you know, do, like, these big, epic, brutal, all-day type things, or all night, and he always has me ride in front because we'd noticed early on, and especially when he was younger, that he would burn himself out. You know, he'd push hard; he'd burn hot, and totally wear out, and have to crash, right? Not crash as in literally crash, but the figurative. And have to take, you know, a long break, get back riding again. Where, you know, old guy with more experience knows what the boundaries are and just keeps a steady pace, keeps it going.
There's pressure there, right? Continuous pressure. You know, you got to get to your destination. You got places to get to. But with experience, you can learn to...Well, you got a hill to push up, and then you slow down afterward so that you can maintain that steady heart rate, just keep going. And that senior mentor can lead the person who's coming up behind to know where those boundaries are. "Okay, slow down [laughs]." "I don't have to push so hard." And you can develop a steady state where you get into that flow state, like, "Hey, I'm good. I can keep on going and going."
And that's not something you get overnight. It's something that you work on over time. And in that social environment with a mentor, you get there. I think that kind of sums up everything we've been talking about. You've got that mentor; you've got the balance; you've got the sprints, and then the time to balance out afterward. Let's do things that way so that we can keep that steady state, so we can go see cool things: look at the sunrise, see the things you want to see.
Until next time on the Acima Development Podcast.
On this episode of the Acima Development podcast, Mike hosts a conversation about software development life cycle methodologies, opening with the line usually attributed to Mike Tyson: everybody has a plan until they get punched in the mouth. He illustrates it with a day from about 20 years ago in the suburbs of Chicago, when he and his wife planned to collect a free piano from an elementary school and a bunk bed with a slide from a seller farther south. The U-Haul turned out to be an hour away through traffic, the truck had no ramp, the movers he had hired gave up waiting, and the piano sat at the top of two flights of stone stairs. His wife's idea rescued it. They offered the movers' money to a construction crew working down the street, and the crew carried the piano out. Mike got home around 11 that night. From there he lays out the history, from the linear process Herbert D. Bennington documented in 1956 and nobody called Waterfall until the mid-1970s, to the 2001 Agile Manifesto, which he points out is four stated preferences rather than a process.
The group resists the idea that Waterfall is simply obsolete. Justin notes that it got NASA to the moon on a budget worth several percent of GDP, and Kyle adds that it made sense when software shipped on CDs you burned and mailed, because there was no pivoting after the discs left the building. Dave recounts building the REALImage 3000 graphics card at Evans & Sutherland, where the chip spent six months in the fab and came back with two leads soldered backwards. Red and green were swapped, and the team had to write a very fast post filter to correct every pixel of every frame. He connects that to lean's idea of inventory, where the assumptions you have not verified bite harder the longer you carry them. Vivian questions whether Apollo was really Waterfall at all and argues the only genuine difference between the methodologies is the size of the planning scope. Will keeps pulling the conversation toward the business side, since engineering tends to love Agile while the people committing to quarterly numbers tend to hate it. Eddy floats a hybrid, Kyle calls it Wagilefall, and Mike credits Will with naming the version that fuses the worst of both: Whirlpool, because you circle the drain.
The last stretch is about trust. Mike argues that iterative work only happens when leadership is genuinely willing to say go build something, which he compares to commissioning a Renaissance artist to paint a ceiling. Vivian asks whether psychological safety runs both directions, since managers also need confidence in their engineers, and wonders how an engineer earns that upward. Matt answers with communication and honesty about what might not work, and takes the position that a bad hire is the hiring manager's failure rather than the hire's. Thomas describes trust as a currency that compounds as you show progress. Mike closes with his friend's dairy farm, where the manure is unavoidable but gets washed into a reservoir, fermented, and spread on the fields. Software works the same way. You will not avoid the mess, but you can build a process that deals with it as you go rather than leaving one enormous pile to land in at the end.
Transcript:
MIKE: Hello and welcome to the Acima Development Podcast. I am Mike, and I am hosting again today. A little side note: Acima has a parent company, Upbound, and we're fusing more. Like, we're more of a family or a team or something. Like [laughs], somehow we're getting closer together. So, we may be putting Upbound in the title in the near future, so don't be taken by surprise if you're listening regularly. That's it.
And with that, as usual, let's...Oh, I should introduce people who are here. We've got Dave; we've got Thomas; we've got Ramses; we've got Justin, Kyle, Eddy, and Vivian. I think we've all been here before, so we'll jump right in.
I'm going to first reference a quote by Mike Tyson. Actually, it probably isn't by Mike Tyson, but it's attributed to him. I looked this up, and I guess there's no, like, clear connection, but he may have said it. And the quote says, "Everybody has a plan till they get punched in the mouth [laughs]." Whether or not he said it, he probably had the same idea. He probably had the idea at some point. And, you know, there's this idea plans don't last very long once you get into, you know, sparring.
I was thinking of plans that I've had in the past that have failed. I had, like, a simple one. My family and I were going on vacation. As we went in the garage, literally in the garage, we were grabbing something out of the fridge before we left and noticed that the fridge wasn't running. Like, "Oh, that's not going to go very well for everything sitting in the fridge," and, you know, I had to redo the plan.
But I got a better one that was further back. About 20 years ago-ish—it was a little less than that, but approximately 20 years ago—I determined there was a few things that I needed. My wife and I decided, oh, we need to get a couple...two things. There's two primary things we needed to get in the suburbs of Chicago. One, we found out that there was an elementary school that was giving away all of their pianos. They didn't think they needed pianos anymore. We thought, "Oh, great, we'll get a free piano and bring it home, and then we'll have a piano," because we didn't have one. So, we thought, "Oh, that'd be great. We'll get the free piano. We'll just have to pay to get U-Haul to bring it home."
We also found that there was somebody farther south in the suburbs that was selling a cool bunk bed that had a slide, and it came down at the top bunk so you could take a slide, you know, it was cool. And we thought, "Okay, this is what our son needs." Came up with plans. So, I took work off that day, and I called U-Haul and asked them to find a location near where the piano was where I could pick up the truck, so I wasn't driving the truck through, you know, Chicago traffic. I'd pick up the truck, drive over to the school.
I called some movers, and I was going to pay them to meet me at the elementary school, load up the piano, put it in the truck. And then we were going to drive down and pick up the dismantled bed from the house of the woman who was selling it; drive home, roll the piano down the ramp because, you know, you're going down. We could do that when we got home and pull everything off. Made all the plans.
I got up, drove to the U-Haul facility, and found out that the U-Haul people had no idea what they were doing. And it was at least an hour away through dense traffic and neighborhoods over [chuckles] to the elementary school. You know, this was 20 years ago. This was before...You know, technology has advanced a lot. In those [laughs] years, they didn't have anything, and thus began the series of [chuckles] things that went wrong. We, like, "Well, this is the U-Haul we get."
Turns out the lift haul that they had gotten for us also didn't have a ramp in the back. It just had a low bed. So, anything that went in and out had to be lifted in and out. Remember, this is just my wife and I and my, like, three-year-old son [laughs], and a very heavy piano. Like, "Well, this is what we get, I guess."
And it took a lot longer to get there because of the traffic than I expected. I call ahead to the movers and asked if they could wait a little bit, but in the traffic, it was going to be, like, two hours. We weren't going to get there until hours after the movers were supposed to meet us. And, in fact, it took us so long to get to the elementary school that by the time we got there, they were just about to close. So, there was, like, one lady who was there who showed us where the piano was. There was one left. The others had all been taken. Like, okay.
And this is this...It was an old school in this fancy, old building with, like, two flights of stone stairs being the only way out, and a really heavy piano, and my wife and I [laughs], and a truck with no ramp in the back. Like, oh, okay. It turns out we actually...There actually was an answer here. And my wife says, "You know, when we were driving down the street, I saw construction sky...There was a crew of people working there, crew of guys there building. How about...You had the money that you're going to pay the movers," you know, because they'd left hours ago. There's no way they were going to wait for us. "How about you offer them the money to come in and haul the piano down the stairs and into the truck?" Like, that's a good idea.
Drove over there, and they were willing. They're like, "Yeah, sure," because we had, like, 100 bucks or something. Like, "Yeah, I'll do it. Sure. Free money." Well, not quite free, but close enough, because it was a big crew of burly guys. So, they picked up the piano, took it down the two flights of stone stairs into the truck. Okay, yay, we're that far.
Next, we had to go pick up the bed, and the lady there had an appointment that night, and we had to...In order to get there ahead of that, we had to get there in, like, 30 minutes, and, by this point, it was rush hour. About two hours later [laughs], we arrived at her house. Luckily, she was really nice, and she had gone to her appointment or skipped her appointment just so she could get us the bed. I was so thankful [laughs]. There was no way. We were just driving through, you know, stop-and-go traffic for hours waiting to get to that place. And so, I was incredibly grateful.
Finally, so now we've got the piano; we got the bed, and started driving toward home. And there was no way to get the piano out of the truck when we arrived there. And I had to have the truck dropped off that night, or I'd pay for an extra day. So, I called somebody. I called a friend. He ended up calling somebody else who had, like, three or four teenage boys [laughs] who were available. They ended up meeting us when we got home, and they came in, loaded up the piano, got it off the truck, and even into the house, which was fantastic.
I drove the truck to the drop-off point and, like, you know, my wife following in the car. Got back and it was, like, 11 o'clock at night, completely exhausted, completely emotionally run through the wringer because there were so many steps in this [laughs] day that had gone horribly wrong and had managed to get out of. And I was just incredibly thankful and relieved [laughs] that I got the pieces together. Like, it was amazing. Like, I probably couldn't have pulled it off twice because so many things had to line up to make [laughs] any of that to work, because it was such an epic failure.
You've probably had plans like this. Everybody's had a bad day where the plans didn't work out, which leads us to our topic for today. We're going to talk about Software Development Life Cycle Methodologies, which is a fancy way of saying, you know, how do you plan for your work?
There is a style of planning projects that's been around since the 1950s called Waterfall. Did some background reading on this. It wasn't referred to as Waterfall until the mid-'70s, but the idea was documented as early as 1956 by Herbert D. Bennington at the Symposium on Advanced Programming Methods.
And what he presented was a series of stages. You do, like, your pre-planning, and once that's done, you move into your planning. There's a lot of different variations on what these steps are, but they...You can look...One possible, well, path through these steps is preliminary analysis, systems analysis, requirements design, systems design, and development.
So, you know, you start your development, like, stage four, then integration and testing, acceptance, installation, deployment, maintenance, evaluation, and disposal. It's a linear path. You start with your plans; you execute your plans; you test it; deploy it; you do your maintenance. It's all good, right? It's not right, but I'll say that [laughs].
This methodology took off in the ensuing decades because there was nothing better. And it's sort of better to have a plan than not to have one. So, people say, "Well, yeah, I mean, at some point, you got to plan, sometime you got to build it, and then sometimes you've got to, you know, move on from there." There's a lot of gaps in this process.
There was a famous event that happened in 2001. A group of 17 software practitioners got together and wrote a document called the Manifesto for Agile Software Development, and it was actually pretty revolutionary. You get some people together and write about the real-world problems they've had, and it has an impact.
It isn't even a process; it's just a list of preferences. If you go and read that Agile manifesto, it's like, "You know, we prefer to do things this way to that way." But following that event, there have been lots of practices that have been built up around, like, getting away from this idea of a linear flow, because it doesn't matter how good your plans are; you're going to get hit in the face, and then everything's going to have to change.
And those well-laid plans, it doesn't matter how well-laid they are; they're going to be wrong. And so, the world has largely moved on from this, that linear approach and come into iterative processes, iterative planning processes that we collectively call agile, agile software development. And there's a bunch of them with different names, things like Kanban, Scrum, extreme programming, popular methodologies. They all share in common this idea of getting feedback early and doing planning in a loop rather than a line.
So, lots of background and a long story. That's the background today, is to talk about what you get from these different planning methodologies, you know, the good and the bad. It's not like using agile practices solves all your problems, but it does address some of the chief problems that people have had in Waterfall. Again, bringing with it its own problems.
I'm not going to talk about all the problems because I've been talking for a long time. I'm going to do as I usually do and shut up for a little bit and [chuckles] let other people jump in and give some thoughts, and then I, you know, nudge the conversation forward. So, thoughts?
EDDY: I think on paper, waterfall, I think, sounded good when it was first proposed, right? Like, get this done. Once this is done, get this done. Once this is done, get this done. Once that's done, get this done. It gives you a structure, right? You're like, "Okay, I know exactly what step I'm on. I know exactly how to advance. I know what's next," right? And, you know, you can easily speak to that.
But I think as software development has grown, right, throughout the years, throughout the decades, throughout centuries even...Well, maybe not centuries, but you quickly realize, right, that you can't go structured like that, right? Things are moving at a fast pace, right? And in order for you to be as flexible as possible, you're not just going to wait for the first step to be done, you know, when you can be working on other things and be proactive.
So, I think, like, a major pitfall of waterfall is that you are stuck, and you can't move on until the previous or the current step that you're on is completed. I would expect someone working in waterfall to hand in a product much later than someone who was working agile, for example.
JUSTIN: So, I have a counterpoint to that. Waterfall methodology was what got NASA to the moon in the '60s, and got the U.S. to the moon. And I think it's, like, one of the greatest successes ever shown of the waterfall methodology. So, it's not necessarily a bad thing. It's just very, very difficult. And if you realize how much money was spent by NASA, by the U.S. government, to get us to the moon, it just kind of blows my mind. You know, I think it's something, like, what, 5 or 6% of the GDP or something like that during those years.
And, obviously, as a software engineering company, or most companies aren't getting that kind of budget. And so, you've got to figure out some other way to do that. And also, it seems like these days...I don't know, it's an interesting comparison because, in the '60s, the technology was moving rapidly as well. And so, it's moving rapidly now. It was moving rapidly in the '60s. And so, it's just like, I don't know, it's kind of the same thing. So, I'm not going to shoot down Waterfall because it got us to the moon --
EDDY: But I also wouldn't support it just because one success story is going to define the whole industry either.
JUSTIN: Yeah, that's true.
EDDY: Just because it worked for NASA doesn't necessarily mean it's applicable to us [laughs], you know what I mean?
DAVE: I resist that characterization. NASA isn't a single fluke.
MIKE: Nor has NASA never failed.
DAVE: Yeah, that's true, too. That's true, too.
KYLE: So, Waterfall and Agile, I think there's a reason that Agile's a newer concept, right, because, like Justin's saying, we have adopted. And why has it been around? Well, it was around because you needed the definitive planning. You were delivering on physical media. I think that's a large part of what has made Agile appealing to us today, is yeah, I can release, you know, an executable any time of day.
Well, you know, in the '90s that wasn't the case. You're burning the CDs, and you're shipping out to your customers. You don't develop that way. You, you know, hit your plan, and you iterate on those kind of cycles because you can't waste the money just burning, and that's going to be across the industry. That's part of where, like, your hardware companies, not all of them, but you've got some hardware companies and stuff like that that are still going to be using Waterfall because they have to have that deliverable at a certain date to ship out. They can't just pivot, like, you can with Agile.
MIKE: That's a really good point. If you get one shot on your deliverable, whether it's your moonshot or your box of disks that you're mailing to somebody when they buy the software, that very much changes how you have to do things.
EDDY: So, you're basically saying waterfall is probably a really good methodology when you don't have a retry mechanism, right, like, when it's all or be done, right? Like, if you screw up, you're done, right? Like, there's no retry [laughs]. So, like, maybe in that scenario, I think it makes sense, right? Like, if you're shooting someone up, you know, to the moon, you can't be like, "Ah, no, that failed. We'll try again later," like, no [laughs]. Like, you're dealing with people's lives, right? So, maybe in that scenario where stakes are different, you know, maybe it does make sense, you know, have a waterfall.
DAVE: Absolutely. Absolutely. Agile has the idea of tracer bullets, where you get something out there, you build...You know, it's like, you want to build a vehicle. Instead of building the entire train, you build a skateboard, and you verify that the transit works at all. And that's your thin vertical slice that you end up getting something that can get you from here to there, but it's just a skateboard; it's not the full thing. And then you scale that up, and that's tracer bullets. And yeah, if every bullet is a billion dollars and 6 astronauts' lives, you're not firing, you know, 73 of these things and just hoping to get closer to the moon each time or closer to Mars each time, right? It's like, you want to nail it.
The comment I was going to make earlier is that, in software, waterfall works if you have built it three times, is kind of where the tipping point seems to be. I can't cite that reference; it might be urban legend. But I heard it at Evans & Sutherland, where we were working on the REALImage 3000 graphics card, and it was our fourth graphics card that we were building. And they're like, "We're just going to mostly do it waterfall." And we had a chip in the fab that was taking six months to build. So, getting everything done and designed all the way up front, including the device drivers that would run on the hardware when it shipped, had to be designed, and then we sent the chip off to the fab.
And then we went and we wrote the software, and we found bugs in it, and yeah, it...The SDLC, the bugaboo of the SDLC is when that chip is finished and in people's hands, and you find out that two of the solder leads, or two of the leads, have been soldered backwards, that's a very expensive fix, right? And that literally happened with our chip. We had one line that the red and green lines were reversed, and it made some of the most beautiful bugs you've ever seen. It was just, like, this psychedelic coloring on this one channel. And we had to stay up late figuring out a very, very fast way to swap two portions of a single pixel on every pixel on the screen at scale, at speed, right? It's like we were running a post filter on every single frame on it.
The thing that I take from this, and this goes back to tracer bullets, is that lean software methodology talks about not carrying inventory. Your inventory is the number of tickets that you currently have open, and it's also the number of assumptions that you have not verified. The longer you carry those, the harder they bite if you get them wrong.
And the thing I love about Agile is that you build that skateboard, and you get down there and you find out that, no, we can't cross this chasm at all right here. Let's not spend six months to build half of a suspension bridge only to find out we cannot cross here. Or let's build it and see if any customers are even driving past this point. Then, oh, yes they are, and they want it. Let's build fast. Let's go in a fast way.
Yeah, low inventory, and yeah, don't fire astronauts at the sun just to...That's a terrible idea.
WILL: So just came in, right? But...
DAVE: Welcome, Will.
WILL: Yeah. Well, it's always good to...So, my question has always been with Agile, right? I get Agile. And I understand from an engineering organization and management perspective, like, why it's good, right, like, why it's good. Because essentially, like, it's all about, like, not bullshitting yourself about your planning horizon and what you know, right? But in the end, when we are all sitting here working in the businesses that make money, and take investment, and have stakeholders, and shareholders, and planning and managers whose planning horizon is even worse, I would argue, because I can deliver a website, you know, quarterly numbers, it's a little bit more voodoo-y.
But they're consistently accountable for these sort of forward projections and forward things. And so, they require these sort of, like, impact-driven planning things to justify keeping us employed on this project, right? Like, they have to go to your boss, and your boss's boss, and your boss's boss's boss. And they have to say, "All right, over the next six months, I'm going to do these things. And you're going to get these wonderful results as a result of all the wonderful engineering that we're going to do," right? And which is the essential friction between, like, Agile and Waterfall, right? Because, like, the way that everybody gets paid is by projecting and promising, you know, significant, meaningful results.
There's a duality between sort of Agile, where it's like, I'm going to build an MVP, and then I'm going to build it a little bit bigger, a little bit bigger, a little bit bigger, a little bit bigger, and incrementally. And, you know, we're going to add all these two-week Lego pieces up and, you know, and we'll have this beautiful project which is going to make us lots and lots and lots of money, right?
EDDY: So, I personally like Agile. That's where I learned to work initially, and I've seen the success stories of Agile, right? Like, I like the idea of simultaneous incremental work, you know, from different teams, right? Amazing. You can be super productive as long as you're very communicative, right, and you're very transparent, right? It can be very successful. And I'd argue that you can hand things in thrice the speed that a Waterfall project wouldn't, right? So, speaking from a success story personally working at Upbound, I have seen it. It works marvelously, right?
KYLE: It depends on management, but yes.
MIKE: There you go. It depends on management, as Kyle said. And back to what Will said, what if you don't have a plan? What if there is nobody saying, "Oh, this is what we want to have in the next six months, the next year, the next two years? This is what actually matters to our customers." What if somebody's like, "Eh, I don't care. Just work on whatever you feel like next," and they haven't thought any strategy? You can get yourself in a lot of trouble. You can build lots of wonderful, little features that don't accomplish anything, and there's a real risk there.
There is no magic bullet, and so Agile is not one. Yes, it can solve problems, but sometimes it's perceived as a substitute for planning. Like, "Oh, you know, we're not going to have a plan." "We'll just be agile as we go." No, that's not being agile. That's making it up as you go, and that's likely to go just as bad [laughs] with Agile methodologies than it ever would with Waterfall.
WILL: It's not so much that as, like, sort of, like, trying to reconcile the necessary business planning and projecting that drives sort of, like, quarterly budgeting, quarterly conversations, shareholder reports, if you, you know, are fortunate enough to work for, like, a publicly traded company. Like, everything goes downhill from, like, all right...Like, it's incumbent on us to, like, come up with these quarterly, if not, like, yearly projections and plans so that everybody can be like, "All right, give us lots of money [laughter]."
JUSTIN: We got to get paid at the end of the day [laughs].
WILL: Yeah. I mean, we don't...You don't have to get paid every day. You can go full [inaudible 22:47], right, and just, you know, sit in your tower with your wizard beard, but [inaudible 22:54]
JUSTIN: Yeah. So, a couple comments on this. One is you do have to have a plan, and you've got to decide how deep you want to go in that plan. And that's all...That's kind of, like, the difference between...well, one of the major differences between Waterfall and Agile it's like, you know, the distance you're planning out and everything, and the direction you're going. But if you are like, "Hey, I want to do this," and then, "How do I get there? I do this, and this, and this, and this. How do I get there? I do this and this and this and this and this," and all of a sudden, you find yourself in a Waterfall situation. And it all depends on how enthusiastic your planning is and how, you know, how deep you plan.
And I think you can slide into a Waterfall type of situation because you promised these deadlines, you know, these items at these deadlines. And you're like, "I promise these things at these deadlines, and you will give me money because I promised these things. And if I don't make these dates, it could be a death march." And so that, I think, is kind of, like, the bad side of Waterfall.
One of the things that was interesting—I think that, you know, I brought back my NASA example, but also all my good examples of Agile—they all had, on both sides of that, they had a very clearly defined set of goals and testing, and things like that they were able to understand what success was. And, success, if you can define that beforehand, I think that is a key to good planning no matter what methodology you go with. And that could involve testing; it could involve, you know, specific things that work, but you got to have an idea of where you're going. And I think that's really, really key to both methodologies.
MIKE: I'm thinking about what you touched on a little bit, Justin, what Will was saying, that there's a lot downstream of that quarterly or annual planning and goals. Because once you start talking about dates like that, then saying, "Oh yeah, I'm going to give you an MVP with incremental value. "Like, well, how much is that worth?" "Well, probably not worth much of anything, but it's the right, you know, it's the first step." They don't translate well.
And this works if you...you have to have some level of buy-in, some sort of, like, suspension of disbelief or of reporting where management says, "Okay, I believe you know what you're talking about. Go and prove this out." And I don't know if I'm going to get what I'm asking for. Now, there's some honesty there because you never do, but, you know, you can have some hopes. In some situations, maybe you do. Like, you know if I do some more marketing, history says I'm going to get more customers. You know, so, there are some places where you know you can pull this lever and get something out.
Building stuff is harder, and you have to tell people, "I'm going to go be this artist. I'm going to go work on my statue, and yeah, you're not going to get anything for a while." And they're like, "Okay." There's a real level of trust that's required for that, and if you have a trust deficit, you're not going to be able to pull it off.
WILL: I mean, that's the thing. I mean, isn't it...I mean, I hate to put it in these terms, but isn't it with Agile you're sort of, like, telling everybody the lies up front versus, you know, in Waterfall [laughter] you never...I mean, like, the level of certainty with Waterfall isn't higher. It's not higher. It's not like, "Oh, well, we're going to get these features." It's not like you've never pushed a deadline or dropped a feature when you're Waterfall planning. It's like, it's the table stakes for software development.
But, like, when you get into a Waterfall situation, you're handing them the bitter truth, like, without wrapping it in the piece of cheese, you know? At the end of a Waterfall, it's just like, "Ah, well, everybody tried their best, and we just didn't make it," understandable and forgivable and, you know, probably shouldn't be surprising if whoever is managing this project has been doing it for more than a month [laughter] you know? More than a month or two.
But with Agile, you're sort of, like, you kind of you're pulling the curtain back and just being like, "Okay, we're going to do something small that works, and we're going to expand upon it, and we're going to take it as far as we can go that makes sense." That's all anybody could ever, will ever do, you know, in terms of, like, your product development. But you're just pulling the Band-Aid off quickly.
MIKE: Right. Which can save you a great deal of money if you find out, yeah, you can't [crosstalk 27:29] [laughter]. But, like, waiting until the end, that's incredibly expensive, right? If you wait to the end to get your failures. If you can put them up...But, like, I agree with you, Will. I totally agree with you. Waterfall's not less risky. It makes everybody feel like, "Oh, I've got a process. I've got a plan. I've got a goal. Everything's going to be okay." It's not [laughs].
WILL: And it gives upper management deniability, you know, not for nothing. It's a lot easier conversation for your VP to have with the CEO to be like, "Ah, you know, we have everybody, like, everybody's working overtime, nights and weekends, and we got to push the deadline out six weeks," you know? That's an easy conversation to have versus, like, "Hey, okay, so, listen, I'm going to let you...I want you to...Let me...We're going to borrow your engineering organization for a quarter, and we're going to make some stuff, and it's going to be cool [laughter]. But I don't know exactly what. We're just going to try some stuff out, you know? But, like, trust me, shareholder meeting is going to be sick [laughter]," you know?
MIKE: Absolutely.
VIVIAN: Okay, this is a bit of a break, but I'm...I have two questions/comments that I wanted to make. One, I wanted to question whether or not NASA actually used Waterfall to get us to the moon. I don't think they did.
WILL: Oh yeah. Oh hell yeah. You can't incrementally go to the moon, you know [laughter]? It's a monumental engineering project. A rocket launch is, like, the essence of Waterfall. Absolutely. Like, did they all do it in one go? No.
MIKE: Except they first built rockets. First, you know, like, in World War II, they built a lot of rockets for military, and they used some of those rocket experts to start trying orbit, right? And then they put animals in orbit, and they put people in orbit, and people died.
This wasn't some unbroken series of successes. And, you know, eventually, after going through all of these steps, they got to the point where they sent people to the moon. So, I think they're both true. It's about as Waterfall as they get, being that, yeah, you get one chance to send that rocket up; and that had vast amounts of planning and an unfathomable expense that went into building that rocket. But it's also true that, where they could, they did something more akin to what we'd call Agile today.
VIVIAN: That goes into my next kind of question/comment is, from everything that I have ever learned about Agile and Waterfall and all of the different sub-processes, the only fundamental difference between Agile and Waterfall that I can find is the size of the planning scope. Agile is just Waterfall smaller and faster. And you re-iterate. And there are details about it, about basically back-propagating the information so that once you've done the implementation process and you realize all the ways that your planning fails, you can revise the plan, and update the plan, and move on to establishing, like, what are the next actual steps to fixing everything? But it's just about the size.
And that's why I made the comment about the NASA moon landing being Agile, because they didn't build an engine in one go. They had experience building engines; they built a new engine, and then when that one failed and blew up, they built another engine, and they iterated and iterated and iterated. And yes, they iterated each of these individual pieces in more of what I would call an Agile process. But then the integration test of putting it all together, that needed more of a Waterfall approach. But, again, that's just the scale of what they could do. You can't iterate on launching a rocket when there's human lives on the line very much. You can do it, but it costs a lot. And I'm not talking financially, but it does cost a lot financially as well.
And so, I'm kind of curious if there's any specific kind of clear delineated difference between a Waterfall process and an Agile process other than the size of the scope of the planned work.
MIKE: I just dropped...If you read the Agile Manifesto, you know, I said that there's some values. It's four lines. Individuals and interactions over processes and tools. Working software over comprehensive documentation. Customer collaboration over contract negotiation. And responding to change over following a plan. So, yes, what is it? Well, it's those four things. That's what the Agile idea is.
So, you could say, well, it's just the scope, and, to some degree, that's true. That's more of the responding to change, right? The short feedback loop. But there's a...there are some, I think, key pieces there in that instead of focusing on the documentation as your source of truth, you focus on the prototype, you know, here's the thing that we're building. This is the source of truth. So, it's more important to build something than to document something. And you communicate to your customer regularly rather than all up front.
So, I think there's a lot of truth to what you're saying. I think there is a little bit of nuance beyond that, in that there's also a de-emphasis on documentation and an increased emphasis on continuous communication rather than communication upfront.
EDDY: So, which one would you say is more expensive financially upfront, and how does that contrast?
JUSTIN: So, it all depends on, like, if you are trying to get return for the money you invest, theoretically, I'd like to bring on the second point there, the working software over comprehensive documentation. The working software that is providing value, that is ideally what will give you your money back and, you know, continue the money roll. But other than that, I think they're both ways that you could just spend gobs of money and, you know, not get anything back.
WILL: I mean, for me, I feel like on an objective level, Agile is a pretty definitively superior solution. I've seen it work. I've seen it, like, the big Agile versus Waterfall, like, on an industrial scale that I've seen personally is...I used to work on a lot of smartphone stuff. And I remember, in the initial days when the iPhone came out, when the iPhone came out, it blew everything completely out of the water. And the number one competitor was Android, and Android caught up at an absolutely breakneck pace, and arguably today is an equivalent experience.
The apps still suck, but Android's just as good as iOS and leads it in many capacities. There's a lot of stuff that you'll get on Android first. And I think the new iPhones, when they launch, are going to highlight that, especially this year. And so, it's just this sort of classic, you know, move fast and break things kind of methodology. And the reason I ask questions, like, sort of like, "Okay, well, how do you reconcile this with, like, business planning and the business cycle and the nature of getting buy-in from people who are investing in you over a long period of time?" is because, like, how do we sell it? How do we sell it? How do we make it work?
Because I'll tell you honestly, like, nearly every place I've ever worked, engineering has always loved Agile, and the business has always hated Agile. And if I want to be able to work in the way that I want to work and the way that I think is best, I need to make sure that the people who are writing my check buy in. And for them to buy in, I have to be aware, and empathetic, and cognitive and supportive of their needs, you know what I mean? To me, that's the productive conversation to have. How can I sell it? How can I not leave my boss or my boss's boss with their butt in the wind? Just saying, "Trust me, bro, it's going to work out for you," you know?
THOMAS: I think, like, the Android market, too, is very pliable. Like, they're willing to push innovation and see how they can change the phone and everything, where you see, like, iOS, a lot of the new iterations of an iPhone, right, are very similar. They're not much different. But then, like, you look at Samsung, you know, they have all the fold options. They have all these different camera options, all these different, you know, screen options and everything. And I think that's really remarkable that they push that boundary.
Even though they think, you know, something is crazy as a folding screen, they're willing to push that boundary into the market just to see how well it works. And it's done well because they're making new iterations of those foldables. And now they're starting to make foldable screens. Like, Samsung has a foldable monitor...not a foldable monitor, but a monitor that you can curve and bend. And it's just cool because the science they apply to one market they're applying to other markets based off the information they got just from success they got on their one product. So, it's really unique.
WILL: But I just...I remember both of the...Like, I mean, I remember when the iPhone kind of sucked, honestly. It didn't suck. It was really cool. The iPhone was super, super cool, but I remember, like, the initial, like, what was it? Nexus, right? The Google Nexus One, that first Android phone. That was dog water, man. That was so bad. It was so bad, and they caught up so fast, and it was...I thought it was remarkable.
And anyway, I mean, I was trying to segue the conversation, you know, out of, like, sort of, like, tech things and more into, like, okay, like, what are the countervailing forces that, if we want Agile, we want to do Agile, if we want buy-in from Agile, that we can make this sale? And we can...and these are the ways it's going to come unglued. And how do we fix it? How do we keep this thing healthy and stable so that people aren't freaking out, or working nights and weekends, or, you know, all these things? And is it coming unglued on us?
JUSTIN: I'm curious for people who have been in college recently: what are they teaching the kids these days? [laughter]
VIVIAN: They told us that they were going to teach us Agile and then had us do Waterfall [laughter].
EDDY: So, it's applicable to the real world. Got it.
VIVIAN: They told us we were going to be learning Agile, and then they had us plan for an entire semester, do a prototype for a month, and then the entire second semester was implementing the plan [laughter]. We literally spent as much time planning as we did implementing. It was awful.
WILL: Would you say that they spent as much time teaching you how to come up with a plan as they did teaching you how to implement the plan, as if, like, coming up with a plan for developing software might be directly and intimately related to your work going forward?
VIVIAN: Not even a little bit [laughter]. They didn't spend nearly the amount of time teaching us how to plan as they did teaching us how to actually implement. But they did have us spend about four months planning and prototyping what we were going to be building, and that was the same amount of time that they gave us to actually implement it during our capstone.
WILL: Eh, I don't know. I wasn't there. I wasn't there. But yeah, you couldn't tell me nothing when I was back at school. Like, all these software engineering, team building, you know what I mean, all this sort of stuff, like, no, I had to learn my lessons the hard way. I would've just blown it off and cowboyed it over the weekend. I would've just, you know? Let's get a case of Mountain Dew, maybe a couple of Four Lokos. Let's party. Let's get this thing through.
VIVIAN: I mean --
WILL: We'll sit in here until it's done, and then it'll be done.
VIVIAN: It was an interesting experience because we had to form a team with people that we did not know, potentially, starting at the beginning of the semester, and then make a plan for an application that we had never built before, using technologies that we had not necessarily used at all before, building a scope of an application that we had never built before. And we had to do all of this planning under the assumption that we would be able to make an accurate enough plan that we could then follow it for multiple months with a fairly fresh team while we were finishing school.
JUSTIN: This is, like, every startup ever [laughter].
VIVIAN: And it was terri...It was great for the first three months of the implementation. We were following the plan; everything was going well. We were, like, adding features. It was all going really well. And then the last month, everything started to fall apart when all of the little things that we didn't realize were missing came crashing down. And that was why, leading up to the capstone presentation, I had two days where I worked, like, 16 to 18 hours each day working on basically reimplementing our entire system to actually work asynchronously and function in a way that actually was demoable.
JUSTN: Every startup [laughter].
WILL: Yeah. Welcome to the rest of your life [laughter].
EDDY: So, I was actually...I was going to mention, so Kyle shared an article that I thought was kind of funny, where it's like, you adopt both philosophies, right? You have Wagile, right? And you can kind of take the best from both and kind of have this hybrid approach. And I first was reading this article I'm like, "Ah, they got to be kidding. This has got to be crap," right?
But no, no, no, like, it's legit, right? Like, you get what best works from Agile; you get what best works from Waterfall, right? Do a hybrid of both, and you get the best of both worlds, right? So, I guess I don't know if that's a recipe for disaster because they're contradicting philosophies. But I'd argue that...so smartphones, because we were just talking about that, are built using a hybrid approach, right? They combine both Agile and Waterfall methods. Why? Because hardware creation requires strict sequential stages like Waterfall, while software apps and user interfaces are flexible, iterative Agile cycles, right?
So, I think that's a prime example where you can have both ideologies kind of coexist, right, and still be very productive, right? In smartphones, you can't just follow one philosophy; you can't just follow one methodology, right, for implementations. I think it's okay to do both. I haven't seen it in practice, but I think the idea is interesting.
KYLE: So, the article that I did post, it was just a Wiki article, for those not able to see the chat, but it was in response to Vivian. That is more akin to what NASA did. You're right; it wasn't full Waterfall. It was more of the Wagilefall or, you know, however you would want to say it, which is not an official methodology, right?
That said, I think you're completely right, Eddy. In hardware, you're going to see more Wagilefall-type things. When it becomes more of a meme—and it is funny—is when you have your glorious startups that start out on Agile, then management gets involved; they need more power; they need more influence; then you're doing Wagilefall. That's when it becomes more of a headache from what I'm seeing.
MIKE: And before the call today, Will was giving some ideas, and I think he nailed it. And maybe he should sell this, but he says, "You take the worst from Agile, the worst from Waterfall." He says he calls it Whirlpool [laughter]. You circle the drain [laughter].
WILL: Yes. Yes. Whirlpool.
MIKE: It happens, you know? You have ceremonies every two weeks that are just soul-crushing because they're so overboard, and you have year-long plans that you have to hit. And so, you're in an endless death march, and no feedback with the customers [laughs]. You can circle the drain for quite a while in misery that way. You can fuse the best; you can also fuse all of the worst.
JUSTIN: Until the money runs out.
MIKE: [laughs] Until the money runs out.
WILL: In this moment, right here, right now, today, sitting in front of you, I could not tell you what release the features I am working on are going in, what release is, like, currently in submission with the App Store, what release has been cut and is taking cherry-picked merges at a furious pace, what release has been cut and has yet to, like, actually be cut or, like, what our schedule is. I have no idea. I've been at the bottom of this whirlpool for a couple of weeks at this point, and I just deal with the ticket that the highest number of people are messaging me about the status of.
MIKE: How do you get the best parts? Like, a story I've been thinking about as we've been talking, because we've been talking so much about aerospace, I think a lot about SpaceX, and there's more than one thing to say about it. One thing that they did do is embrace the idea of iteration: let's not be afraid to blow up the rocket, and if you follow --
EDDY: Without people in it.
MIKE: Without people in it, yeah. Without people in it. Now, that wasn't maybe possible back in the Apollo days, right? So, I mean, we have things today that allow us to have some of these more agile processes. But if you look at the space launch system that NASA recently set up, do you know how much it costs to launch one of those rockets? Each rocket, I looked this up, is approximately two and a half billion dollars per flight, and it's disposable. That is, you don't get to reuse any of that. It goes up; it's gone.
EDDY: So, what attributes to that being so expensive? Is it because of all the salary, all the planning, all the organizations involved? Or is it the equipment, like, the material that encompasses? Like, what exactly is the most verbose expenditure when you're sending a rocket up to the moon?
MIKE: Some of that is the cost plus contractors. Like, they've fallen into all the dysfunctions that you can get of development, where they farm it all out to contractors, and the contractors get paid no matter what, and then they get paid additionally for the cost they go over. So, all of the horrible horrors of contract shops, they get all of that. And it's required to be in certain states for the senators who said, "Oh, I want it in my state." And so, it's poorly managed. Sorry, NASA, it's poorly managed [chuckles]. And I think most people see that.
If you look at what SpaceX has done, it is just shocking. They do over 100 rocket launches a year, and you don't even think about it anymore. They've got rockets that they've sent up, like, 40 times, and they're still launching them, right? The difference is so stark that if, you know, 20 years ago, you tried to explain it to somebody, they'd laugh in your face, because they're not even comparable.
SpaceX has sent up, like, I can't remember the numbers. I think it's more than half of the satellites that have ever been launched, and most of that's in the last two or three years. It just...It's a freight train, and there's nothing else like it. You can do it, and you can do it right by, you know, you still have to send up that rocket, right? There's still very much some waterfall aspect to it. But you can do that iterative development, and it works. And not only does it work; it blows everything else out of the water.
Now, there are some competitors; there are some competitors in China that are starting to do reusability now. And so, you know, there will eventually be competitors. And I'm not trying to say that everything about the SpaceX model is good, nor did NASA make personally all those choices, but, you know, it can work when you have a process that really does truly iterate.
MATT: Which you don't get with government bureaucracy.
MIKE: Right. But then again, NASA did get us to the moon. If you put the right people in charge and get it going...and I think that's what I'd like to finish up the discussion today on, and I think Will was pushing on this a little before. How do you get there?
Now, I suggested before that you have to have some level of trust between management and engineering. If you don't have anybody that'll say, "Okay, go build some stuff," then it's not going to work. Because then you're going to be told what to build; you're going to be told the timeline that you're going to build it in, and all those constraints are going to handcuff you to the point you don't get the flexibility that you want to actually be agile. You know, you're going to be agile on paper because you're going to have those ceremonies, but you're not going to be agile in practice.
So, I think it starts with that level of trust. You have to have leadership that's willing to say, "Hey, engineers, go build some stuff. We're going to give you room to build stuff, and we're going to support you on that. We do expect you to actually be working on things and working on deliverables. We're going to be working with you. But, you know, feel free to go through the process that it requires, because we understand that it requires that."
And that's a challenge, right? It happens anytime you tell a creative person to go build something. You know, you go tell your Renaissance artist, "Okay, go paint me a ceiling." There is a level of trust there that I don't know what I'm going to get. I hope it's good. And it can work out really well if you're willing to do it. Get the right people; let them do their thing. But you have to have trust.
So, I don't think that's everything, but I think it's a start. I mean, what do y'all think?
MATT: Trust is imperative.
VIVIAN: I mean, we've talked about it on numerous previous podcasts, but, Mike, you like to always go back to the psychological safety. And, normally, when it's discussed, it's kind of discussed from a top-down perspective where the subordinates, like, just the engineers actually doing the work, need to feel a sense of psychological safety in the work that they're doing.
But on the flip side, do the managers not also need to feel a sense of psychological safety about the confidence in the ability of the engineers and the work of the engineers to actually be effective for the business or the organization? Like, does it not go two ways in order to build that trust? And if it goes that two ways, how do you build, like, managerial trust as an engineer?
WILL: Yeah, I don't know. Like, the things that people will just be, like, they'll just take my word for like, "Okay, boss, I got it. Peace," shocking to me. You'll find out more on that later. But my gosh, I don't know, man, I mean, hmm, I don't know. I have to defer this one. Like, how do you know you're in good hands?
MATT: It's called communication. It's the key to everything.
WILL: Well, sure.
MATT: If you're communicating...it's easy to build trust if you can communicate with the people that you want that trust from. If I'm working on something, I instill trust by informing, being honest and transparent: "Hey, this may or may not work, but this is what I'm doing. And if it doesn't work, then we'll iterate, and we'll learn, and we'll fix those mistakes, and we'll move forward. But we will move," right? That's how you build trust.
And one of the people on this call has reported to me in the past; I feel like I put a lot of trust there. And I just trust the people who have reported to me to get things done. And as someone who hires those people and builds that team, that's on me to be able to do, right? And if I'm hiring the wrong people, then that's me at fault, not the people I'm hiring. That's my failure.
WILL: I always tell my boss it's his fault for hiring me [laughter]. You did this. You did this to yourself. You deserve it.
MIKE: I'm thinking about dating, you know, kind of the quintessential I don't know you, and I'm going to get to know you over time, build that trust. You know, people who are forming a relationship, you know, they meet; they give each other opportunities to meet their needs and see if it works out. Does it work? Oh, it did, so maybe I'm going to give them a little bit bigger chance. And over time, after giving somebody lots of chances, they keep on following through, you believe, "Hey, this is somebody who's consistently going to meet my needs, and I can trust them to do what I need them to do."
And I think that that kind of pattern applies everywhere. I'm going to give you this job. You know, I just hired you. I'm going to give you this task, and it's probably a small task. I'm going to see if you come back in two or three days with it done. Oh, you did? Great. You accomplished something and I, you know, I have a problem solved. Now I'm going to give you a bigger task. If you keep on delivering on those tasks, then you get more and more trust. And I think that you have to develop that over time by following through, like, by bringing the receipts. You know like, "Hey, here, I built this thing, and it works. It's making you money."
And I think that you have to be...Surprisingly, you can get away from that as an engineer and think, "Oh, I want to build my beautiful castle," and not be pragmatic and say, "I want to make my company money," because they're not paying us to build a beautiful castle. I mean, we can try to build a beautiful castle as we go, because then it'll resist the invaders over time. But you better start with some hovel that will let you sleep through the night.
THOMAS: Yeah, I was actually kinda going to say something similar to that, but I feel like trust in itself is a currency, right? With that trust, you get, you know, if you can build progress from that trust and then you get momentum because of that progress, then you get success. And I think the more success, the more progress you can show, the more trust that's going to be added into you, and then you just keep amassing the more trust, the more trust, the more trust to take on bigger projects and produce a more successful project.
VIVIAN: An iterative agile approach seems to require an initial investment of trust in the engineers to be capable of starting, of doing that work. And then from there, that cycle of trust can continue to build and grow until there's enough trust that you can continue that approach without a ton of concern—that you can know that the engineers that you've hired, you know that they're capable, so you can trust them to do the work.
With a waterfall approach, there are times where it might be necessary because, again, there are hard deliverables that you can't really iterate on. But, otherwise, it seems to come down to a lack of trust. There is a lack of trust in the engineers to be able to iterate and continue to actually build progressively better work. And so, you try to avoid trusting your engineers by laying out all of the requirements beforehand with this idea that your engineers will just be able to do it because there's this plan that's laid out, and you can just...They either hit the plan or they don't, and regardless, you have this plan that you've laid out, and so you've done your work. You don't trust your engineers, and so you lay out a plan so that it's impossible to fail.
And this feels very similar to how a lot of AI development is kind of being pushed. You want to not actually write the code, just write the specs for the code, and then have the agent run until all of the specs are satisfied. And that can work, but, again, you're trying to essentially motivate sometimes the wrong problem because sometimes, in the process of writing the specs, you miss portions of what actually needs to be solved. And so, having more trust between managers and engineers and being able to create that iterative business process, it requires a higher level of trust, but it can be more effective, as we've talked about overall in this call.
WILL: You know, as I think about it, on one level, you know, okay, we talk about, like, sort of, like, the agile planning and, like, you know, like, agile planning versus waterfall and stuff like that. But, I mean, I really...One of the cool things about agile is you really don't need, like, company-wide buy-in. You can just do it. Like, your team can just do it. Like, okay, well, I have, like, I have this waterfall plan. Who knows what's going to happen with it?
And just, like, within my team, we can all just be agile and say, "Okay, I've got a two-week sprint. This is what we're going to get done in this two-week sprint. This is the MVP that we're going to get done in this two-week sprint. We're just going to get this...We're going to get this thing done, and we're going to get a demo up," and then, you know, the next two weeks; and the next two weeks; and the next two weeks; and the next two weeks of that. And you can just do it, you know?
MIKE: How else would you do it without taking the initiative and accountability, "Hey, I'm going to do this"? If somebody's going to tell you to do it, maybe you're not agile [laughs], right?
JORDAN: You know, the trust thing from Vivian is a great way to put this, is, like, where do you put your trust—in your developers or in your specs? And, you know, writing good specs is one way of building trust, or at least of forcing trust. But at the end of the day, it's like, if you trust your engineers to make you a great product and to steer you in a good direction and to, you know, do all that, I think that that trust is key to actually being successful. You can be successful the other way, as evidenced by NASA and other examples and things like that.
But I think it was Eddy that put the cartoon in the chat that had the poo at the end or the poo in the middle, but [laughter]...
EDDY: I can't take credit for that. That was Kyle. As much as I want to [laughter].
JUSTIN: Like, both of those actually, I think, are very indicative of, you know, of what can happen. But on the second one with the agile, if you are, like, encountering poop as you go along, hopefully, you can change direction where there is no poop. So [laughs]...and I think that is, you know, that could be, you know, a good guide for life.
EDDY: Or you just live with the poop [laughs]. You live with the poop, and you move on [laughter].
MIKE: You minimize the poop by steering in a tighter feedback loop.
MATT: You know, there's something to be said about that, though, and it goes back to that building castles. And Mike and I discussed this yesterday on a call we were on. There has to be some progress over perfection with things. And when you're building software for a business that needs to generate revenue, it needs to progress. And sometimes we don't get to build things perfectly like we would like to, but we build things that work. And if we're building those things scalably, then they will get some of that perfection over time. But, usually, especially in large enterprises, you end up with a lot of tech debt, you know? And it's things like code smells. But if those code smells aren't harming you, then you leave them be for now. And when you find on high-traffic times of the year that all of a sudden your system's coming down, you know, maybe they should've been addressed.
But you have to find that balance, and there's always going to be a little bit of debt when you're trying to build software to generate revenue for customers.
MIKE: I have a friend who has a dairy farm. If you have a dairy farm, there's going to be poop [laughs]. It's part of the business. But you can make the most of it. They've got a whole set of equipment to wash it down into, like, a reservoir that ferments for a while [laughs], and they spread it on the fields, and it makes everything healthy, and it all works out [laughs].
EDDY: Yeah, but wouldn't you eventually just acclimate to that smell, and then [crosstalk 58:58]
WILL: The pile, the pile doesn't go down.
JUSTIN: I am waiting to see how you relate this to, like, software development, so bring it home, Mike [laughs].
MIKE: The end result is, yeah, there's going to be poop in your software development. It's going to smell. There's going to be smells on the right; there's going to be smells on the left; there's going to be smells. And, you know, where those cows are feeding, it's going to stink. But you can develop some systems to --
EDDY: Who are the cows? Are we the cows?
WILL: [crosstalk 59:33] cow.
JUSTIN: Let him finish. I want to hear this [laughter].
WILL: I make the milk, which is the money [laughter]. Are the users the cows? Maybe the users are the cows. I like that better [laughter].They make money. I milk the cow.
MATT: Well, we're the ones producing, so I think we're the cows [laughter] [crosstalk 59:53]
WILL: I don't feel like a cow anymore. I feel like a farmer.
MIKE: The whole enterprise involves some degree of poop [laughter]. And sometimes you're going to step in it, and it's going to stink, but you can minimize, you can minimize the smell. And sometimes you can even use some of the bad parts to your advantage. I don't think you're ever going to get...
MATT: Sometimes you're the one leaving it.
MIKE: Sometimes you are. I've stopped in a cornfield before on my bicycle; I have to admit [laughs]. There's going to be some of that reality, and it's not going to be perfect. There's no such thing as an ivory tower in dairy farming that doesn't have a smell, and I think the same applies to software. But you can still build some amazing things and make a lot of money, and that's okay.
There's a process here that's going to be okay, even if it's not perfect, even if it smells sometimes. The end result is somebody somewhere gets to pour some milk on their cereal, and they'll pay for it. I visited the farm yesterday, so it's on my mind [laughs].
JUSTIN: Dude, I think that's a great way to end it [laughter].
MIKE: And that's what I think. I think that's the landing. There's going to be some poop, and that's okay. You just need to try to minimize it. And you don't want to have a great, big pile at the end that you have to land in. If you develop a process to deal with it as you go, it's almost certainly going to be better.
And with that, until next time on the Acima Development Podcast.
In this episode of the Acima Development Podcast, the team digs into what happens to engineers when the org chart shifts underneath them. The timing is deliberate: Acima recently consolidated teams and realigned around value streams, the structure recommended in Team Topologies. Mike opens with sleep. After five years of falling asleep next to his youngest son, on the floor as often as the bed, he can now drop off anywhere. A habit he had held since infancy turned out to be trainable once he had no choice.
Guest Seema joins from parent company Upbound on the eve of her 21st work anniversary. She traces a career that began in 2005 as a PowerBuilder developer in the servicing department, moved to classic ASP corporate applications when that department closed, pivoted to call center and IVR work during COVID, and landed on the RACPad point of sale team about six years ago. Six or seven office buildings and more reorgs than she can count later, two things have kept her there. One is staying relevant, which for her meant telling leadership she was bored and asking for harder work. The other is the community of coworkers she can call when a deadline like the RACPad Mexico rollout gets tight. She recommends Mel Robbins' The Let Them Theory and Simon Sinek on finding your why, and she starts her mornings by counting small wins.
The rest of the panel pushes on where acceptance ends. Dave argues for judging change by its trade-offs instead of by good and bad, and defends leaders who cut through a stalled debate even when half the team ends up unhappy. Vivian, an intern about to join a new team, describes trust in the people around her as the thing that made two months of constant change workable, and notes that strong engineers tend to hold tightly to one or two things and stay loose about everything else. Jordan brings the counterweight from a startup acquired by a large bank, where the rules could not be changed and the honest options were acceptance or an exit. The point everyone converges on is the why. When leaders explain their reasoning and take feedback seriously, people absorb hard changes. When they skip it, the resentment is about not being heard rather than about the change.
Transcript:
MIKE: Hello and welcome to another episode of the Acima Development Podcast.
A bit of context before I give more of the intro. Acima is part of a parent organization, Upbound. And, you know, we've got people from Upbound that, you know, we have joined with, with lots of experience, which will be relevant today, as we've got Seema, who's joining us from, you know, from Upbound, who'll be joining our conversation today, will be very relevant for the conversation.
But first, I'll give you a bit of an intro. I'm going to talk about sleep. So, you [laughter] ever tried to sleep in someplace different than your own bed [laughs]? And you know how hard that is. You know, it takes, like, days. You know, you go to a hotel, and you never sleep as well, wherever it is you have to sleep. And I think almost everybody has experienced this. It's a pretty universal experience, and something that I've historically had to deal with.
A few years ago...it's getting to be a few more years ago now, probably more than five years ago. I used to be, like, a side sleeper and sometimes even sleep on my belly. And I did a lot of camping with my kids, like, in the backyard especially. And laying on your arm on the hard ground overnight doesn't do good things. And [laughs] I noticed that after a summer, like, wow, I'm getting tingling in my fingers. This isn't a good thing.
So [chuckles], I had to learn to redo my sleep position, which was really, really hard at first because changing something that you've done for decades is just...it's a really hard habit to break, you know, something you've been doing since probably I was a baby, right? And [chuckles] making that change was really hard, and, eventually, I got used to it. I can now sleep on my back, roll over my side a little bit. It works.
And then I've got [chuckles]...my youngest child is a terrible sleeper. He...well, let me rephrase that. He actually sleeps pretty well, but if he's alone, he just doesn't sleep well. So, I usually lay next to him to get him to fall asleep, and I've done that since...he's five. He's five years old. So, I've been...I had to do it so much, I ended up just usually falling asleep next to him, right [chuckles]? And I have fallen asleep on the floor. I have fallen asleep on his bed. I have fallen asleep somewhere in between the floor and the bed [laughs], you know, just about anywhere.
And having dealt with that for the last five years, I'm to the point where he doesn't need to lay on my arm to fall asleep. He's okay with that. You know, I'll read a book to him while he's falling asleep. You know, he loves laying on my arm, but now he'll do without that. And I can usually go somewhere else for a while, but he'll still wake up in the night and say, you know, "Papa," and needs somebody there.
So, I'll tell you, five years of sleeping in random places makes you get really good at dealing with sleeping someplace different, and now I can sleep anywhere [laughs]. I go to a hotel, I just drop right off, and I am fine. Uncomfortable, fine, you know [laughs], with sleeping on the floor, I sleep great. It doesn't matter. If you go through enough of these things, eventually it breaks you, and you're just okay.
And [chuckles] I slept...I was traveling down to our corporate office in Texas about a month ago, and I hate hotel pillows. They always give me a kink in my neck. I got a terrible kink in my neck, but I slept through it anyway. I still got a kink in my neck a month later. Like, if you see me [laughs] wincing a little bit, that's why. But you know what? I still sleep fine. So, apparently, even though it's so hard to undo that, you can train yourself to be able to sleep anyway in different places.
Today, we're going to be talking about dealing with change, particularly org changes, and all the changes that happen in an organization. You know, we like to hunker down and do our engineering work, but sometimes we can't always do that. Recently, we've kind of realigned our team structure to be oriented toward value streams, which is considered best practice in the industry. I mean, this is a pretty standard thing to do.
There's a well-known book, Team Topologies, that goes in all the research about team structure that a lot of us have read. It says we should be doing exactly this. We have the stream-aligned teams, platform teams, enabling teams, complicated subsystem teams, is what they recommend. And, primarily, you should have these stream-aligned teams, and we've been organizing around that. We've been gradually moving this way for years, but, you know, we've really made a significant change where we consolidated some teams recently. So, it seemed timely to talk about change.
Now, Seema has been with Upbound, I think, longer than any of the rest of us here, so she's a perfect person to talk to dealing with these org changes that happen over time. And you've survived, Seema, and you've survived it. And you're still here to tell the story, or more stories, and hopefully some dirt that we could hear [laughs].
So, that's the topic today, and tactically, how do you deal with it? Because it's disruptive, right? Anytime you have change, it's disruptive. It slows you down. It makes things harder, and it's painful; it's awkward. Change is not fun. So, what do you do? That's what we're talking about today.
I'm going to start by opening things up. And do y'all have any initial thoughts, any of you, Seema or anybody else, about, you know, what do you do to deal with change?
SEEMA: Yeah, what a great topic, Mike. But, first of all, thank you so much for having me here; what an honor. And, to be honest, definitely it's my first time to be on Acima Podcast, but it's my first time to be on any podcast.
MIKE: [laughs]
DAVE: Oh, nice.
SEEMA: So, I'm super excited, but a little bit nervous as well, but looking forward to talking to you all for the next 45, what, 50 minutes, right?
As I was mentioning earlier, talking about my name, in all our teams' transcriptions, you know, every time somebody says, "Acima" it shows up as Seema, so maybe it's time that I should change my name, right [laughter]? Well, that was one fun fact.
Another fun fact about me, like, talking about how many years I have been here, Mike, I don't like to talk about this fun fact because I'm a little bit shy [laughs]. I don't want to share this, but tomorrow is my 21st work anniversary with Upbound, so yeah. So, I don't like to share it with people, but then, obviously, Mike has given me this opportunity to be on this podcast, so I wanted to share it with you guys. And I couldn't be more proud to be talking about RAC and changes and various reorgs we, or I, have gone through and still survived, I guess, you know?
So, just to give you guys...can I go ahead and give a little bit of intro about myself, Mike?
MIKE: Please.
SEEMA: Yeah. So, I joined Rent-A-Center on August 8th, back in 2005 as a PowerBuilder developer. I don't know how many of you still remember PowerBuilder. It was a software used to develop the client-server applications. So, I joined as a developer to build applications in our servicing department. But then, I think, two or three years after I joined, they decided to close the servicing department because it was not cost-effective. So, basically, they gave the responsibility for servicing our items to the individual stores.
So, at that time, I moved into our corporate applications department, the team that maintains or builds applications for the corporate departments, like HR, finance, legal, and things like that, right? So, I was...I learned ASP, classic ASP, not even .NET. We were not in .NET [laughter] at that time. So, it was classic ASP.
So, I was doing corporate applications for a couple of years, I think 8 or 10 years. And then when...I think, seven or eight years, around six or seven, when COVID hit, you know, our priorities changed. So, they pulled me in, you know, our CPO at that time, they pulled me in to help. I don't know how many of you remember; we used to have call center in Atlanta for the...except [inaudible 08:14].
So, because of the COVID, you know, our stores physically they were closed, right? But we still needed to serve our stores, our customers. So, they wanted to build an IVR solution, right, so that our customers could call 1-800 number and they get information on their accounts, right, how much amount due and things like that. So, that was something new for me because I had never worked with, you know, call center software. So, I did that.
And then we also implemented auto dialer because we wanted to gain efficiencies the way we kind of make outbound calls for collections or account management. So, I did that during COVID for two years. I think we were mainly focused on call center improvements or efficiencies.
And then after that, I came back to supporting the corporate applications, you know. But then five years or six years back, you know, the things in the corporate department or corporate applications world, for me, it was kind of getting stagnant. It was kind of slow. I was not feeling challenged.
And my daughters, I have twin girls; they are 23 now. So, they went to college at that time, like, six years back, 2021. So, I had kind of, like, to be honest, I felt a little bit depressed because I have twin girls. They went out of the house at the same time, so I was feeling a little bit lost. I had extra time, so I wanted to get more responsibility, wanted to do some more challenging work.
And at that time, you know, this opportunity to work into the point of sale system, RACPad, came. So, definitely, I mean, I don't know how many of you know me, but my passion is serving our customers, customer centricity. I mean, I love working on...I mean, directly or indirectly, all of us, obviously, we serve our customers, our coworkers, right? But working on RACpad as POS it allowed me to impact our coworkers day in, day out, like, directly because, you know, POS system, as we all know, it's so critical, right?
So, I mean, fast-forward, I am so proud to have taken that opportunity. Last five or six years have been the most rewarding time of my time here at Upbound, and I'm super proud of all the work that we have done as a team. And during this time, definitely, I mean, last 20 years, we have gone through several reorgs. I can't even remember how many buildings we changed. I think six or seven [laughter] buildings, physical buildings, we have changed.
DAVE: Oh wow.
SEEMA: Yeah, multiple reorgs. So, talking about, you know, what still kept me going and why I'm still here, I think, two reasons, two main reasons, right? And, I think, definitely after that we can talk about change and get everybody's perspective as well. But to answer your question that what kept me going and why I'm still here, first of all, to me, staying relevant is most important. And, fortunately, I always had the leaders who supported me, who coached me, and who allowed me to stay relevant.
Like, the example that I gave you guys, right, where I didn't feel that, you know, the work I was doing it was challenging enough, so I spoke to the leadership, and they allowed me...they gave me this opportunity to move into building a...into, you know, a team that was building a new POS system. And that was entirely new for me, new technology, AWS, new functional domain, everything new. It was basically like joining a new team, but, you know, with everybody's support because they believed in me, and then definitely I was motivated to learn and serve our customers. So, to me, staying relevant is most important, and, luckily, I was able to stay relevant.
And second definitely is your coworkers, right? Like, yeah, I...when you are at a place working for so long, you develop those relationships, you know, strong working relationships. And the company, the place, it becomes your home away from home. It becomes your safe place, right? So, definitely, I mean, you know, staying relevant and building strong relationships. I have many friends at work who I can reach out to any time. So, that's obviously...those are the two main things that kept me going through these reorgs and why I'm still here.
MIKE: Well, you know, you're the, like, the face of RacPad now, right [laughs]? It's your ba...[laughter] Like, everybody knows that's your thing, and you've seen that project through and been willing, so kudos.
SEEMA: Thank you.
MIKE: So, that's a major undertaking for the business. You know, that drives all of, you know, like, kind of, like, everything that happens over that line of business, so it's a big deal.
SEEMA: Thank you.
MIKE: And you talked about...well, yeah, of course, it's just the truth. But [chuckles], you know, you talked about a couple of things, you know, being willing to make those...well, maybe more than a couple, but, you know, being willing to be stretched. You know, not seeing that change as a negative, but as a challenge, as an opportunity. And you talked about getting the support from the people around you, and then the informal support of the people who you just like working with is what got you through. Seems like we always come down to the people, and that's what you latched onto.
SEEMA: Yeah. Just to add to your last point, as there is a saying that, you know, it always takes a village to raise a child, right? So, I believe in that. Definitely, like I said, you know, I have twin girls. We had to get a lot of help raising them. But I feel same applies in our professional life as well. You know, we have to have that village or community, right? And that community can be your peers, your mentors, your leadership, your fellow coworkers, and people who are reporting to you, your own team.
You have to have that community, you know, who you can reach out for any type of emotional support in tough times, like, tough times meaning when you are going through change, reorg, or even a tight timeline, right? I mean, I can tell you guys from my experience that last year when we were under aggressive timelines for RacPad Mexico rollout, I mean, definitely I had to reach out to people just to sometimes vent out, you know, and sometimes to seek help, you know, or seek advice.
So, definitely people having those friendly relationships and just being kind to each other and supportive, that means a lot because when we are going through change or reorg, I mean, we don't realize sometimes everybody is going through their own kind of battle. Some people can share; some people they do not. And, you know, but we have to be really kind and supportive and, you know, just lean on each other basically.
MIKE: Oh, that's fantastic. Thank you. So, we've got a few of us here. Oh, you know, I didn't introduce the whole group. I talked about Seema.
SEEMA: Oh yeah.
MIKE: We've also got Eddy, Kyle, Dave, Vivian, Thomas, Ramses, and Justin with us today. And what do you all think? What are your thoughts, either triggered by what Seema is saying, or your thoughts about dealing with change in general? So, Seema is really focused on that village that you use to get through. What's your experience?
DAVE: Love that. I love that. The thing that is rattling around my brain, you were talking about sleeping in weird places. And this is going to be...I'll throw a stake way out far, and then I promise I'll connect it.
MIKE: [laughs]
DAVE: Which is you guys will always hear me talking about what is the trade-off? I very rarely say, "This is good," or "This is bad." I want to know what the trade-off is. What are we getting out of it? All the time. Never say, "Always do this feature," or "Never do software this way." It's like, what do you get out of it? There's no such thing as a side effect. There's only effects. What do you want? Okay, where am I going with this?
My dad grew up in Lehi, just a block away from where the Thai House restaurant is—for those of you who know that—on Main Street, about 100 yards from the train tracks. And freight trains, cargo trains would go through every hour all night long. And, like, it's...at that distance, you know, 40 bazillion tons of coal being hauled up to the refineries, it's like a Richter 7 earthquake just every single time. Like, things would rattle off the walls. And the thing that blew me away was my dad telling me...I'm, like, "How did you get to sleep?" And he says, "Eh, you'd be surprised what you can get used to." And then he cocked his head, and he said, "Actually, I would wake up if the train was late." Which literally what he had gotten used to was the house is supposed to be shaking right now.
Now, you can say the house rattling around keeping you from sleep, that's bad. You shouldn't have that. You shouldn't have to...well, why is it bad? It's like, industry is happening; civilization is happening at scale. It has an effect right here. And my dad just adapted by, "Well, okay fine," right?
So, then we fast-forward to, like, organizational change, and somebody comes in and says, "Oh, we're going to do Scrum," you know, for, like, the seventh time in three years or whatever. It's just [chuckles] like, "Oh, no, da, da, da," right [chuckles]? "Oh, we're going to do this, or we're going to try this, or we're going to do this other thing." And sometimes you're like, oh, this is dumb, and I don't want to do this, or, you know, this software is ridiculous, or the way this is built is wrong. Okay, but what's it buying us? What is the trade-off, right? And some of the trade-off is faith. It's like, we're hoping that this will fix some things that we've got, but we definitely are addressing some concerns.
We've had some organizational changes here at Acima based on some budgeting problems that we had. And you can tell that some of it is people who their primary motivation is let's never have a budgeting problem again, right? It's we just don't want that to happen. Unfortunately, the engineering team, we're now a little bit overscheduled and a little bit...you know, it's because of that, right? It's like, we're over-monitoring. Okay, fine. But it keeps the company afloat. It keeps us in business, and I like having a job. I love it when my paycheck clears every Friday.
So, yeah, that's kind of where I'm going with that: what is the change? What are the effects? If we can step back from good and evil or from this is bad, or this is wonderful, you can kind of step back from fearing change to saying, "Okay, I have curiosity and wonder about what this change is going to bring. Tell me what it is."
I watched Scrum burn out in 6 different organizations over 10 years. And at the job prior to this one, they announced Scrum, and they're like, "What does Scrum mean to us?" And I say, "It means we're losing an entire day to meetings every sprint." And they're like, "No, no, we're not." And then about three months later [laughter], they're like, "Dave, you called it." And I'm like, "Yeah, okay."
But then they shipped us off to Scrum training. And they actually said, "This is why it all connects." And I'm not a stand for Scrum. I don't get paid by them at all, and I don't particularly like it as a particular practice. But the basic principles of how things connect together, Scrum just...it codifies it and puts some ceremony around it to deliver those things.
If you worship the ceremony, you're going to end up with the same problems that Scrum has always had. But if you pursue the principles behind it...and that's where the wonder and the curiosity gets you. What is this change bringing us, and how do we get there from here?
MIKE: I actually wanted to latch onto something you said. I love what you said about stepping back, being kind of a dispassionate observer, like, oh, you know, what can I learn from this? There's always something to learn. But also, you know, like, what's the good? What's the bad here? Or, you know, what can I get from this, and how can we make…
Like you say, if you go back to the principle, if you go back to the principle, maybe there's something good here. Well, that goes a long way also in collaborating with other people, people who are going to come in and introduce change in your organization: whether it's the new person, whether it's an intern coming in and changing your daily schedule, or, you know, a new executive coming in and pushing some sweeping change.
And the knee-jerk reaction, "Well, this is stupid," gets you nothing [laughs]. You maybe say your frustration, and you're still frustrated. But if you take that kind of attitude...I've been thinking a lot about this lately, you know, what's the intent here? And how can I take my experience and what I know about how things work and take their intent and try to come up with a compromise? How can I bridge this? Because then it's an engineering problem.
You know, how can I solve this? How can I work this out with this person who's presumably acting in good faith? Because I think usually people are. You know, they're trying to make things right, trying to make things better. How can I collaborate with them and find shared common ground? Because we're both trying to make the company stay afloat, right? And succeed, even grow. How can we align here?
And, usually, if you understand where they're coming from, you can talk about...it's like, "Oh, I know that you want to have this be more customer-centric, so that's why you're introducing these changes. I like that, too. That's something that really matters to me. And I see that, you know, you're wanting to, you know, do policy X. You're wanting to change the way that we're doing sprint ceremonies. Okay, why are you wanting that?" And if they can give me the why, then we can usually find common ground, and maybe I'm not even going to agree, right?
There's probably a lot of things I'm not going to agree with. I've had kind of a similar experience with Scrum. Like, yeah, there's some good parts and some less good parts, but I know that people are trying to make it work. And, usually, if you're willing to act in good faith and say, "Well, I'll play with you. I'll do this," then people will reciprocate, and you can come to a good spot of common ground. And that helps a lot rather than just living with the frustration and resentment.
VIVIAN: I'll throw my 2 cents in there, since we're talking about change. As an intern, I came in kind of in a place where the entire system that I was going to come into was change. There was no established baseline. There was no, "I'm used to this, and this is how I like doing things." It is, "I'm going to walk in on the first day. I'm going to get introduced to my mentor. He's going to explain to me what we're going to be doing this summer, and then I'm just going to do it." Like, on one hand, it is an immense amount of change, and on the other hand, there was nothing holding me to my old structure before that.
And so, it was very much a situation where I was just learning by asking constant questions and figuring out why other people do the things that they do. And that is one of those things that immediately started to build my trust in the people around me and in the organization in general to allow me to understand the reasons behind each of the things.
As I kind of gained more experience and interacted with people that had both frustrations and things that they liked about how the organization runs, I see the decisions that are being made. I kind of start to understand why the decisions are being made, which builds a lot of trust for me. But now as I've gotten, like, a little more settled, and I'm coming back on Monday to, like, keep working and joining a new team, I'm terrified of the change.
Because even though it's only been two months that I've been here, now that I'm starting a new position and everything is changing, like, all of the things that I've learned and, like, gotten used to interacting with my mentor, so on and so forth, are suddenly shifting and needing to basically go in trying to be that newborn baby that doesn't know anything, but also have the experience to be able to apply that into my job has been a real mental challenge.
But it is helped a lot by the trust that I have in the people around me, knowing that they have, like, Mike, what you were saying, the best intentions in mind, and are always trying to do what is, at the very least, the best for the organization, and I believe, usually, the best for me as well.
MIKE: Yeah. One thing we didn't tell you, Vivian, is on your first day full time, we send you to spend your first three months to work down in the mines below the building.
VIVIAN: Oh okay. I'll [inaudible 23:39] yeah.
DAVE: There's a thing called bait and switch. You're going to want to Google [laughter] that [laughter].
JORDAN: The young ones yearn for the mines [laughter].
DAVE: Yes, they do [laughs]. But Mike made the comment about, like, if you dig your heels in, nothing good comes from that. And I'm going to resist that even just a little teeny, tiny bit. We've watched --
MIKE: Dig your heels in?
DAVE: Yeah, I'm going to dig my heels a little bit. No, there is value in skepticism, and if you've got an organization that's going...I'm dangerously close to saying collaboration isn't always great [laughs]. Although Eddy can tell you, I did hang up on the bullpen call today because I wasn't getting my work done, and I was very frustrated. I was like, "You guys are getting me tied up. I got to..." So, I hung up on them. But we had some decisions that were paralyzing the merchant portal team in architecture.
And, Mike, when you stepped up and Andy came in to be our manager, he kind of threw his weight around a little bit, and he just like, "No, we're just doing this." And I was like, "But wait, wait, wait, wait." And the thing is, is that a land war that had been going on the team for two years just ended in that moment, and half of us weren't happy. But half of us were going to be unhappy no matter what. And so, slicing that off very, very quickly was powerfully useful.
I tell the kids that come to skills clinic that what is the value...You guys can't see it on the...people won't be able to see this, but on camera, I've got a thing that I literally etched into Purpleheart and lacquered it. And it just says, "Are you mad at AI? Why though?" Because when you crash out at the AI, you're just telling yourself about yourself, right? You're getting mad at a machine.
And I've since come to answer that question. If you get mad at the AI, it lowers its temperature. And it makes it a little less flighty, a little more conservative, a little bit more risk-averse. By the same token, if you tell, you know, Claude or Gemini, "You're doing great; I love this. More of this," it will raise the temperature, and it will get more creative, and it will branch out more and go look for more things.
I'm not saying you should weaponize this against other people, or even particularly against AI, but even the concept of, like, [inaudible 25:46] scale versus should I dig in, like, there's so much clarity you can get by digging in as long as you're right before you made the decision. If you're trying to explore and find the decision, then yeah, holding that creativity and wonder and acceptance is key.
MIKE: Just to call out, I think there's times that it's important to stand your ground.
DAVE: Yeah, absolutely. Absolutely.
MIKE: I also think that at every moment you are digging in your heels and fighting whatever comes along, you're not accomplishing things. You're just a, you know, a professional --
DAVE: Jerk [laughs]?
VIVIAN: Pain in the ass?
MIKE: A professional roadblock, right?
DAVE: A professional roadblock, yeah.
MIKE: And that's not a way to run your career. You choose the things that matter most. And there are things that are worth...you know, I've left a company before because of choices they were making, and with, you know, several people on my team because, no, not going to do that. And so, there are things worth taking a stand for. And whether or not, you know, the specific structure of your sprint ceremonies, I just don't think that that's the place to typically stand your ground. You know what? I can't [inaudible 26:54] do that.
DAVE: Yeah. That's not the hill I want to die on.
MIKE: Exactly.
SEEMA: Talking about the change, I know change is never easy. I have always resisted change, to be honest.
MIKE: [laughs]
SEEMA: It's, you know, as brain, obviously, we get used to the habits, right? I mean, that feels like a safe place, but change is here to stay, I mean, we all know that, right? And these days, I think, change is becoming even more frequent and harder. I mean, we see our friends, our coworkers, you know, getting impacted. So, it's not easy, but, like, a few things that I think I have learned recently to keep myself sane and to keep myself moving forward, you know, a couple of things, like, for example, keeping an open mind, right?
I have recently been reading this book. I think maybe you all might have already read it. It's The Let Them Theory by Mel Robbins.
DAVE: Oh, I love that book.
SEEMA: Yeah. So, basically...yeah. So, she says that, you know, let them, you know, let them take care of the things that you cannot control. Do not waste your energy fighting or overthinking or worrying about it. You have to find things that are under your control, you know? And once you are done thinking let them, you have to think about let me. Let me focus on things that I can control, right? What can I do to add value and help move the needle for the business while things are still settling in during these times, while we are going through reorgs or changes? Because change is here to stay. It doesn't matter how much we resist, right?
So, this is a great book. I mean, it doesn't matter, it's professional life or personal life, The Let Them Theory by Mel Robbins. So, like I said, even I resist change, but you know, this book, I have learned a lot from this one.
And then there is another, you know, I think most of you might already know Simon Sinek, right? He is a motivational speaker. I like to listen to him. And then I really like the way he talks about, like, all of us, we have to find our why, right? So, if we find what keeps us, what makes us happy, what keeps us moving forward, what motivates us, and we focus on that thing, it will automatically bring positivity in our life. We'll start feeling that sense of ownership and belonging. And once we do that, it will shift our focus from worrying to feeling accomplished.
So, those are the two things that I have been, you know, kind of trying to practice these days. And there is one last thing that I really like, and then that's, again, by Mel Robbins. That's basically, you know, practicing gratitude. Be more thankful. So, she says that we should start our day, like, in the morning, you know, just kind of thinking about few things that we have accomplished recently, right? Small wins.
Don't focus on big wins. Don't wait for any big projects or, you know, any big wins that, hey, then I'll be happy. But, like, try to find happiness in the small, small wins. And I think that has helped me tremendously as well. Like, I start my day saying, "Okay, I'm the best. I can do it. I have done it," you know? So, that kind of, you know, wires your brain to start thinking positive.
So, those are a couple of things that has really kind of helped me stay positive and kind of accept things as we are going through these changes and kind of, you know, not worrying about things that are outside of our control.
MIKE: I love what you said about gratitude. We should maybe dig into that a little bit. One other thing I wanted to catch before we move on, though. You don't think that technology is slowing down [laughs]?
SEEMA: It's not slowing down. It's changing rapidly.
MIKE: Exactly [laughs].
SEEMA: I mean, talking about AI. Yeah. So, it's --
MIKE: I...that was a [inaudible 30:59][laughs]
SEEMA: Yeah, yeah. Exactly. I mean, things are changing so fast, I mean, especially in the age of AI. I mean, you go to LinkedIn, and you see everybody posting about what they're doing, new tools every single day. And that's why sometimes you feel overwhelmed, right? That's why, I think, it becomes even more important that we should look at celebrating, like, small wins in the day-to-day life, right? You know, what did you...did you do something to help your fellow coworker, help a customer, you know? And celebrate that, I mean, small wins, right? Because, obviously, nobody is perfect. Like, we all have things to learn, but, I guess, [inaudible 31:37] from each other and be patient. We can do that.
VIVIAN: I have a couple of things. First, I'd like to push back. I do think that I'm perfect, but that's just me.
SEEMA: [laughs]
VIVIAN: I know the rest of you guys can't live up to that standard, but...
SEEMA: [laughs] No, I'm nearly perfect as well, Vivian.
VIVIAN: Very nice. Very nice. [crosstalk 31:56] I'm glad to have someone else on our level.
SEEMA: Awesome.
DAVE: Wow. She's changed now that she's got her offer letter [laughter].
VIVIAN: Yeah, this is the new me [laughs].
DAVE: I'm going to go home and Google bait and switch [laughter].
VIVIAN: As you should. Yeah, it's a new concept. I really loved what you were saying about...what you and Mike were saying about gratitude. And I just...I wanted to point out something that I've noticed about especially, like, really good workers, and especially engineers, are often extremely, extremely opinionated about a very few specific things.
They have one to three things that they really hold, like, near and dear to their heart. Maybe it's the style of programming that they like. Maybe it's a specific software that they really, really like to use, and they invest all their time in. Maybe it's using split keyboards and ergonomic mice. I don't know what it is, but all of these, like, super incredible engineers and super incredible workers have these little kind of almost pet projects that they attach themselves to.
And, at least from my observation, when they are capable of holding onto these things and letting the rest of the things be things that they just discuss and go where other people want to guide things, they're able to fit into different organizations and work well with others, but still hold onto the skills and capabilities that make them great at what they do.
It's as soon as these engineers start to hold onto everything, like you were talking about, they become roadblocks, and they don't do these things. But if they don't have these little pieces that they hold onto, like, they're Vim user, and they cannot ever do anything that doesn't have Vim key bindings, as long as they have that to hold onto, they get to experience the joy of using the things that they love. And, like you were saying, they get to, like, almost be grateful of the things that they love doing, and maybe that's writing code by hand, or maybe it's prompting an AI, or maybe it's using Vim motions in whatever application they use. But that kind of allows them to continue to upskill while being adaptable because they have a thing to continuously hold onto and be grateful for.
DAVE: Okay, she can stay [laughter].
MIKE: The minds will humble her anyway [laughs].
VIVIAN: I, especially going through my internship, but as I'm, like, just kind of launching my career, despite my enormous ego, I'm really trying to learn as much as I can from all of, like, the incredible engineers around me, and take, like, bits and pieces of each of what makes them great at what they do so that I can continue to get better as an engineer.
So, it's just been something that has been very on my mind, and I feel has been very helpful as I've needed to adapt to a lot of change because it allows me to kind of see the bits and pieces that make other people great. And so, I can trust that their bits and pieces are valuable, if that makes sense.
JORDAN: So, I do have a comment on corporate change. One company I was working for, after you guys, got acquired by a large bank. And this large bank had a lot of rules and procedures that they dictated that we follow. And it was very frustrating at times because it was, like, all of these rules and procedures probably don't apply to this smaller company that was a startup. And the startup was, you know, successful. That's one of the reasons why they got bought out. But this larger bank had many, many arcane rules around, you know, software engineering that made it difficult, well, software engineering and specifically cybersecurity, that made it difficult to, like, progress, and it was very, very slow.
And, you know, dealing with that kind of change was kind of frustrating. Realistically, we just had to accept it or leave the company. It was nice in some ways because we were paid well [laughs]. But it was bad because it was, like, you know, trying to make change at the larger corporate level just couldn't happen.
And so, what we had to do, or what I had to do, was dive into, like, the reasons why all of these little things, you know, why they existed, and I still disliked a lot of them. But some of them I got to the point where I was like, "Oh, okay, I can see why that's useful." And you really had to dig in to find out what the cause is and do the research.
And also, if they come in with the attitude, they being the corporate overlords, come in with the attitude of, "You will do this," and not give a reason why, that makes a lot of difference in terms of, like, how the people receiving it, the message that they receive. So, if you ever find yourself in the position of, you know, having to dictate to people the way things should be, if you aren't sympathizing with them or, you know, if you aren't looking at it from the way that they see it, you're not going to have a happy workforce, so...
MIKE: I couldn't agree more. You're reminding me of a meeting I was in. Somebody was introducing change and didn't give any reason why, just saying, "This is what we're going to do." And, actually, they gave a little bit of reason why, and the other people in the meeting gave really strong explanations about why that explanation didn't hold water at all. It wasn't applicable, and none of the answers landed. It was just like, "I'm not hearing it. This is how it's going to be." And I have never seen people get so upset [laughs]. I think it's the worst meeting I've been in all my life. The people were so, so upset. Same topic --
DAVE: Because of the change or because they weren't being listened to?
MIKE: Because they weren't being listened to. The exact same thing with the reason why and with listening was presented a week later, and there was basically no objection at all. Because people were told, "This is why we're doing this. And it's going to be a little frustrating. You know, there's going to be some stuff we're going to have to work through here, but this is why we're doing it. Please let us know what you think." And it landed fine. That why makes such a difference. And, you know, the first meeting was given with good intent, right? Just the delivery didn't land because people didn't hear the why. People weren't being listened to. It makes all of the difference.
SEEMA: Yeah, I mean, that's a great point, Mike, and, I think, in all the, you know, reorgs or changes, like, David mentioned the Scrum, I mean, you know, as an example, but, I think, change management is most important. That's the key. You know, the top leadership, they have to provide you that clarity, and there has to be a way, a process established, so that everybody can provide their feedback.
Definitely, I mean, some of it, you know, might not be, you know, worth taking action, but, you know, there has to be a way that you should be able to provide your feedback to the top leadership, and they should listen to you. That's the key. That's what I feel like.
JORDAN: Yeah. In my experience, you could usually, like...I've been in a bunch of different companies where a new director comes in to a group and dictates, "This is the way it's going to be. I'm coming in and making these changes, and you guys just trust me, bro [laughs]," or something to that effect. Or not even "Trust me, bro," he's just, like, you know, they would just come in and make these changes. And we would, you know, come back from those types of meetings and look at each other and be like, "Okay, how long do you think this director's going to last [laughter]?" Usually, it was, like, less than a year with the directors that came in with, you know, with that type of attitude.
MIKE: I think, to Dave's point earlier, decisive is fine. Decisive can be great if you are listening and willing to explain your justifications. And sometimes the justification is, you know what? You're both right. There's really not a perfect answer here. We just need to pick a side, and so I'm going to pick this side here, and we're going to run with it. And people are okay with that [chuckles] because they understand that.
You know, we get it; we've got to pick a side. There is no perfect answer. It's fine, and we'll be fine. And if, you know, a year from now we decide, you know what? We learned from this, and the other side was better, we can make a change. Like, the decisiveness is fine if you're open to that feedback, and you are willing to explain your justifications and be a human being, engage with people, be vulnerable.
DAVE: The thing that, I think, is interesting about it is Mark Twain famously said, "It has been my experience that people who are brutally honest tend to enjoy the brutality as much as the honesty." So, I hold that in one hand. These are the people that dig in or just, you know, they get off on the power trip, right? That's where they're getting their itch scratched.
On the other hand, I tried to Google who it was that untied the Gordian knot. I didn't realize it was a real person. It was Alexander the Great. There was an oracle who said that whoever could untie this really complicated knot tied to an ox cart would rule all of Asia, and Alexander walked in, drew his sword, and sliced the knot in half. I don't know if that's a true story or if it's apocryphal, but I love it, and you can see there's utility. There's time for swift, sharp action, and there's time for...you don't have to be a jerk about it, right? It's, don't be nice; be kind, I think is a good watchword there.
VIVIAN: Okay, I'm curious. I mean, basically, all of the people that I'm talking to have experienced a lot more organizational change than I have. So, how, as someone who may have change, like, forced upon them, essentially, do you adapt and deal with those different leadership styles?
Because as much as I can say we all want to follow this perfect advice that Seema you've given about, like, essentially understanding, and listening, and all of these things, not every leader is like that. So, if you encounter a leader that isn't or a leader that is, how do you adapt to that change, and how do you approach things differently? Is that where you start to stick your heels in the ground, and if things aren't going how you feel they should go you leave? Or, like, what is kind of the calculus there?
DAVE: I generally go by the rule of once you get above about two levels up on the org chart, it is easier for an executive to solve me than it is for them to solve the problems I'm having [laughter]. So, if I can take it to my team lead or if I can take it up to...like, I'll go as high as Andy. But if I'm banging on the door of somebody at Upbound saying, "I need an accommodation," or, "I need this," or, "I need you to solve this problem," and it's something that should have been solved kind of at the ground level, but they've dug...
It's like, I like my paycheck clearing, and that's...if we have a why, we can bear with almost any how. That's a nihilistic answer, which is great because that quote is actually from Nietzsche. But at the same time, embrace what you can change. Just because somebody's three levels up the org chart doesn't mean you can't persuade and get their ear.
Balaji, our new CTO, is a breath of wonderful fresh air. He's listening all the way up and down. It's absolutely delightful to have him on board and to feel like, "Hey, I could come up to this person and say, 'Hey, what about this?'" I really like that. Might not get my way, but I feel like I would be heard, and I like that.
SEEMA: Yeah. And to add to what David mentioned, Vivian, to answer your question, I mean, Mike, Balaji, and everybody in the leadership, they have scheduled, you know, as an example here, the change that we are going through, right? We have had several meetings where we got to ask the questions, you know, for clarity, for transparency, right? And Mike and Balaji and others in the leadership they have been very honest about it, right?
But at the end of the day, like, change is here to stay, right? That's why, like I mentioned earlier, like, at least what has helped me, you know, focus on things that you can control. That's it, right? I mean, outside of your control, we shouldn't overthink. And I know I do it all the time. I am an overthinker. I worry; I stress out. You know, I'm kind of, like, an emotional person, but that's what hurts me. That's what I have learned that I should just be focusing on what I can control.
Definitely, I mean, I need to share my feedback or my perspective with the leadership. But, like David mentioned, right, not every time you'll be able to change, you know, what you are saying. And sometimes, you know, that might not be right, like, from growth perspective or the company's overall vision perspective, right? That's where we need to find our why. And we need to stay relevant so that our goals, our purpose is aligned with what the company's overall vision is. And sometimes it kind of takes some time for all of us to kind of understand and digest that, and that's why change is hard.
JORDAN: I was going to give you, like, an example of what happened, like, it was literally my first real programming job out of college. A director came in; nobody liked him. The engineering manager directly beneath the director was very well-liked, and everybody loved him. And he was very vocal about everybody's dislike for this director. That went on for about four months, and he left, and then shortly thereafter, probably half the engineering org left. And after that, I was one of those ones that left as well.
And just to give you, like, the other side of this, if you...and, you know, play devil's advocate, if you can't effectuate change in your organization, and what you can do is effectuate change in yourself, make sure your resume is up to date, and start looking [laughs]. And just to be frank, when I left that job, I more than doubled my income, and it was nuts. But I would not have left that job if not for that bad director and because the mission of the company was really good.
But yeah, you need to continually think about yourself and your career. And if you are in a place where you feel like your leadership isn't up to snuff, if you are ready to go on and look for elsewhere, have confidence in yourself and be ready to make that step.
I mean, all of us have probably been at that point where we, you know, we decide to switch jobs for whatever reason. And this could be that reason if the change in the organization is too great for you to stay there.
DAVE: I've been fortunate to have a wife that I've come home on two or three occasions where I'm like, "I quit my job today." And the first time, she was very distressed. And then I explained to her why, and she was like, "Okay. Yeah, that makes sense." And the second time she was like, "Uh, it was kind of about time." And the next job I got was a $20,000 pay rise. And she's like, "I wish you'd quit sooner."
And closer to 10 years ago, I started a job. I was there for one day, and on the way...and we carpooled home, and on the way home from that job, I told Liz, "I think I'm going to quit my job tomorrow." And she said, "Okay." And that's a terrible skill to have taught your wife [laughter]. But yeah, it's like, sometimes you clear your paycheck, and sometimes you clear your conscience. And as long as you're holding by your principles, you can bear with a lot of terrible house, right? It's if everything was easy, somebody else would be doing it.
MIKE: I was thinking something along the lines of what you're saying there, Dave. Most companies are not Enron, right [chuckles], where they're doing something clearly wrong, but occasionally they are. You know, I quit a place…I don't know if I mentioned it earlier in the call or if it was in the pre-call; I've quit a place before where it was clearly sketchy.
This was bad business model. I'm sure that legal action was taken against them. Me and, like, I think, all of the engineering team left except for one [chuckles]. And then he left, like, two months later because it was hard for him to get another job immediately, not because of his skills but because he had some special needs that made it hard for travel.
DAVE: Sometimes you got to keep health insurance going or something, yeah.
MIKE: So, you know, there's times, yeah, you get out. Usually though, again, if it's a debate over how Scrum gets implemented, that's not really a question of values. And so, I think, you need to ask yourself, you know, is this something that is really an issue of values, or can I...you know, is there a reasonable counterpoint where I can see somebody else's perspective here? Work with it. If it's reasonable to work through, then it's probably worth doing it.
Now, again, there's times it's...and sometimes just for your career, sure, it's the right thing to go. But, you know, you've...I think you have to look at that. Is this something that I'm going to have a hard time sleeping at night over, or is this just me being grumpy?
DAVE: It's worth studying and honoring your own answer to...if you give it your thought and, you know, you can respect your own answer to that. Zach, who's frequently on the call, I've worked with him on the data team. He was my boss, and he would tell me often, he says, "You never quit a job. You quit your boss."
SEEMA: Oh yeah. I couldn't agree more with that.
DAVE: I tease him about it that because, two months later, I transferred back to engineering. And I keep telling him, "I did it just to get away from you."
SEEMA: I mean, that is so true, David, what you said, right? I mean, the other day I was reading an article on LinkedIn, and that's exactly what they were saying, right? Like, for example, our company CEO or even our CTO, you know, they have their own vision, right? And we go...everybody gets excited, obviously, you know, we're doing new things and growth and things like that.
But, ultimately, at the end of the day, I feel like it's your immediate, I guess, manager or your boss and the team that you work with. You know, you work, like, 8 to 10 hours a day. You have to make sure...and that's why, Vivian, earlier I made that point that you have to make sure that you are staying relevant, right, and you find your why. And you have to make sure that you are motivated; you are driven; you are adding value day in, day out.
And to Justin's point, I mean, if you can't accept it, you know, you can go somewhere else. But what's the guarantee that you won't go through change there, right? So, maybe we might as well just stay here, wait for some time, right, as an example, right, and, you know, learn, accept, and then see how it goes. You know, that's what I have always done, you know. As long as I am getting to learn, I'm growing, you know, in my career, I'm good.
DAVE: Your job is part of your career.
JORDAN: What came to mind while you were speaking was, if you are changing jobs every year, the problem may not be the jobs; it may be you [laughter].
SEEMA: Maybe it's just your mindset, right?
JORDAN: Yeah, maybe your mindset. So, always be thinking about that.
DAVE: Or maybe freelancing might be for you [laughter].
SEEMA: Well, that's what they say that everybody, you know, we should all have kind of a hobby on the side, like, a second...you can say it like second job, but not from kind of a paycheck perspective, but we should all of us...I mean, I don't, but then that's what, like, I feel like maybe I need to, you know, develop a second hobby or second job, you know, that keeps you happy.
DAVE: I got that advice too late [laughter]. I was all in on my vocation. And I'm like, "No, this just supports my hobbies [laughter]." Or the other way around, my hobbies support my job [laughter].
SEEMA: I like that [inaudible 51:37]
MIKE: We could probably dig into that hobby thing and do a whole episode on that. I'm tempted, but no. Maybe let's save that for another time because that's a --
DAVE: I would like to do an entire episode on that, because I had lunch with our local team and with Vivian and Jordan. And Vivian is every bit as much of a dilettante as we could ever possibly want on the team [laughter]. She's done everything, and I love that. Kindred spirit. I would love to just spend a whole show on that.
VIVIAN: I mean, two things. One, I love being able to ask a question and just turn the podcast into the Advice for Vivian podcast. It's been great. I can't wait to keep that up.
And two, yeah, if we spend a whole podcast, like, glazing me for just having hobbies [laughter], I'm not going to complain about that, but [laughter] --
MIKE: I would say that's a good place to tie things up. You know, we've talked about, you know, how do you deal with change? And we've talked about a few things. You know, it's being gentle and kind, but we actually hit on some really, honestly, some kind of challenging things, like, suck it up, right [laughs]? But present it differently.
You know, we said you have to prioritize. What do you actually care about? And sometimes that answer is, "I care enough that, no, I can't be here anymore." But, usually, that answer is probably not...that's probably not so. The answer is, "I can deal with this. I can work with this. I can grow, and this is a learning opportunity. And, you know, I can reevaluate this later, you know, after we've made some choices." And that collaboration, building that community, building that village, and being a contributor rather than just the guy standing in the road and blocking everybody's way really makes a difference.
And, like I mentioned earlier, it seems like it always comes down to the human element when we dig into these issues. Leaning on your team, working together, makes a tremendous difference. This is a social endeavor. And if you can have that group of friends that you're working with, you can usually get through it.
Until next time on the Acima Development Podcast.
On this episode of the Acima Development Podcast, Mike opens with military surgeons as his model for triage, doctors who decide fast and accept ugly outcomes because the only goal that matters is getting the patient home. He uses that to introduce time management, and Will immediately splits the problem in two, since a downed server and a rough quarter call for very different responses. In a real emergency Will cuts off limbs to save the body. He kills the API that is taking the server down, flags off the broken subsystem, or pulls a bad release, and he describes a feature flag as a tourniquet that stops the bleeding without killing the patient. Mobile complicates this because a shipped version cannot be clawed back, so the circuit breakers have to exist before anything catches fire. Kyle pushes on the harder case, which is triage when one boss wants revenue, another wants the release, and another wants one customer happy. Vivian answers with mass-casualty triage, where hospitals categorize patients by tag and pre-decide who gets treated, and Kyle counters that his problem is everything arriving tagged as immediate.
That leads to the episode's core argument, which Matt states plainly: priority is singular, and juggling several at once means none of them get done well. Will adds that ranking the queue is a manager's first job, so a manager who cannot rank has already lost the plot and left the engineer to figure out which failure will cause the least grief. He admits he keeps a second thread going anyway, because the first one gets blocked waiting on another team. The panel gets specific about human throughput. Will puts a good work block at roughly two hours, task-switching cost at thirty minutes, and realistic coding time at about six hours a day. Vivian objects that productive hours vary by person and by life stage, citing her own 1:00 to 4:00 a.m. window in high school, while Will argues that a schedule nobody else shares collapses the moment you need to coordinate. Mike admits to twelve to fifteen meetings a day and triaging which of four stacked invitations to attend. Matt estimates that eighty percent of meetings could be eliminated, and the group agrees a meeting should be small and should end with a decision or an action item. On decision-making itself, Mike points to the psychological cost and to fear of being punished for a wrong call, which pushes teams into paralysis. Matt's position is that a decision beats no decision, and Will's tactic is to make the call himself, email his boss, and keep receipts.
The last stretch turns to sustainability. Vivian argues that leaders have to avoid putting people in positions where every option causes damage, and she uses competitive swimming as her example, where teammates blew out their shoulders training twenty-five hours a week instead of the twenty they could sustain. Companies want twenty-year employees to stay another twenty, so breaking people the way a military breaks soldiers does not pay off. Mike compares it to parenting, where rule by beatings works at first and then works less and less, while respect and clear consequences take longer and actually hold. He notes that his own boss makes a point of leaving at 4:30 so the team sees what a normal day looks like, and stays late only when the situation genuinely calls for it. Mike closes with his personal system, which is early morning quiet time before the day fills up, often on a bike ride, where he picks the few critical items and launches them first so they can move through other people while his calendar takes over. Vivian gives the line the episode lands on when she asks whether the first priority should be prioritization, and Mike agrees.
Transcript:
MIKE: Hello and welcome to another episode of the Acima Development Podcast. I am Mike, and I am hosting again today. With me, I've got Vivian Moore, Will Archer, Kyle Archer. We've got Eddy Lopez and Ramses Bateman.
And I'm going to start off today by talking about military surgeons. I've not served in the military, but I've read about this. You can imagine that, in the military, particularly, you know, in, like, active warfare, there are a lot of injuries. And they've got to set, you know, a whole medical establishment within the military.
The striking thing that I've read about...and, again, I haven't served there myself, so I'm speaking secondhand or thirdhand, having read about it. The military surgeons are incredibly pragmatic. That is, if somebody comes in and they're badly injured, they don't think about the plastic surgery this person might have to get later. They think, "I need to save this person's life. I want this person to walk away." And, you know, if they've got to remove the leg, they're going to remove that leg, you know, whatever it is they've got to do. I'm not going to go into the details because it's graphic [chuckles]. But, you know, they have to be exceptionally pragmatic about what they do.
In general, they're not being sentimental. They're saying, you know, "What can I do to save this person's life? Everything else is secondary. How do I keep this person moving along?" And so, that's what they do. And they've got to deal with all the gruesome details of it.
But because they are so concerned about making those decisions, making them early, making them immediately, right, and having one priority: how do I keep this person alive? How do I get this person, you know, home? They save lives. They make all the difference between that person going home and not going home. And those are tough choices. But if you think about the end goal, they're making the right ones. They've made the choice there that they're not going to get caught up in the stress of the moment because they care about saving somebody's life.
We're building software. We're not on a battlefield. Hopefully, you're not on a battlefield. There may be some people out here, you know, building software for the battlefield. That does happen. But most of us are doing stuff that's much more mundane. But there's some lessons that we can learn there [chuckles], lessons we can learn about prioritization.
So, I gave that intro because today we're going to be talking about time management. And if you're like me, there is...And, I think, like most people, there's more to do than you can do. There's always more to do than you can do, and sometimes it's just a complete torrent. There is such a constant stream of stuff coming in that you can't even think about it.
You can't even think about prioritizing because it's just...it's just a flood. And that is a hard situation to deal with, right? It is an endless challenge. We were talking a little bit in the pre-call about how we all deal with this, and I certainly do. So, we're going to talk about that today because I think that it's something that we can all grow from.
So, I'm going to start with thinking about these military surgeons as the kickoff point and their pragmatic prioritization with a solitary goal in mind, because I think that's a great place to kick this off. And rather than add anything else myself, because I've been talking for a few minutes now, where would you all like to run with that?
WILL: Well, I just want to...I want to narrow this down to, like, are we talking about an emergent situation where the server is down and you need to fix the server right now? How do you do it? Or are we talking about, like, a rough sprint or a rough quarter? Because they're different.
MIKE: Ah, they are different. And I think that's a critical distinction. I was thinking about that again ahead of time myself, because it's different rules, right? Going back to the military triage, there's some situations where if you don't act now, then you don't get a chance, and there's other situations that can wait a little bit. And in any emergency room anywhere, they make those decisions all day, right? They do the triage, and that's why you go, and you sit, and you wait for three hours in the emergency room. Again, I haven't had much experience in the emergency room, luckily, but I know that's -–
EDDY: I'm trying to equate, like, how you're mapping that to, like, engineering, right? So, are you saying, like, someone is on their deathbed; they need priority. Is that equivalent to a software engineer saying, "Oh, the house is on fire. We're no longer...Our server's not up. That takes precedence"?
MIKE: It does. So, server's down, server's down. You're bleeding money, right? Your lifeblood is that revenue coming in, and if you're down, then you're not doing business. Depending on the size of your business, you might be losing millions of dollars a minute, you know? That is a direct threat to your livelihood. And the tech debt you have that makes everything really slow, yeah, that's important, but it'll be the same way tomorrow.
WILL: It's true. Well, I mean, like, in those situations, like, really what you're trying to do, like, I, think, you know, like, I am much like a civil war surgeon. I'm lopping off limbs. I'm applying tourniquets. I'm applying, like, real, like, macro carpet bombing type solutions. Like, oh, this API is taking the server down? Not anymore. Like, we're just not going to, you know, we're not going to run leases, or we're not going to do...Whatever it happens to be. Oh, this SKU is crashing with 500s, and it's taking all that stuff down? Like, what SKU? I've never heard of that. We don't sell it. We're done [laughter], you know?
And I will do those things, you know, like I will cut off a limb to save the body. Those are the sort of actions that I'm immediately looking to take. Like, is there a feature flag? Can I feature flag this off? Can I turn this gateway off, you know what I mean? This subsystem, can I turn it off? I'm turning things off, you know?
It's like, oh, did we push a version? Well, we're not pushing that version no more. That version [laughter] goes away now. Like, 11.9? It sounds like an 11.81 day to me. Boom. Just because I mostly work in mobile apps these days, you can't just claw back a version. You don't just stop that on a dime like you do if you're operating a web app.
So, you have to have built out, like, the sort of, like, fail-safes and circuit breakers and feature flags and stuff like that, so you could turn off features. But you had to build that first, because, God help you, if you release a version of your mobile app that gets a person in a crash loop, where, like, they open the app up and it crashes before they can do anything, like, that's spectacularly bad. And you have to get that off people's phones with the help of an ultra mega giga corporation that may not prioritize your users or the existence of your company in a way that you do.
EDDY: So, like, what's the feature flag for a surgeon, you know, who's –-
WILL: Tourniquet. Like, I think of a feature flag like a tourniquet, right? It's still there, but the blood is not pumping through this thing and, like, leaking out, spraying out all over the ground, right? We're going to tie that off, you know, it hasn't died yet, but, you know, it's no longer receiving and leaking blood, you know? And that sucks, and it's highly hazardous because, like, one API rarely exists in a vacuum, you know. You can start seeing them job queues start to pile up [laughter].
KYLE: So, in the medical world, I feel like it's easy enough to say, like, you can triage based upon, like, the specific need, like, dead or alive, right? I do feel, like, how do you triage when you almost have to have several variables? And in that, I mean, one boss is worried about money; one boss is worried about new releases; one boss is worried about a specific customer being happy, right? How do you then triage your entire workload when you're adherent to several bosses, several categories, I guess?
VIVIAN: I'll throw my two cents into this. To keep the kind of medical field analogy, I feel like that more equates to the process of being a surgeon when, for example, like, a natural disaster happens. And you've got 400 people that are all showing up to the hospital. 10 of them will die in a minute, 50 of them will die in an hour. And the rest of them may need immediate medical treatment, may not survive the week without immediate medical treatment, so on and so forth. That's a multi, like, stage trying to prioritize. You might let someone die because you know that trying to save them, although you probably could save them, might cost the lives of three other people.
And so, that kind of process of figuring out where to prioritize, I mean, at least from my experience and what I know about the medical field, they literally have specific rules in place. They categorize different people with different tags. Like, they have, like, a black tag, like, that person is going to die. And then they go from there to basically have a system of prioritization from that perspective. Like, they literally just have to categorize every single risk into a simple system that allows them to make more informed decisions from there. And they've kind of pre-made those decisions.
KYLE: Right. But how do you handle it when everything you're getting has been tagged as immediate?
EDDY: Well, you basically say, "Oh, you have a broken leg? You know, you can be in a wheelchair. It's fine [laughs]. You know, you can get around. It's not a big deal."
WILL: If everything's a priority, nothing's a priority.
MIKE: That's --
WILL: Ultimately, like, if you have a supervisor, if you have a boss at some level, and if your boss is not able to rank, you know, your current task list in order, right, from 1 2, 3, 4, 5, to however many numbers you need, if they can't rank those tasks, your boss is not doing their job. And, at that point, what you're triaging is, you know, like, a dysfunctional chain of command.
Because if you were in charge of delegating tasks, assigning job queues to your engineers, and you don't know what the priority of anything is, this is, like, your literal first job, like your first job. And so, if they've broken down, right, you know, then you're...I don't know, man. I mean, like, who's more important, and what's more urgent? And, like, you know, then you're sort of, like, managing systemic collapse really, and that's an art, not a science, really. I don't know if I can, you know...
MIKE: No, I think you nailed something, Will, and I don't know that I had even thought about this before. I recently got a new boss, and he's really good. And I can tell you exactly what my priorities are right now, and that's what I think makes him really good. There have been times where it seemed like the priority shifted day to day [chuckles]. I've definitely been in those priority-shifting environments.
Right now, for the last month, I know exactly what my number one priority is. And after this meeting, you know, after the podcast, I'm going to go into a meeting to work on that number one priority, even though it's late Friday afternoon, because that is the number one priority. And it's important to the company, so that is number one.
And number two, eh, I think I probably know what number two and number three are, and that helps a ton. That helps tremendously. And I don't know that I'd thought about it in quite that context before. It sounds like one of the most important things that the...you said it was job number one. Yeah, one of the most important things that the leadership can do is tell you, "This is what the priorities really are." And that's hard, because that requires making a choice.
You know, all of these medical analogies we're making, these are awful choices. Like, Vivian was talking about the black tag, like saying, you know, "I'm not going to help this person." Nobody wants to do that. It's gut-wrenching. But if you don't do it, worse things will happen. You've got to make that choice.
You've got to make those hard choices, because if you don't, worse things will happen. There's no escaping the consequences, so get out ahead of it and make choices that lead to less bad ones.
MATT: One of the issues is, people use the word priorities, plural, when really it's priority. You can only have one. So, you take care of your priority, and then something else becomes that priority. If you try and juggle all of them at once, usually, none of them will get done well. So, it's all about focus and priority.
EDDY: On the individual contributor, though, right? I don't know if that necessarily makes sense as a company, right? Like, I think, company-wide, you have to be able to juggle priorities, right?
MATT: All of them will roll up into a single priority.
WILL: Like, gosh, man, you know, like, I agree wholeheartedly, like, on the realm of an individual contributor, like, I can really...I don't know. I mean, like, I actually like to have two because of, like, the sort of latencies around, like, I like to have two things cooking because of latency. Like, I like to have two threads. I don't like to have three threads, and I get really grumpy with four threads. But, like, I really like to have two, because I will be waiting. Like, I'll get blocked. So, I mean, really what I want is, I want one priority, like, this is my thing. And then I want a back burner that I can work on, like, while the main priority is, like, blocked for some reason. That way, I'm just always cooking. I mean --
MATT: Sure. Most of us work that way, right? But there's always something that's the most important.
WILL: Yeah, I know.
MIKE: If something comes up, the other gets pushed away.
WILL: I mean, and, like, and I want to say, like, okay, well, like, I'm an engineering manager, right, or I'm a director, right? I'm a manager, and I've got 10 devs. I should be able to have 10 priorities: 1, 2, 3, 4, 5, 6, 7, 8, 9, 10. And I will assign them, you know, based on suitability, you know, to my 10 developers, and I have 10 priorities. And that sounds, you know, that sounds plausible, right? The reasoning is clear. That doesn't work [chuckles].
MIKE: It does not work at all.
MATT: Right. Well, because your priority isn't those 10 priorities. Your priority would be delivery, right?
MIKE: That's right. Those are the projects.
MATT: And each of those engineers would have a priority, which is one of those 10.
VIVIAN: I just love bringing it back to the analogies, especially the computer analogies. This is just, like, how a computer system is designed. Every single, like, core on a CPU can only execute one task at a time. But you can do 30 different things on a single thread just because it's designed just to shift priorities really, really, really quickly.
And the more things that you're trying to do, the faster you have to be able to switch priorities. But at the end of the day, at any given moment, you only have one priority. You may be dispatching a hundred different tasks to all of the cores on your GPU, but each time you're dispatching a task, your priority is dispatching that task. Like, I don't know –-
EDDY: Computers can --
MATT: And which one of those tasks is going to perform better if you're running a single task on multiple threads or multiple tasks on a single thread?
EDDY: Okay. But that's only possible and realistic if you're running on a computer, right? Realistically, right, your brain is not wired to multitask to that degree, right? So--
WILL: Yeah. The rule of thumb --
MATT: No, I'm strongly against multitasking.
WILL: Like, how much parallelism can a dev handle, right? A dev's ideal work block is about two hours, and their task-switching cost is about 30 minutes. And, like, their maximum programming dev time throughput is 6 hours. And, like, you know, work around that math. I mean, like, you know, different people are different, but they're not an order of magnitude different. Like, that's basically it: two-hour blocks, half-hour context switch time. I think after three hours, most devs sort of, they need, like, a meeting they can zone out in or a snack or something, you know? And, I mean, like, exceptional people, but, like, don't make your plans around exceptional people. Make your plans around, like, just normal humans doing their best.
MIKE: That's why the Pomodoro time is an hour, right?
WILL: The Pomodoro time is 25 minutes, but that doesn't work for anyone.
MIKE: Is it 25?
WILL: It just doesn't. Like, 25 minutes, you're just kicking in. You're just kicking in.
MIKE: Yeah. But, like, you say, order of magnitude. I actually agree with your time ranges. I think they're about dead on. Couple hours, 30 minutes time to switch to the new task.
WILL: Give me a two-hour block, you know? I have never seen a company dedicate ever, on any level, for anyone, ever, like, a block of, like, "Just don't **** with the devs in this block of time. Just let them do something [laughter]."
MIKE: We have, at times, tried to carve out some time, and, you know, some specific time for that, with mixed results. But there's an attempt. Particularly, you know, here at Acima, we've attempted to do that. And, I think, for the most part, individual contributors are getting some time, well, quite a bit of time in the afternoons. Looking at the individual contributors here, does that seem about right? Do you get some time in the afternoons? I'm seeing Kyle say, "No. No way [laughs]." Eddy, maybe? Eh, maybe [laughs].
VIVIAN: I mean, I can say, as an intern, that I get a decent amount of time in the afternoons.
WILL: Guys, come on, man. We're just meat robots. We're just sad meat robots [laughter]. Like, this sad brain juice we're squeezing out, like, trying to make magic happen in our brains. Like, don't give them the time in the afternoon after you, like, beat them down with, like, four hours of...It's like, okay. Run free. Make magic. Come on [laughter]. That is awful.
VIVIAN: Okay. But I do want to push back on this. I think part of the challenge of, like, having a dedicated time for engineers to do their work is that every single person that I've talked to ever in my life is functionally productive at varying hours throughout the day. When I was in high school, my most productive hours were from 1:00 a.m. to 4:00 a.m., whereas in college, that shifted to much earlier in the morning, because I had actually woken up that time. And it has changed since to now, and it's kind of variable depending on the day.
So, as much as I would like to be able to dedicate, I have three hours here; I'm going to do three hours' worth of work, on some days, I can do six hours' worth of work in that three hours, and on other days, I can do maybe an hour. So, I would love to say, "Let's dedicate a set amount of time every single day, or, like, three times a week to just working," and I bet my body would, and my mind would accommodate that and get used to that. But I just don't know if, especially, engineers can have the dedicated consistency to that for that to be as effective as just them finding the time within the time and giving them the flexibility to orchestrate their time enough to know when they will be effective.
WILL: I have worked a lot of jobs where I could work just exactly how I wanted to work. Like, my optimal programming schedule, like, the best programming, like, I've ever done, best work I've ever done in my life, like, I'd wake up at around 10:00. I'd roll into work around noon. I'd work for about 10 hours; my boys and I would hit the bar, because we all worked together. We'd close it out. We'd drink till 1:00. I'd be asleep at, like, 2:00. And I would do that six days a week, and that was the best for me at the time. And, like, having, you know, also had the experience of working at adult jobs [laughter].
I mean also, like, having kids, you know what I mean, that are going to wake me up at the crack of dawn seven days a week, no matter what. Like, you need to bend yourself to the needs of the team. I just don't think...Like, if your vibe is from, like, 1:00 to 4:00, right, which is, like, I feel you. I feel you. However, it's just not going to work, you know what I mean?
Like, it won't work because, like, I mean, ultimately, in the end, right, like, I talk about, like, these sort of, like...Like, I always like to have two threads going, because my thread will get blocked constantly, right? I'll get blocked because I'll be like, "What even is going on with this API?" And then I got to get the API team in, and then they have to tell me the answer to my question so they can build around the API. Or I need to, like, get a server spun up, and I need to get the DevOps team to spin me a server up or something, something somewhere, somehow. If it's from 1:00 to 4:00, then I'm...the first time I need to coordinate with somebody else, I'm sunk. And so, I mean, if you want to exist in a society, you know, and not just be, like, a lone wolf cowboy, you got to [inaudible 21:29]
MIKE: I'll tell you my constraints. I average somewhere around, somewhere between, like, 12 and 15 meetings a day. I tell you, there is not much time in between there to do other things. I have very little control of those gaps.
WILL: Can I ask, I mean, like, I feel like that's an organizational anti-pattern. Like, 12 to 15 meetings, like, even if it's, like, a half hour, you know what I mean?
MIKE: Well, some of mine don't go to...So, and this actually goes to...So, I have to do triage on those meetings because sometimes there will be a stack three or four deep. And so, I look at those four meetings and say, "Okay, which one do I attend?" And, you know, there's other ones that'd be good for me to attend to, and I just don't. I'll send an apology, say, like, "Sorry, I can't make it this time."
I have to carefully manage my time, or I will get nothing done. There'll be, you know, nothing but meetings morning to night every day. And they'll expand beyond the boundaries of the workday until there's just nothing but meetings 24/7.
MATT: Yeah, that's part of the position. So, you know, as you move up in an enterprise, that's what happens. And that's also why my most productive hours are, like, 7:00 p.m. to 4:00 a.m. That's when I get things done. My days --
EDDY: You also don't sleep, Matt.
MATT: Yeah, this is true [laughter]. But I spend my day meeting planning those types of things and organizing. And then if I want to produce, that happens, like, 7:00 p.m. to 4:00 a.m. And then I start my day at about, you know, 6:00 a.m. the next day usually, and that's just how it goes.
WILL: Oh, you got to be geeked out on peptides 24/7, man. This is, like, because I'm old, okay? Like, I'm old. I remember the before times [chuckles]. I remember the before times. I remember them very well. And, like, this kind of meeting didn't happen, because you had to have a conference room, and there were only so many conference rooms. And you couldn't screw around. And there were only so many seats in the conference room. There were only 10 chairs at that table, you know?
But I find myself, I'm perpetually pulled into these 40-person meetings. And, like, a 40-person meeting where the presenter is just screwing around, freestyling, a 40-person meeting, and that's a lecture. You are giving a lecture in a lecture hall, and a big one, like, a big one, and, like, I don't know. We have meeting cancer. Like, people need to make some decisions on their own. Like, and the fact that we have this unlimited meeting bandwidth allegedly, you'll find these sort of, like, serially abused decision makers --
EDDY: So, this is going on a tangent, right? But, like –-
WILL: Didn't used to do it.
MATT: Well, you hit a key point. You said decision-makers. Generally, meetings like that, there are no decisions, and they are a waste of spend.
WILL: Massive.
MATT: You know, they cost the company massive amounts of money, and you generally gain nothing out of it. And I would say probably about 80% of the meetings people have can be eliminated.
But the problem also is, especially if you're in a big enterprise and you are working across the world, that feedback loop has to be tight, and communication has to be there, and that's a hard problem to solve.
EDDY: So, if you want decisions to be made in a meeting, you have to grossly reduce the number of attendees in a meeting, right? So, I think the first step is really, who really needs to be there? And find the optimal amount of individuals that need to be there, right? If you can't do that, I feel like meetings are wasted.
MIKE: Well, and that meeting better be organized around making a decision as well.
MATT: That's right.
MIKE: If there's a meeting that's not organized around making...okay, well, there can be a couple purposes. You better be walking out of there with a decision, or you better be sharing some important information that you're not going to be able to easily share any other way, because you can probably get to that in an email.
MATT: And, usually, meetings --
WILL: Yeah. I mean, honestly, if it's information sharing, it should be a video, or an email, or a Confluence page. A meeting is the absolute dumbest way to do that because maybe I'm going to forget. I'm going to forget. I'm going to sit in front of you. I will take notes, and I will forget everything you've said to me [chuckles].
MATT: Well, generally, meetings can happen with a couple of people who are going to be decision makers. And if you don't leave with action items, that meeting was completely worthless. And then you share your information with your team, whether it be an email, during a standup, whatever. But to make meetings effective, they need to be extremely small groups. And if you're not participating and adding value, you should get up and walk out or hang up, just not be present- because it's a waste of your time and the company's money.
WILL: Absolutely. But, I mean, I guess here's the question that I've got, right? Because I've never been, like, a manager of managers. I've never done it, right? Like, everything has been, like, very, like, first party. Like, I said I was going to do it, or I said you were going to do it, and, like, and I'm not making promises on promises.
Like, why can't the managers, like, why can't managers just represent their team? They know because their boss has communicated their priorities, right? And they have access to the Kanban, like, hopefully, and, like, what all my people, all my direct reports, are working on right now, and, like, what my priorities are. And, like, why does it have to be you?
Like, I understand, you know, like, sometimes, there's conflicting priorities where, like, my 1, 2, 3 are your 6, 7, 8, and one of us is going to have to change, right? Like, my team and your team, we are going to have to align, right? And daddy's going to have to, like, sort it out, you know what I mean? Like, my understanding of what I need to do, you know what I mean?. And I'm just going to be like, "Mike, Mike"
MIKE: Yeah [laughs].
WILL: I'm not being mean, but, like, what is it? Okay. Somebody is going to have to shuffle their deck. Who is it? And you got to tell me, because I don't want to hear your mouth [laughter] when things didn't get done. But, like, well, why can't the managers just do it? Just do it.
Because I've run into this thing over and over and over where there's this sort of, like, this square root, right, of people who can make decisions and make things happen, and there's not always even a charge. There's a lady, I won't name her, but, like, H.K., and, like, if you message H.K., she's not, like, a big boss; she just runs things. And if you message her, you might not get the answer you want. It's like going to the Oracle. The Oracle's going to tell you [laughter] the truth, not necessarily the truth you wanted, but, like, the truth you're getting. If you go to her, it's going to get settled.
MIKE: I think a lot of people, well, maybe, in general, there is a psychological cost to decision-making, like, measurable. They can say that, yeah, you can, like, have people make a bunch of small decisions, and it's harder to make a big one, because every decision takes something out of you. And unless you cultivate that, unless you grow that and are willing to do that, or, you know, have encouragement to make those choices...We've talked about psychological safety before.
If you feel like I'm going to be punished for the wrong consequence of my choice, then you're going to put off that choice. And if you have that systemically, which is, I think, the default, then you have a whole bunch of decisions not getting made until they have to. Because people are only going to make those decisions that they absolutely have to make, because they're going to minimize that work. They're going to minimize that cost, that risk, because any decision I make is something I have to be accountable for. And so, I think --
MATT: And that's okay. Accountability is good, right? And, I think, early on in my career, I felt that way, that I was afraid to make the decisions, because if I made the wrong one, I'd be punished or chastised in some way for it. These days, I'm a very different person. You know, those of you who work with me know I have no problem making a decision. And, I think, the idea is: make a decision, because some decision is better than no decision. Learn quickly and pivot quickly. It's okay to fail. It's okay to make the wrong decision. And the way you improve and move forward is by correcting those wrong decisions quickly.
MIKE: Well, institutionally, there's a huge cost to not making those decisions. And so, I think what you're saying is exactly right, Matt, but that has to be encouraged, actively encouraged. And if you don't have strong leadership that's actively encouraging people to make decisions, and leading by example, making decisions, encouraging people to make decisions, then it tends to rot. And that's a really bad situation. You know, everything just falls into stagnation. Nothing gets done. It's awful.
WILL: Like, as a little worker bee, like, my tactic has always been, like...because I frequently get pulled into these sort of, like, tug of war things, right, where I'll just make my own damn decision. Like, I'll just be like, "Okay, listen, this is what I'm doing." Boom, boom, boom. I send an email to my boss, and then when it all comes down, like, it's like, I've got it in writing. I told you what I was going to do. There it is. And, man, even when you're faced with systemic dysfunction, just clear communication and some gentle keeping of receipts [laughs] will go a long way.
MIKE: And that proactiveness, being willing to make those decisions in an organization that has any semblance of health, is actually going to make you float to the top. Because people need somebody who's going to get something done. And I've very rarely seen somebody kicked out for being decisive. There's foolhardy, and that's different [laughs]. Being decisive means considering the options, you know, saying, "Oh, these are the pros and cons. I'm going to make a decision." It's not, "Hey, I'm not going to pay attention to any of the consequences. Let's just go."
So, I'm not talking about the, you know, let's just take needless risks. But looking at the options, making a choice, and running with it, that's how you get things done. And most leaders, even dysfunctional leaders, are going to see the team members who are actually getting things done, and they're going to appreciate it.
WILL: Yeah. And if you tell me, like, "No, it's not A, B, C; it's D, E, F," then it's like, "Okay. All right." Somebody just needed –-
MATT: Then we'll make an adjustment.
WILL: Yeah. Somebody just needed to tell me what letter was on top. Let's go.
MATT: That's one of the things I really like about my teams is they do make decisions, and they're autonomous. Because I am not a fan of micromanagement, by any means. If I have people on my team, my expectation is trust, and that they're going to do the job that they set out to do and get it done. If not, then I probably have the wrong team, and that's on me, right? Because I'm the one who builds the team. I'm the one who runs the team.
But I think I've learned a lot from you, Mike, and you bring up psychological safety. And back when I used to report to you, that was it. You trusted me, allowed me to make decisions. We got things done, and here we are, you know? Both of us have stepped up in our careers. And I think a lot of it has to do with that, just being able to make those decisions and get things accomplished. If you're spinning wheels all the time, then what are you doing?
WILL: Well, I'm kind of curious, right, like, flipping it on the head. Because I was thinking about...I was thinking through this thing. You know, and, like, Mike emailed me the thing earlier today. And I was thinking about, like, sort of, like, you know, like, your sort of, like, classical leadership breakdown, right, where, like, you have too many conflicting priorities. And somebody will say, like, "Everything's first priority."
And so, it's just like, okay, you know, you've given up the game, right? Like, I was like, okay, all right, my manager's lost the plot. And now I need to do their job and figure out what, at the end of the ride, is going to give me the least amount of grief, which is unfair and nearly inevitable over the course of your career, that you're going to be forced to do that, you know what I mean?
Like, it's just a heuristic, right, where I always think about these things, and it's like, you know, a manager needs to set priorities. Well, what about the other side of the coin, where, you know, you're the manager, and you're the leader, right? And you're just like, "Hey, the house is on fire. You need to work harder," you know. Because we've been there, right?
Like, I am by the lake watching people water-ski in front of me literally right now. I hope the mic isn't picking it up. But, like, last week, the roof was on fire, and I saw the wrong end of midnight every day, because things just needed to go out because of reasons. And I needed to make sure that, like, you know, like, I wasn't letting my team down while I was going on this vacation that had been scheduled for several months [laughs], you know?
And, like, we can all talk about, like, we can all talk about, like, sort of, like, you know, like, do a good job as a manager. Do a good job setting priorities and managing bandwidth. And it's stuff like that, which is, you know, conventional wisdom. But, like, what about just, like, boots in asses? Because that has to happen too. Everything that has ever been made has been born in blood and sweat and tears. There's no exception.
MIKE: So, think about a parent. There are parents who try to maintain order with beatings. And there are those that go about it in a different way, with strong, you know, enforcement of consequences without violence or disrespect, but rather, you know, "These are the requirements, and those can be consequences to your actions. I'm going to give you some freedom, so you're going to experience some of those negative ones sometimes," right? And help build them. And a constant development of respect, so when you say, "This is important," they actually listen to you.
So, you can go down the first route, and it kind of works at first, and it becomes decreasingly effective over time. The second route, it takes time, but it leads to long-term healthy children and [laughs] you know, and mechanics that actually work. And I think the same thing applies to adult relationships if...Well, sure, you're going to be in situations that are bad. You say, "The server's on fire. Go do this." And if that person's not going to go do that, well, that's a person problem [chuckles]. They should...maybe that person shouldn't be on your team. Maybe they're having a bad day.
So, I mean, there's exceptions, but, you know, you deal with that. But you get there by communicating those priorities. Most people want to do the right thing. If you're not there saying, "Hey, the server's on fire. This is what you should be doing," then you're neglecting your duties. I mean, then it's on you.
You're right. You need to be out there saying...and sometimes it's, "Hey, if we don't get this out, it's going to be bad. Sorry. Can y'all put in a late night tonight [chuckles]?" And you lead by example, and you do it yourself, then you can make it happen, as long as it's a time-limited experience and not continuous, because then you're in a permanent emergency. And was it really an emergency to begin with?
VIVIAN: I mean, I love just going back to examples, and analogies, and so on. But just thinking about the prioritization, one, yes, you have to kind of gentle-parent, at least in my experience, almost everyone you work with. Because as much as we like to think that we are adults who can always control our actions and the outcomes of our actions and our choices, we control the choices that we make, but there are things that are out of our control. Sometimes you've been working on an emergency for a month straight, and you're burned out. And as much as you think you can continue to work, your brain is telling you that you can't.
And so, at least in my opinion as an intern, as someone with tons of experience, it is, like, kind of the job of a leader to make sure that each person is not being forced into situations that make them make decisions that hurt something no matter what. Because everyone is going to have to make those decisions, but at the end of the day, you shouldn't be needing to...well, either I'm going to continue to burn out, and it's going to hurt the company, because it's going to burn me out even worse, and it's going to take me a month to recover. Or I take the break now, and it hurts the company, because I need two weeks to recover. Like, that shouldn't ever be a case that occurs, but that requires forethought. That requires specifically choosing people.
Like, it's a hard problem to solve, and it's not as simple as, like, prioritizing which task to work on. It's prioritizing: do I take the loss now of this engineer that needs a vacation terribly bad, or do I not let them go on vacation because we're still trying to get prod up and working as best we can, but then we're going to need them gone for, like, two months after? I don't know.
I think about, like, the cases of the best militaries in history. They don't function the best because the leaders will beat the soldiers down the best. Like, the commander beating a soldier will only work to a certain extent. And, at the end of the day, that soldier will not fight as hard as a soldier who is dedicated and believes in their cause. But a soldier only believes in the cause and is willing to fight for a leader if that soldier has had experience with a leader who is, one, willing to stick up for them. Two, that soldier knows that the leader is, like, also committed to the cause, and that you don't need to die month after month, year after year, and constantly throw yourself onto the battlefield.
WILL: I mean, I will say this, right? Like, the military's going to mess you up. They're going to mess you up like a car wreck [laughter] real good. They're going to mess you up just for practice. Just for practice, like, real good. Real, real, real good.
MATT: Mess you up for practice, yes. Think about sports, for instance. You practice hundreds of hours for a couple hours of the actual game time, right? And it's preparedness. But, again, I agree with what Vivian was saying, in that you're going to get way more out of someone who you have flexibility with, who can take the time they need.
Those hours that we were talking about earlier of productivity are going to be way more productive with someone who's healthy mentally and physically and can actually put focus into that time. And if you get burned out, you can't put that focus in, and you're not effective. So, someone who puts in three hours of really effective time, likely, if they're the same skill level, is going to be way more effective than someone putting in 40 hours of time if they're extremely burnt out.
MIKE: So, the manager has to then not just take into account business priorities, but the effective leader has to care about more than just that priority list. They have to pay attention to the world around them, know the people, get in there. And that's hard work.
WILL: I mean, because, like, I believe really strongly in sustainability. I believe strongly in the sustainability of an organization. But I've also, like, done the best work that I could possibly do, right? Like, I've achieved that. It's just not like that. It's weird and obsessive and, like, over...you're overexposed and, like, overworked and, like, freaked out, and, like, you know what I mean?
Like, 100% is, like, so much further than most people, you know, imagine. If you haven't been there, it's just a freaky, weird, feverish, bordering-on-insanity sort of experience. And, like, on one hand, I'm like, "You can't live there. You will literally go insane." But it's cool to get there. It's cool to taste it every once in a while. But it's also, like, sort of, like, why would you do that writing business software?
MIKE: [laughs] But it's true.
WILL: I want to get to this, like, feverish state, like, you know, optimizing our lease workflow algorithm by 20 basis points.
MIKE: But it comes down to where we started, though, which is, there are some things that are more important than others. If you're sitting in Ukraine right now, for example, you may be writing software that has life-and-death consequences.
WILL: Yeah. Like, go nuts, you know?
MIKE: But most of us are not. Most of us are not. And that sustainability matters, and that's part of your prioritization.
You mentioned before having a good boss. You said that he gets off...He makes a point of people seeing him leave the office at 4:30 because he wants to set a standard that, on the average day, we should be done on time. Now, there's been some craziness lately that meant that he's probably been around for a lot longer than that. So, he's also leading by being there when there's the exceptional circumstance. And then when that's over, I expect him to be leaving the office with everybody seeing him at 4:30 again. Because you have to say, "Yes, we're building business software. Yes, it's important, but this doesn't work if we don't set boundaries."
VIVIAN: Matt, I love the sports analogy that you brought up, because, I mean, like, you were saying, you could potentially train 60 hours a week, breaking your body right to the limit, doing everything you possibly can, and you will not win the competition at the end of the day. No matter what sport it is, that is not going to be sustainable. It will not work. Our bodies and our minds, are a part of our bodies, need time to recover.
Like we were talking about at the beginning, like, an engineer only has maybe a three-hour block, probably a two-hour block, where they are, like, really locked in and able to focus. And they need 15, 30 minutes of break time in between to kind of recharge, get a snack, get some food, get 200 more milligrams of caffeine. And, like, over the long term, if you want to be a successful athlete, you can't push your body right up to the breaking point every single time, because then you'd get injured.
I mean, I was a competitive swimmer in high school, and even in high school, the number of times that people would have shoulder injuries because they tried to swim for 25 hours a week at full tilt, rather than, like, the 20 that they could really reasonably sustain, was remarkable.
And especially in a corporation, where you want the people that have been there for 20 years already to be able to work for your company another 20 years, because they have all the institutional knowledge, know how your systems work, and can be the most effective engineers and leaders, I feel like it doesn't make sense to take a military approach of breaking your soldiers. Because soldiers retire after 20, 25 years, unless they become a general and need to stop being on the front lines and being broken. People will not sustain that level of destruction for as long as we want to hold people in the company.
MATT: Yeah. Well, and, in most cases, you don't have to perform when your life's on the line, right, and bullets and bombs are flying past your head, and you fear for your life. That's why the military does what they do, and that isn't sustainable in most aspects of life for sure.
WILL: Well, I mean, like, the military's just a different animal. Like, they get you plenty of downtime, too, but it's a wholly...I don't know. It's weird to, like, analogize business to the military, because they're just way, way different animals. But, you know, I don't know, I always like the sports analogy because, like, we're just meat robots, man. Like, you could sprain your brain exactly like you sprain your ankle. Like, you think you can't, but you absolutely can. It's just harder to think about.
MIKE: I've done a lot of teaching, and sometimes, you're not going to learn anything. Like, there's that time that it's not going to work. Doesn't matter how much you want that information to go in there; it's not going to work, and you've got to work around those constraints. The brain is not infinite. It's got its limits.
WILL: Absolutely.
MIKE: So, we've talked a lot about prioritization. We've talked about limits. We've talked about making chunks of time.
One thing we haven't really talked about is, one thing I try to do is, I reserve some of that time before work, you know, early morning, because that's my quiet time I get when I actually get a moment to think. And I will say, "Okay, what are the things I need to do today?" And if I can get those things done, and I usually try to start my day with those, and they're usually critical things. If I don't get these things done, the bad things happen, right [chuckles]? I launch those at the beginning of my day.
And those gaps between meetings, or sometimes in meetings when there's a dead spot, I am following those things through. So, hopefully, by the end of the day, or maybe the next day, those things I've set in motion are going to happen. And I think that's one of the most important things that we can do is related to that prioritization. That day's going to get out of control, and some things are going to get out of hand. And those 5 meetings you had at the beginning of the day might turn into 10.
But if you get those critical things rolling and get those things through, maybe you only get three things done, but they were the three most important. For me, that's one of the most critical things I think I can do is starting off with, loosely...and, of course, I prioritize. Sometimes it's not perfectly prioritized, but, you know, I've got this pool of things that I need to get launched, and I launch with those. And they'll go through a process, you know, people will get back to their messages and [chuckles], you know, people will respond. They'll go talk to somebody else. By the end of the day, usually, I've got some answers, right? And I've pushed things forward a little bit.
All the prioritization in the world doesn't help if you don't have the time in the day to work on it. So, I think you need to take those moments when you do have some time, whether it's early morning or some quiet time, to get that list and get that going, because then, at least, something gets done that day.
VIVIAN: So, are you suggesting that your first priority should be prioritization?
MIKE: Yeah, absolutely [chuckles].
VIVIAN: Okay. So, this is just...I'm curious; I don't know if we have the time for everything, but for people that have a lot of tasks on their plate, a lot of meetings, have a lot less time to spend time prioritizing, what are the systems that you use to do that prioritization efficiently? Like, do you have a clear to-do list of things with, like, labels, or do you have it all in your head? How do you manage these systems that all need to be running at once?
MIKE: I totally have a to-do list. I also use my message tools, you know, say, Teams, Slack, email, you name it, that I have messages unread that I manage. So, I know I can go to these. But honestly, I try to take some quiet time every morning. Went on a bike ride this morning early and cleared my head, because [chuckles] without it, my mind was just, you know, there was too much stuff. And I came home and I, like, "Okay, I know these are the things I need to get done," and that's exactly what I launched into when I started my day.
WILL: I'm really curious, like, because you're in Chicago, right? So, you're Central Time. So, you're an hour ahead. Even with Texas, an hour ahead of Mountain, right, like, what does your typical day, like, calendar-wise, look like? I'm really kind of curious. This is truly prurient interest.
MIKE: Yeah. So, I aligned, years ago, my calendar to roughly correspond to Mountain Time. So, if you look at my workday, in Central Time, it's 9:30 to 5:30 or so, more or less.
I start my day, so this is kind of outside the bounds of that, but I generally start my day real early. 5:00.
WILL: 5:00?
MIKE: Yeah. Usually.
WILL: 5:00...okay. So, like, you wake up at 5:00 o'clock. Bike ride, kids, like, whatever. So, you've got, like, four and a half hours before, like, work starts.
MIKE: That's right.
WILL: Okay. And then, like, 9:30 to 5:30. Those are pretty, like...are those hours, like, broadly consistent?
MIKE: Yeah.
WILL: Yeah? Like, cool. All right. All right. And then, like, 5:30, and then, like, what do you do? What do you do to recover? Like, what's your wind-down look like?
MIKE: [chuckles] I've got three young kids at home, so my wind-down is usually make dinner, take the kids where they need to go, mow the lawn, go, go, go. And then fall asleep next to one of my kids by putting them [chuckles] to sleep in bed and then wake up the next morning. And so, for me, that early morning time is my only real quiet time, where I actually have that time to regroup.
WILL: Interesting. Okay.
VIVIAN: It just seems that every single person, like, no matter how busy they are, needs at least, like, a couple of hours of just being on their own to let their thoughts clear and to think about things, like, no matter what. Like I said, mine in high school was, like, 1:00 to 4:00 a.m. Yours is 5:00 a.m. to 7:00 a.m. No matter what, like, it seems like a fundamental human need just to have some alone time, like, almost every single day.
WILL: Yeah. Like, you got to have a certain amount of time off. And, like, I've found that, like, if I don't have a certain amount of, like, just decompress time, I'll steal sleep to get it, and that's not good.
MIKE: No.
WILL: I try really hard for 7:00. It's usually more like 6:00, but, like, you don't want to go under 6:00. Don't go under 6:00. Don't do it. You're not going to be good.
MIKE: Well, I think that's a...We're in a good spot to tie this up. Because we've talked, you know, we've ended with, as you said, Vivian, I think your first priority needs to be prioritization. Take that time. Make that the priority, so that you can actually start filtering these days. That gives you at least some control because otherwise, you have none. And you're not going to be in control of it, and it's going to be in control of you, and it's not going to end well.
Lots of rich material here. We could probably go into aspects of this again sometime. Thanks, everybody. This was great.
Until next time on the Acima Development Podcast.
This episode of the Acima Development Podcast tackles how job seekers, especially new grads, should present themselves to employers. Mike opens with an analogy comparing modern hiring to online dating: both have shifted from relationship-based filtering to app-driven processes that strip away the context you'd normally get from a trusted network. Interns Vivian and Jordan share their recent experiences, with Vivian noting that her degree, hard classes, and high GPA did almost nothing to land interviews, while the opportunities she did get came through people who knew her. The panel adds that credentials can even backfire, since some hiring managers screen out prestigious schools or high GPAs, assuming those candidates will be hard to work with or quick to leave.
The group converges on what actually matters to interviewers: evidence that a candidate can think, solve problems, and tolerate the constant frustration inherent to software work. Jordan's apartment-scraping side project, built to solve his own problem, was what interviewers consistently asked about, and Mike recalls hiring someone whose homeschooling app proved she could identify and solve real problems even if the code was rough. Dave and Will emphasize the psychological side of the job, including grinding through failure, reading far more code than you write, and managing imposter syndrome. They also make a half-joking but sincere case for taking work at sketchy startups, since even dubious employers provide real experience, connections, and a paycheck while you build toward something better.
The dominant through line is that connections beat credentials. Nearly every panelist got their jobs through people they knew, and Dave offers a detailed playbook: cultivate loose acquaintances rather than close friends, ask curious questions over lunch, and let warm introductions bypass corporate filtering. Justin recommends joining active local industry groups like OWASP or user groups, where presenting your work makes you visible to people who hire. On the AI question, the panel agrees on tailoring effort to the audience: if an AI will screen your resume, let AI write it, but a human touch still differentiates when real people are reading. Vivian's closing takeaway captures the consensus, that human connection transcends any amount of resume optimization, and Will signs off urging listeners to add everyone they've ever worked with on LinkedIn.
Transcript:
MIKE: Hello and welcome to another episode of the Acima Development Podcast. I'm Mike, and I am hosting again today. With me, I've got Dave Brady.
DAVE: Howdy, howdy.
MIKE: We've got Eddy Lopez, Vivian Moore, Justin Ellis, Will Archer, Kyle Archer, and Jordan Dickerson. And we had a lot of interest in the pre-call here today, and I've been thinking about how to present the topic.
So, there's been a huge transformation in the way people meet each other over the last couple of decades. 20 years ago, you say, "I met somebody online," you'd get some side eye. "You met them where? On the internet? Are you one of those people [laughs]?" And now date using an app, right? It's how it's done.
DAVE: It's de facto.
MIKE: Yeah, de facto, exactly. So, fundamental shift, right? Fundamental shift in the way that that's done. There's some consequences to that. There is a loss of a lot of the filtering that you got previously because a lot of times previously you'd get people out of this network of people that you knew. It still happens, right? People still meet each other that way. They don't follow the de facto route because that's not working. They still meet somebody through somebody they knew.
But, you know, there's this lack of filter because you're going to be meeting somebody you met on, you know, on your dating app. You don't have all the context that you have, and the people say, "Oh no. Stay away from that guy [laughs]." And so, there's a lot more of this...who am I talking to? And the sudden evaluation that you got to do.
Now, I haven't been dating for a long time. Very happily married, I have been for [laughter]...28 years now. It'll be 30 years in just a couple of years. I've been married for a long time. Glad I'm not dating [chuckles], that can be painful. But I don't even hardly remember doing it that way [chuckles] back when.
But there's that evaluation process you go through very quickly. You know, is this...first of all, am I safe with this person [laughs]? Are they going to kill me? Followed by, you know, what's the next choice? You know, are they gross? Do they smell bad? You know [chuckles], to the acceptable...[laughs] Are they an acceptable person to be around? And, you know, they gradually move into your circles of trust, and you see how close they're going to get. And you have to go through the evaluation process.
The same thing has happened in hiring, where hiring has very much been taken over, in many cases, by this whole stream of filtering that happens online before you ever meet people. And people spend a lot of work carefully crafting their resume to meet keywords. And sometimes we miss the right people. Sometimes we miss the right people because they didn't have the right keywords.
And then when you meet somebody, you know, you have to go through that evaluation process. I've done a lot of interviews. I've done a lot of interviews over the years. I interviewed two people today, as a matter of fact [chuckles]. It's a thing I've done many times. And it's always tricky, right? Because you have to try to go through that filter because you don't have the context. You know, how do I quickly evaluate whether this person is somebody I want to work with? And coming from the other side, if you're looking for work, that's a tricky problem, too.
I know, Justin, you've recently done this. We've got Vivian and Jordan who were recently applying for an internship. You know, so we've got some folks here who've recently been through this process. It's hard. What do I put on my resume? And this broader sense of, "How do I present myself to employers?" is a tricky one.
And that's what our topic's going to be today. How do I represent myself, and what do I do? Just kind of a broader sense. What do I do? Should I get certificates? Should I not get certificates, or should I even put them on my resume if I've got them? Should I do networking? Should I go do volunteer work? Should I go and work on projects and put them on my GitHub? Do none of those matter at all [laughs]? I just have to put the right keywords on my resume. That's the topic we're going to talk about.
And I could throw some of my spin on this because I have, like I said, done quite a bit of interviewing. I might actually start with the interns we've got here because I know that they have very recently been going through the hiring process. And I'm curious: the first question I'm going to ask is, what did you focus on? And that [laughs] can give us a launching point. You know, what did you focus on that you thought, "Hey, this will maybe work," as you put it on your resume? And then we can maybe talk about how that landed.
VIVIAN: Coming out of college, I assumed that a degree, working hard in school, taking difficult classes, having a high GPA would be the ways that I would get a job after college. I was immediately confronted by the fact that that did absolutely nothing to do anything for me, especially in the industry that I came into in 2024.
WILL: Not nothing. Not nothing. Not enough [laughter].
VIVIAN: Not enough. Definitely not enough. I would say, for sure, not nothing. It taught me the skills that I needed to eventually do the jobs that I could potentially apply for. But in terms of actually securing an interview and moving on to the next steps, that was next to inconsequential in terms of what it actually provided for me.
The situations where I have been able to, at the very least, have an interview to potentially move forward to the next step, it has been finding a place where not a lot of people are applying for, or knowing someone who already works there who can give me a reference for something that I can apply for that either, again, not a lot of people are applying for. Or they know that I'm capable, and so they can vouch for me to get me onto the top of the pile.
WILL: God bless a sketchy, trashy, bottom-feeding, scum-sucking, borderline felonious startup.
MIKE: [laughs]
WILL: Who among us, who among us didn't break in, like, on the bottom of the barrel [laughter]? Oh yeah. Oh yeah. Oh yeah. I want to see...I mean, go lower. How [laughter]...how low can you go? If it pays [inaudible 06:38] better than DoorDash and the check only bounces [laughter] half the time --
DAVE: Half the time [laughter].
WILL: You're in, baby. You're in.
VIVIAN: So, the lesson I'm learning from this is compromise your morals and values.
WILL: Yeah. Yeah.
DAVE: It's just...be ready to settle.
WILL: Yeah. Yeah. There you go. Yeah, you got it. That's a wrap, boys.
MIKE: [laughs]
KYLE: I was going to say, I think it's interesting, too, for new grads to join the job market. I don't know how many hiring managers had the bias of...I won't give names, but the location that I went to [laughter] after college, he had a bias in the sense of what schools. If you had certain schools on your resume, he wasn't interested. He had hired those schools before and no longer wanted to deal with them.
If you had certain GPAs and you're thinking, "Okay, cool, 3.9, 4.0, that's great," nope, you were out of the list. He didn't want the high GPAs. He wanted 3.4s. He wanted 3.0s. He had a certain opinion of those. So, I'm saying, sometimes, it's even awkward in the sense that your school, your GPA, your classes, those might even negatively affect you for some individuals.
WILL: I have run into people who will not hire from certain top-tier schools, right?
KYLE: Oh yeah. Top tiers are not uncommon for...especially smaller companies.
WILL: If you went to Stanford, there were people who were just like, "No, dude," like, either they suck, you know, and they went to Stanford, but they couldn't hack it. Or they're just sort of, like, you know, they're overqualified, and, like, they're going to bounce, you know, at the earliest opportunity because, you know, I'm not Google. I'm not even Facebook.
DAVE: You need to be good enough, have a degree from a college good enough that I trust that you can do the job, but not such a nice enough place that I'm going to look at your resume and go, "I don't want to pay for that." Because you're going to start here as a new intern, and you're not going to know anything. I don't want to pay Harvard for that because you're going to have expectations.
KYLE: Well, there's an interpersonal concern, too, with either the higher GPA...the individuals that have the higher GPAs or that go to the more prestigious schools, right? How interpersonal are they going to be? How easy are they going to be to work with?
VIVIAN: I mean, I can speak from personal experience. I know that the shift from my high school GPA to my college GPA changed me as a person. I went from about a 4.4 to a 3.6, and I became a much better person at the 3.6 than I ever was at the 4.4, even though I was way more academically successful.
I fully see and agree with all of those pieces of information, and I've had specific confirmations. I had an interview with a place that I won't name at the moment, but they talked to me and basically walked me through my resume and went, "Okay. Well, you want to do this job, but you're overqualified for this. So, we know you're going to leave in a year, so we're not going to hire you." Like, they explicitly told me that out in the open because they just didn't want to deal with the fact that I might be overqualified.
But, like I was talking about, I was sitting trying to go for the bottom of the barrel, apply for anything I possibly could because I could not find a job in the industry.
DAVE: Be aware that, like, it's tempting to phrase that story as value negative. But I would say it's value neutral. They may have self-selected out of a bad fit for you and for them.
I had the exact opposite where...I'm a dilettante. I've flunked out of college six times, legitimately. My resume looks like the kind of person...it's, "Oh, you've had one year of experience eight times," that kind of thing, right? I hopped around. I was everywhere. My father-in-law said, "You're like a horse maneuver. You're all over the place."
And I went to a shop, and they said, "You work on really interesting things. You're constantly jumping to new stuff. We're not going to hire you because we need somebody who likes boring. We need a support person for this product. The person we hire is going to become the masc..." It was a small startup. "We want you to be the support guy, and you need to know this product inside and out. And that's going to take a year or two to train you. And then we want you to be happy to stay on just that product for another two or three years while we go off and build the interesting thing." And I was desperate for a job, and I treated it as value negative. And, in hindsight, it was a really good self-selection.
VIVIAN: No, I feel the same way. As much as it hurt in the moment because I was like, I need a job and I want this so bad, at the end of the day, I think, although I couldn't see into the future, in the future, that value was brought by not wasting the time doing the job that wasn't going to, in the end, progress and improve my quality of life and career.
DAVE: Yeah, 100%.
WILL: Eh, I'm still going with it. I'm still going with negative. Like, I got to pay the mortgage, man.
JUSTIN: You got to get the health insurance.
WILL: Yeah. No, I got that Obamacare. I'm all right. I'm all right. As long as I can write the check, I'll be okay. I don't know. Like, I'm more [laughs] [inaudible 11:49]
DAVE: I 100% agree with you. We're talking about timescales, right? Like, you got to pay the rent. You know, I'll flip burgers, man. But a year from now, two years from now, a burger joint's got to know I'm not going to be here, right? You are rent money. You are not career.
WILL: I just choke it down to my inability to, like, [inaudible 12:08] them on effectively enough. That's a personal failing.
MIKE: So, Jordan, what did you think would be valuable on your resume?
JORDAN: Well, I haven't quite graduated yet. I just finished my sophomore year at the U, so I didn't think that would be as helpful, especially because every internship I applied for was telling me they wanted me to be a junior before they would hire me on. So, I mostly focused on the personal projects that I built outside of school.
My biggest one was a thing that would scrape all apartment listing data in Salt Lake City to find me the best deal on apartments. I built that in Java Spring Boot, and that's what interviewers would bring up the most. So, I kind of moved that up on my resume every time I redid it.
DAVE: Especially because you built it because you wanted it, not because, oh, I just figured I'd throw something up for resume fodder, right? It's like, okay, this guy knows how to scratch an itch. Yeah, I like it.
MIKE: Also how to pay less on rent.
JORDAN: Yeah [laughs]. That was the biggest thing.
DAVE: So, we could pay him less [laughter].
WILL: I think more than anything, like, if I'm...because I've hired junior developers before. And that's a standout to me because, like, more than anything, you know, in my advanced years, the thing that matters the most, I feel like, in terms of, like, longevity and effectiveness in the business is mostly psychological.
Do you really like making stuff, and can you deal with the inevitable frustrations of just the unending torrent of nonsense that the trade is going to throw at you? Like, it's just not that hard to parse JSON. It's just not. Anybody could parse JSON. I mean, I could teach anybody of slightly above room temperature IQ to read some JSON and read these logs and do it, but --
DAVE: I disagree with that. This is so weird to be the person disagreeing with you in the other direction. But there is a programming test called FizzBuzz. And [laughter] if you talk to a junior developer, they will cross their eyes and kind of tilt their head, and they'll think about it. And if you ask a senior developer, they look at you, and they go, "That's not a programming test. That's a typing test," right?
It's literally print the numbers 1 to 100. If the number's divisible by three, print the word fizz instead of the number. If it's divisible by five, print the word buzz. If it's divisible by both, print fizzbuzz. So, it's literally one, two, fizz, four, buzz, and so on down the thing. And a senior will be like, "That's a typing test."
And I was offended by that test and its simplicity, and then I ran into three people in a row I was interviewing...I was hiring at that point for my company, and I ran into three people in a row that couldn't do it. So, there is a floor. There is a floor.
I agree with you if you are thinking of people that kind of meet a minimum thinkability standard. That shop was where I came up with the idea that I don't care if you program in the language that we use here. And I know I'm going to have to teach you our codebase. I don't mind teaching you a language, but I cannot teach you how to think. And that's what I interview for when I interview candidates: can you think?
WILL: Fair enough. I mean, I don't know, I mean, like, me personally, like, if you've built some projects of significance, like, on your own, again, this is, like, just me, I wouldn't recommend anybody try this out because of the way the industry is, especially right now.
But, like, somebody self-taught who built some stuff and it works, and you can use it, and they did it all on their own versus, like, you know, a bachelor's degree holder without, I would lean more towards somebody who's, like, actually built some stuff, and they did it on their own and they have, like...because it signifies some of those intangible qualities. But I feel like the way things are right now is, like, everybody's like, "I want both [laughter]," you know, which is, I don't know.
MIKE: Well --
WILL: I don't know if it'll stay that way. I don't know. There's a lot of feeling around, like, this is a brand new world. Everything's different now, and I'm just like, reaally? Really different? It doesn't feel that different where I'm sitting.
MIKE: There's, not only in software, but across many industries, not most, I've seen statistics to suggest that new grads are in a really weird spot. So, new grads are in a weird spot, like, across the board. That's a complicated phenomenon. Maybe won't go in here. But, yeah, I think that whether that's going to persist, I can't say.
WILL: I feel like cheap, easily exploitable workers will always have a place in the American economy.
MIKE: [laughs] So, I would also agree that the experience, it matters. I'm thinking about people...quite some time ago, I interviewed somebody who'd been at home doing school with their kids, and she built an app to manage that. And it was probably bad code [chuckles], right? But she saw a problem and solved it, and had the code and was using it, like, actively using it because this solved a problem, from zero to that in some relatively short amount of time. Like, oh, wow, that's impressive. And that is somebody who's gone on to be very successful in their career. Hired [chuckles] at an entry-level, but has gone on to be very successful. Yeah, that really does make a difference.
And I'm inclined to say something similar. We don't hire people to write code. That's part of it. We hire people to solve problems. You know, if somebody has been about the business of solving problems, they're somebody that I'm very interested in.
I also really align to your thoughts about the psychology of it. I've said, in the past, and I've probably said on the podcast, that I've felt many times that software engineering is primarily about frustration management. That's what you're living with is endless frustration and being able to deal with that and get through it. It's related to the problem-solving, right? Here, have a whole bunch of problems you can't solve. Good luck [chuckles]. Knock yourself out.
And if you can navigate that, navigate the imposter syndrome, like, should I...Am I even somebody who can do this: navigate the, you know, emotional pain of failing over and over again, of going and discovering stuff and trying to do stuff where you don't know enough, and so you are kind of incompetent? You know, all of those things is challenging. And then, in the end, if you can solve problems, that's somebody that you want to hire.
KYLE: It's interesting that you kind of bring that up, because while we were in the middle of the pandemic, my wife was like, "Maybe I want to do more of a career like you do." And she's like, "Maybe I should get into development." So, I got her doing a few programming tasks and sitting down and working on some of these things. And she was doing something a little bit beyond a "Hello, world."
And she ran into a problem, and we debugged through it, and she ran into another one. And it was one of those things where she was just kind of like, "This is annoying. I would not want to do this all day [laughter]." And I had to be like, "Hun, this is what you do all day."
DAVE: This is the job [laughter].
KYLE: This is the job. Like, if you don't like this part, you're not going to like it. Yeah, she quickly pivoted from that interest, but...[laughs].
DAVE: That's great.
MIKE: That was the right choice.
DAVE: That's completely valid. That's fantastic. I look at this job, and it's a Rubik's cube. I get paid to work puzzles all day, and I love it because it's puzzles, to me. But I can absolutely see it as being maddening at the same time.
EDDY: Actually, I'd argue that getting a different outcome after an hour is a lot more exciting to me now than having to read someone else's hogwash code, right, that made sense to them at the time [laughter].
Like, I spend a lot more time reading than I do writing, you know, and so I have a different level of elegancy and standard, right? So, what I usually tell people is, yeah, like, yeah, you got to learn to, like, push through, like, all of the mundane stuff. But also, I say, "Do you like to read?" Is a huge one. I'm like, "Oh, you don't? You struggle with reading? Sorry, dude. Like, that's all you're going to do is just read, read, read, read. And if you can't do that, then..."
DAVE: This is deep-ends knowledge work. Yeah.
WILL: How do you like writing passive-aggressive, like, emails [laughter]?
JUSTIN: Hey, that's my job [laughter]. Here's a vulnerability. I feel like you should fix this, you know [laughter].
WILL: Before the company is bankrupted [laughter] or all our servers melt through the floor because they've been taken over by crypto mining bots. What do you think? What are your thoughts?
JUSTIN: What do you think [laughter]?
MIKE: I think that a quality engineer would care about these things. I don't know [laughter], how do you feel about it?
WILL: Ooh, I got one. I got one for you. This is an interesting defect. However, I thought perhaps the back-end team could fix the back-end problems because I do not have access to that codebase. But if there's something you guys would like me to do without access to the code, please let me know [laughter].
JUSTIN: Just add a language barrier in there, too, then --
MIKE: That's right [laughs].
JUSTIN: Then you've got my job to a T [laughter].
WILL: Do the needful, boys. Do the needful [laughter].
DAVE: If this podcast ends with two of us quitting our jobs, I'm going to call that a backfire [laughter].
WILL: If this podcast was live, I would be flipping a coin as to whether my laptop was going to get bricked by the time I stop recording [laughter]. Sorry, guys. Let me just put you on mute. I'm getting a call from HR [laughter].
JUSTIN: You get calls? Wouldn't that be just an email [laughter]?
WILL: Screen just turns black.
JUSTIN: Oh, man. So, one of the reasons why I suggested this was to talk about what kind of shines in, you know, what you guys are looking for, and we've mentioned grades, pluses and minuses. We've mentioned things. But the thing that was, like, key there was, like, show that the candidate thinks, and you can do that in several ways. I mean, obviously, you do that through a live coding test that they're live with you and, hopefully, not, like, looking at an AI prompt on the side. But --
MIKE: I actually watch for that. Like, I watch their eyes.
JUSTIN: Yeah, I know [laughs]. That is something that we deal with, too. But it's like, there are a couple different ways to kind of establish yourself as somebody who can work. And I think one of the biggest ways is just, like, the people who know you personally or who have worked with you can vouch for you as someone who knows how to work and knows how to think. That, I think, is, like, the number one way that you can convey to people that, hey, you can think, and you're a good candidate for this job.
Sometimes it's difficult to know if, you know, if you're a friend of a friend of a friend. I mean, LinkedIn, for all its pluses and minuses, it is good for, you know, if you're going to have a long career, which hopefully you will, it is good to stay in touch somehow. And you aren't going to stay in touch through, like, Instagram or something like that. All that's kind of personal.
But LinkedIn is fulfilling a function of professional connections, such that you can remember people from, you know, five jobs ago, and, you know, it'll remind you. That's one way. Another way is just, you know, making sure you keep journals so you can write down the people you work with. But I've found that LinkedIn has actually been a net positive for the most part for me. But there's definitely just that function, that core functionality of remembering who you've worked with, and seeing if a candidate has worked with somebody that you know that you can ask them independently about. That's really good for that purpose.
DAVE: Yes to all of that, and I just realized the thing I'm about to ask is going to be a topic shift, so I'll leave it in the water a bit. When we pivot, Justin, you talked about certificates in the pre-call, and we all got very excited about this, and my question is largely revolving around that.
WILL: Oh, I just want to throw in an endorsement. Like, if you're listening to this, everybody you went to school with, especially the people who didn't suck, and everybody you were on a team with, whether they sucked or not, add them all. Add all of them. All of them. Because, like, you know what I mean, the thing about it is, is, like, they know you. They know your work. Like, hopefully, you weren't terrible, and they'll be like, "Well, you know, Will showed up every day. He was rarely late to stand up. And, you know, you could do worse." I'll take it. I'll take it. That'll get you an interview at the very least.
So, I mean, like, I've gotten jobs from people that I didn't like, and they didn't like me. But, like, they knew I was reliable, and I knew that I wouldn't lose my house. At the end --
DAVE: Those people are significantly...like, no joke, loose acquaintances are way more valuable than close friends when you are looking for work. Because your close friend is going to want to, A, is going to...if you say, "Hey, who's hiring?" they're now immediately filtering for good matches for you. They don't want to send you to a bad company because you're going to come back to them and go, "Man, the job you told me to go get..." Even if you're like, "I promise you I'm not going to be upset if you send me to a bad place," right, they will feel that way. And also, in reverse. I've had it kind of reverse, where it's like, ooh, there's a really...shop over here, but I don't want to burn my reputation if I recommend somebody who's kind of a dud. Loose acquaintances.
Okay, to be fair, if you're a dud, that's going to follow you no matter where you go, because wherever you go, there you are. But loose acquaintances are people who they don't care if you get the job or not. And they're not asking for...they're not going to be thinking, "Who is hiring the exact shape of you? Who is hiring somebody with seven years of SQL Server and two years of Rust?" whatever, right? It just...that's a zero-hit query, right, for most people.
But two things to do: find your loose acquaintances and ask open questions that have nothing to do directly with the job. "Hey, who's working in Postgres? Hey, do you know anybody that's doing geographical queries? Who do you know that's doing GraphQL stuff? Did I hear you say you were working in microfinance? Who else is doing that?" that kind of stuff.
And then you don't go ask that person for a job. You say, "Hey, can you introduce this person to me?" You get an introduction, then you go to that person and say, "I'd love to talk with you about..." whatever it was you asked your acquaintance about. "You guys are doing geographic stuff. You guys are doing microfinance. I'd love to talk with you about it. Can I pick your brain?"
And this is where it gets expensive. I usually buy them lunch, and I show up. Everything is fascinating to me, so I...it's like a podcast. I don't have a topic or a subject. We just jump in, and I get fascinated, and off to the races you go. Take that attitude to a lunch with somebody and just say, "Hey, tell me about...what kind of database stuff do you..." And have questions. Get interested. Get curious. If you say five words during the entire lunch, they will think you're a genius because you got them talking, right?
I can't remember what the quote was that...two prime ministers, Benjamin Disraeli...somebody said, "At a dinner party, if you had dinner with him, you felt like he was the smartest person in the room. But if you had dinner with Winston Churchill, you felt like you were the smartest person in the room."
So, ask questions; get curious; get interested. Then on the way out, it's natural. It's like, "Hey, are you guys hiring?" Half the time they're like, "Yeah, and I get a hiring bonus if I recommend you. Send me...don't send it to HR; send it to me." Now you've got a warm recommendation to HR, and that goes right through most of the corporate shielding, and it's a fantastic way. So, there you go. There's my secret to finding work.
WILL: I like that, but I also sort of wonder about the denominator, right? I mean, because it's, like, sort of, like, you're going to have a certain number of shots on goal, right?
DAVE: Yeah.
WILL: And then it's kind of expensive per try, right?
DAVE: It is.
WILL: Where it's like –-
DAVE: It is. It's not a scattershot. It's spearfishing; it's not net. It's not casting a net. That's absolutely right.
WILL: Well, exactly. I mean, and so it seems to me that that is suited for a particular kind of job hunting, which is, like, A, you're not missing any meals, and B, you have something specific. And, like, I would almost say, like, implicitly, you're a little bit more of the senior developer, in that, like, I am looking for this sort of a role doing this sort of a thing, and I have something to offer. Versus just, like, I need to not --
DAVE: Maybe.
WILL: I'm a baby turtle, and I need to make it to the ocean before the gulls pick me off, you know, by any means necessary.
DAVE: I'd be really impressed if a junior dev took me to lunch and said...this is a podcast call. Please take me to lunch. I'm hungry [laughter]. But if somebody came and said, "Hey, talk to me about what you guys do," and then I come to find out that they're very, very junior... that actually happened.
I'll name a name. Brandon Hayes is a Ruby developer that was here locally. I honestly don't know if he's still doing Ruby. He moved back to Texas, but he came to me...2010 and said, "I'm in marketing, and I hate marketing. How do I get out of marketing?" And I said, "Come to URUG." Somebody told him to start coming to URUG. That's how I met him. He was coming to the Utah Ruby Users Group. And we sat and started talking, and I didn't get him to a company that hired him, but he picked my brain.
And because he was a little older, he was a little more mature, so he had real experience. He had 10 years or, you know, 8 or so of experience as a marketer, and he was good at it. So, I was able to give him kind of, like, customized...I guess that's the other thing is, if you're good at something and it's not the thing you want to do, get a job where you can do both, because then it goes on your resume.
And if you can get...and they said, "Well, we'll hire you half, 50/50 marketing and programming." And he's like, "Should I take the job?" And I said, "Yes, and be aware that they're going to work you 90/10 at best. You're going to have to fight for every second you get to program. But after a year, you're going to have one year programming experience on your resume, and that's going to be your ticket into the next place."
WILL: Yeah. Yeah. Like, I worked with Brandon, not in the 50/50 marketing role, but, like, when he was just learning to program. Utah Ruby community is, like, crazy small. I think the Ruby community in general is crazy small. Yeah, I don't know, someday I'll get back to it.
DAVE: It's well-connected. It's very personable. Yeah.
WILL: So, I really love the community. I like the language, too. Someday I'll get back to it.
JUSTIN: I really like what you brought up, David, about the community, and that goes back to, like, your connections to other people. I'm mainly part of two communities, like, industry communities. One is the local OWASP chapter, which is, like, OWASP Application Security Group, and it is actually really small. But we have meetups, probably I attend every other month at least.
So, we are active, and it's local people, so, you know, people in, you know, Utah County, Salt Lake County, and Davis County. And there's probably 20 people, 25 people that attend off and on. But we know each other, and if anybody's, like, looking to hire people, it's like your network has just expanded so much because of the people in this, you know, cybersecurity group.
And so, that's...I'd recommend people go out and join an industry group that is active locally. It's not something that you're remote or anything else. This is somebody you go out and have lunch with, like, once a month or once every other couple of months.
And the other thing that that brings is, like, we do presentations at that event. And so, you present, and you get the opportunity to show other people, like, your technical expertise. And when they're looking for people to hire, they're like, "Oh, yeah, Justin presented on AI security two months ago, and it looked pretty impressive, and we're hiring for somebody for that. And so, let's go talk to that guy." So, it's a lot easier to present at that local group than, you know, say, at DEF CON, or something like that.
So, that, and I'm also a member of UJUG, which is the Utah Java Users Group. And I go to that one every couple of months and say hi to people, and, you know, you just get to know people there.
DAVE: I'll just say I don't think I've been to a URUG in the past year that there wasn't somebody looking for work or looking to hire. And so, like, that's an open...that's a fair...there's no shame. That's what we're there for, in addition to learning cool stuff.
WILL: Yeah, I mean, it's a lot like getting in shape, right? Like, it's a process. It's a long-term process, and you can't do it in a day, you know, when you have to, like, sort of bootstrap it from, like, a cold start. Like, it was like, "Oh, I got laid off, you know? Like, I lost my job, what do I do?" It's like, okay, well, step one is hop in the time machine and go back six months and do all the right things that you were supposed to do.
But, I mean, I will say that, like, I mean, not for nothing, and, like, I say this with the full understanding that I have, like, a family, and I've got a full-time job. And I try and do things that aren't sitting in front of this keyboard like a lab rat. And so, I don't do any of this stuff, like, not nothing, but, like, the absolute bare minimum. I'm not better than anybody. Like, I'm not doing any of this stuff.
But the thing that makes it such a differentiator, especially, honestly, like, if you're in the sort of, like, scraping just to survive stage, is nobody wants to do it. Everybody walks in the door at 6:00 o'clock if you're lucky, 7:00 [laughs] if you're not, or worse. And they don't want to sit down in front of the laptop and, like, hit congrats on the promotion to your project manager that you hated five years ago because they always wrote screwed-up stories. But now they're a principal product owner in a company that you might want to know about six months from now, you know?
And, like, nobody's doing that, and that's what makes it such a differentiator. And it really is a needle mover, and it's a needle mover because it sucks for everybody, and nobody does it, and everybody hates it. And if you're willing to do it, you know what I mean, over the long term, you know, it will move the needle for you in a really meaningful way. You got to stay after it.
I mean, like, a lot of the stuff where, like, I tell everybody who's a junior who's looking for work is to, like, just go out and do personal projects. Share them on the internet. And, like, you'll find these blogs of people who are doing exactly this, and they run for six months at most. And then they're never updated again because they got a job in the industry and they're working now, and they don't have time for that [laughs].
MIKE: There's a through line in all of the comments so far. You know, Vivian talked about finding people who would...through connections. Like, you know somebody over there [chuckles] and working those. And now we're talking about LinkedIn and maintaining those relationships. We talked about getting involved in communities to have those relationships, so the connections.
So, we started the call talking about, you know, dating and how we used to depend on those relationships to make it happen. And everybody goes to the app. And the same thing has happened to job searches. You know what? You know how easy it is to find job search by just throwing yourself out on the app? You will never get a bite. Maybe you get a bite. I've never gotten a bite that way in the past. I think every job I've ever gotten came through a connection I had. So, we're saying that the connection network never went away. It's just been papered over by these apps sitting on top that maybe they work sometimes [chuckles].
WILL: I mean, foundationally, like, I think, it boils down to, like, availability bias, right? Hiring managers are lazy as hell. They're lazy as hell. Not lazy, they're not all bad people. Some of them. I'm sure there are exceptions. But you got to make it easy. Don't make hiring managers think really hard, you know? Like, they get a stack of 100 resumes, and they want 10. They want 10 so bad. Make it easy to be one of the 10, and, like, the better you could do that, then, you know, the more successful you're going to be.
People...like, there's nothing that makes a hiring manager happier than, like, a hot lead, where it's just like, oh my God, you're saying, like, Dave knows a guy, and I could just skip this entire rigmarole? Like, I don't have to go through the marathon of, like, interviews, and callbacks, and reschedules and, like, all this stuff? I could skip it all and just hire Dave's guy and get back to work? Oh my God. Make it easy.
And the easier you can make it, and, like, the wider a net you could cast...because, I mean, like, all this stuff is dumb. It's dumb. Like, there's 10 guys in the stack of resumes better than me, that's it, you know? Like, I mean, honestly. Like, if you went through the whole stack, I guarantee you could find somebody better than me every single time [laughs]. But I got somebody to send you a text message. And it's just like, "Okay, just hire him. He's fine [laughs]."
MIKE: I would also suggest where you should put your effort, and that's kind of where we started this. Where do you put your effort? Do you go get a bunch of certifications online? Is that going to help you? I'm guessing that is less, you know, based on everything that we've said, and this is based on my personal experience as well. I don't care if you've got five certificates. In fact, I'm like, "Why do you got all these certificates?" Unless it's, you know, very specific that it's something that I care about, which is probably not what it is.
But if it's somebody I know and they can vouch for you, bam, you're in the door. That's a big deal. Anything you can do to build those connections is probably more important. So, laying out a practical approach here. It's probably more important than meaningless credentials. There are meaningful credentials. If you go through four years of school, that was a lot of work. That was a lot of money. You proved that you could get through that [chuckles]. But there's those other things that are worth a lot less.
WILL: Yeah. I think most of the online credentials, pretty rough. Pretty low impact, I got to say.
KYLE: Even then it's your skill level, right? Because, I don't know, Mike, tell me if I'm wrong. Do you care if your seniors have a four-year degree?
MIKE: No.
KYLE: Does that impress you? That's...
MIKE: [inaudible 39:13]
KYLE: It's just new hires, right? So, I mean, even then, it's like...to me, it'd be more impressive if I saw a junior with more certificates than if I saw a senior with five certificates, right? If I saw a senior with five certificates, I'm just like, "Okay. Didn't you already know this? Why'd you take the certificate test?" Like [chuckles], I don't know [laughs].
MIKE: Yeah. Yeah. Well, when I'm interviewing, I'm trying to deduce the problem-solving skills. I'm getting a sense of their experience. I'm not asking what your GPA is. Never once have I thought, "Oh, what was your GPA?" I've thought, you know...never has that even crossed my mind as I've hired. I've looked for somebody who's curious, right? Curious, somebody who sniffs a problem and they can't resist working on it until they get it solved. Somebody who can, yeah, break through the obstacles, get things done, somebody who can deal with challenges. Those are the kinds of things I'm looking for.
JUSTIN: One thing I would like to invite, like, the newbies here, is to look into volunteer work in, like, a passion, something you find interesting that you enjoy doing and that you can contribute meaningfully there as well.
And it just goes back to, like, the cybersecurity group I'm part of. I'm pretty passionate about that. And going and, you know, having fun with this group of friends that you're all part of and that you are doing meaningful work together. That is actually...it makes it not a task, not a chore, to reach out beyond yourself.
So, figure out what you enjoy doing and see if there's, like, some sort of community group or something like that. And everybody needs a programmer [laughs], every single one of those community groups. But that, and then, you know, if it is an IT type of skill or industry that you're in, cybersecurity or something like that, reach out.
WILL: I mean, I'd say, like, this, I mean, like, not for nothing, but, like, there is a through line that I think nearly everybody, like Dave, Mike, Matt, Kyle, I don't know whether you're a member of the club, but, like, sketchy startups, sketchy [laughter] startups, man.
Like, because here's the thing, right? Like, you'll get...again, the money is highly dubious. Highly dubious. Like, if you've ever had a paycheck bounce, like, I know I have, but you will build stuff. Usually, these guys are hustlers, and you'll get, like, a year of experience and, like, stuff that you've done. The world is full of sketchy startup hustlers, you know? It worked for me, you know? I don't know what to say. Like, it worked for me [crosstalk 42:00]
JUSTIN: Yeah. So, to expand on that, Will, there are a ton of entrepreneurial groups that form in Utah and other places that, if you can go to these, like, presentations that they're doing, like, the elevator pitches and whatever, and you say, "Hey, guess what? I'm a programmer," and they're like, "Ah [laughter]. Somebody who can make my idea work." So --
DAVE: There used to be a group here in Utah County that was CEO Breakfast. The generic form, if you want something a little less depressing, Vivian, the generic form of Will is where there's muck, there's brass. You're not going to find gold sitting out in the middle of a field because everybody's combed that field a million times. All the easy gold's been picked up. If it was easy to pick up, why isn't anybody picking it up? But if you're willing to get down on the latrine, get up to your knees and shovel it, and people see, "Oh, you've been shoveling." Yeah.
And, at the bare minimum, if you go work at a skeezy place and kind of hope for the best and it doesn't work out the best, you'll learn what you don't like. And you'll learn how to spot it.
WILL: It's true. Well, I mean, and, like, you know, those places have massive turnover, and that's just build your network [laughter]. Like, they've all been scattered to the four winds. It sounds bad because it is. It is bad, but it's better than nothing.
DAVE: Will, we need to do job bingo. Like, it's like, you know, I5, CEO went to prison. B6 [laughter] --
WILL: Oh man.
DAVE: I have I5 [laughter].
WILL: Oh, I got it too. It might be the same guy. Yeah, I got bad stories. We can save that for the post-call because, like --
MIKE: But, you know, it comes down to what your specific needs are in the moment. There is some moral value in feeding your kids and [chuckles], you know, paying the rent.
DAVE: [SP] Begrudging.
MIKE: And working for a place that pays you most of the time can get you there. And putting up with some bad things while you're doing that is, you know, arguably not lowering your standards, but doing what you can in the situation that's been handed to you, and allowing you to move on to what you care about more.
If you have some flexibility, do the volunteer work, right? Well, then you're not getting paid anything. It's the same sort of, you know, that you're taking the opportunity to go do some real work, make those network. It's a similar sort of approach depending on what your needs are.
WILL: I mean, I'll say, like, you know, like, volunteer work might be an unsung hero there. Because, like, I'll be totally honest with you, I mean, like, most nonprofits, they got a lot of rich people around.
MIKE: [laughs]
WILL: I'm not kidding. Like, who donates to nonprofits? Like, rich people with bad egos, and, like, guilty consciences [laughter], you know what I mean?
JORDAN: And a lot of money that they're willing to give.
WILL: A lot of money and guilty consciences [laughter]. And --
DAVE: I feel like this is the bad advice podcast, is like, "Okay, what you want to do is go work for the mob, and then...[laughter]"
WILL: It's not...I guess I'm trying to, like, I suppose, like...I'm being absolutely sincere, right? I mean, in that, like, it's not just me. It's not just me. My career is not the only career that fell off the back of a truck, you know [chuckles]?
MIKE: Looking for work in the year after the 9/11 bit was not great. And, yeah, you're going to pay me to do work, and I get a check? Okay, yes. I'm not even going to look at what the amount on that check is [chuckles]. If it's money, I'll take it.
WILL: It's the truth. It's the truth. I don't know.
DAVE: I'll put a pin on that one as well. Don't be terrified by a bad job market. It's a game of percentages. If you are in the top 10%, then when the market is really, really low, you actually are gold at that point, because they're looking for people, and you're the right people to look. So, you're only competing with yourself and the person next to you.
WILL: Well, I mean, and it's...I mean, something is always better than nothing. And luck is also a factor of time.
DAVE: Absolutely.
WILL: You know, it's a matter of shots on goal. If you're just hardheaded and you're just going to keep on knocking on doors until one of them opens and you're not sort of overly precious about quality of the offer that gets you in the door, the cream rises. Like, there's just not that many good engineers in the world. There's just not.
If you're good and you're persistent, then I believe...and maybe this is just a, you know, a naive, like, Boomer-esque article of faith, but I just don't see that many good engineers in the world. So, that, like, somebody who has the talent and the sand for it and is willing to stick it out is not going to be successful in the end.
VIVIAN: Okay. This may be taking it too off topic, but I have questions. The first one is, from a business perspective, from the other side of the aisle, how do you approach kind of the inverse problem of you have 300, 400, 500 people who have applied for this job? How do you know that you're picking the right person? And, obviously, you're not going to know that you're picking the right person.
But how do you find the people that you know are a bet worth taking, and how do you, like, make a positive return on that bet? Because every time you're making a hire, you're making that, essentially, a gamble, because you don't know. No matter how many references they have, you don't know. Like, is the reference way the only way to kind of even the odds, give yourself that edge in the gamble?
MIKE: Go to a role-playing game store and buy dice with a lot of sides [laughs]. There is some real aspect to, like, the keyword matching, having some of that. And you know what the biggest thing is about a resume? No grammatical errors. If it has no grammatical errors, if it's formatted correctly, you're better than 90% of the resumes out there.
WILL: Really? I mean, because, honestly, like, AI tools, like, I have, like, I mean, for better or worse --
MIKE: Maybe it's changing.
DAVE: People don't run AI. I work on an AI site after work talking with people. And they submit articles and documents and short stories, and there's...like, the summaries that they write, the AI will do that for them, and they don't. And I'm like, "Bro, you have AI. I know you have AI [laughter]. You're on an AI site."
WILL: I don't know. Like, I'm a weirdo, and I have no...I have only the loosest possible, like, relationship with how normal people think, you know? Like, I project, like, my theory of mind onto people. It's not very accurate. I got a lot of misses. But, I mean, like, it's just, like, I put it in Claude. I put, like, the job description into Claude. I put my, like, big resume, right, where I just have everything but the kitchen sink, you know, and I put that into Claude, and then I season with embellishment as needed.
Listen, if you're not cheating, you're not trying [laughter]. And I assume that that, like, five minutes of, like, half-assed labor is what everybody would [laughs]...
MIKE: And that's what differentiates you. That's what differentiates you. And even in this AI era, yeah, you're going to get a lot of AI-filtered resumes. But the people who have actually gone in and gotten the weird stuff out and made it actually match them and added some human touch, they'll stand out.
WILL: That's so weird to me. You know, I guess, yeah, sure. Why not? Okay. Hmm. Who knew? I mean, I don't know. I wonder a lot about, like, you know, the next step in the dystopian arms race, whereby, you know, AI will generate you a thousand plausible resumes, and a recruiter or a hiring manager or whoever needs to winnow that down to, like, a human-processable amount of data, right?
I mean, you think about, like, 1001-page resumes. It's like, I can't read War and Peace to hire a junior developer, you know what I mean? And so, like, what's the next step in the arms race? It's like, is it going to be like an AI call, you know, where I have, like, some AI bot who's going to give me a call to, like, just, like --
MIKE: Maybe.
WILL: You know what I mean? I mean, is that how it's going to go? Because, I mean, fundamentally, I do have a lot of empathy for hiring managers, because, like, I don't think people who are applying for jobs can appreciate the magnitude of 100 resumes. They're all pretty good. You're stuck with this person. Whoever you pick, you're stuck with them for the next, you know, two to five years. And, like, that relationship can be an unending torment, or it could be your new best friend, depending on, like, how good of a job you do in, like, sniffing out who's going to be a goer and who isn't. It's a tough thing to do, and it's a daunting amount of work. I don't know. It's hard to do. And you're on the hook in a big way, in a big way.
VIVIAN: Up until this job, I never used an LLM for doing any sort of coding beyond the Copilot auto-complete that would finish one or two lines of code for me, which essentially was just an advanced, like, linting auto-complete. And I never used AI to do any sort of cover letter writing, any sort of resume writing, any of that. But I fine-tuned that to the best of my knowledge.
I had multiple recruiters and people look over it to try to give me the best advice. Often, it was conflicting. I tried to do specific things to add a light touch of humor, a human touch. And I wrote every single cover letter that I applied to every single job for by hand, which was well over 100 jobs that I applied for, writing an individual unique cover letter per job. And I think I maybe got two, like, responses back. And I don't know if there's something --
DAVE: That's a high hit rate. That's a high success. I'm not kidding [laughter].
MIKE: It is.
VIVIAN: No, I know.
MIKE: Like, wow, you got two [laughs]?
DAVE: 1,000 to 1 is the standard
VIVIAN: The two that I got responses from were people that I knew [laughter].
WILL: There you go. There you go.
VIVIAN: And so, like –-
DAVE: There you go...
VIVIAN: Like, at the end of the day, like, I put in a tremendous amount of effort towards this thing, tried to do all of the things right, at the end of the day, on my resume. Is there something to be said about the fact that we are moving to a place where the data influx is simply untenable to be processed by a person? And that no matter how much I want it to be written by a human because it gets that human touch...if they're requiring a cover letter, and they're the 40th job I've applied for that day, I don't have enough time to write a full-page cover letter to outline my experience that's tailored to this specific job. I need something that can do it faster, and also something that knows what an AI wants. I don't know what will be ingested and processed to be exactly what is wanted, but an AI is the AI that might be doing that.
DAVE: If your AI represents you well without doing triads and em dashes and looking obviously like an AI, that's going to tell me that you're good at managing your AI, and that's a job skill that we're hiring for. And if your AI makes you look like a drooling idiot, we'll go, "Hmm, yep. Keep trying."
WILL: That's a great point, Dave.
DAVE: Sorry, Will, I cut you –-
WILL: And so many people miss AI insights that you...sorry, I was trying to do AI speak. Yeah, I don't think it worked [laughter].
DAVE: You're right to be curious on this topic, yes [laughter].
WILL: I think you could look at it in two ways, right, and say, like, 100 applications, right? That's a daunting amount of applications. You can look at it a different way, right? And I would encourage people to look at it this way, right? 100 applications, an hour per application, that's 100 hours. That's two and a half weeks.
What? Like, I got bad news for you about the rest of this year once you get this job. Like, oh, it's going to get so much worse. That's a sprint, you know? Like, oh, I spent a sprint on my new job. Okay. You know, I mean, and, like, that sort of, like, I don't know, hardheaded, like, grindy attitude. Like, I mean, I guess that is, like, maybe symptomatic of somebody who's successful in the industry, like, you know what I mean?
Like, I've been doing this for a long time. I'll probably be stuck here till I die. That's the attitude that I approach it with. And, not only that, but, like, when they raise the bar, and they raise the bar, and they raise the bar, then I just...I look at it like, "Oh, well, look at all these people." Like, I don't have to outrun the bear; I just have to outrun all you guys, you know?
And so, it's just, all right, you know, like, okay, well, people gas out at 100 hours. Okay. So, I just have to go for 101 hours, and then all you people are going to drop out. All right. That's not even that much. Like I said, that's a sprint. That's a hard sprint. And, I mean, obviously, like, because of the pipeline issues, like, it will take you longer than that, because they don't turn around in an hour, then that's really the problem.
But I don't know. I mean, like, that's just sort of, like, you know, the bar will raise, and raise, and raise, and raise, and raise. But people are still people, and they will get tired. And as long as they get tired first, I win [laughter].
DAVE: Everybody needs something. The more people you talk to and ask them what they need, somebody's going to be willing to trade you a paycheck for it.
KYLE: I think, like, what we're kind of coming to, though, is she's...like, Vivian is like, "So, how do I make this more human in my applications and everything?" You go with the most human approach, which is you know somebody. That's kind of...I feel like the end of the story is like...not the end of the story, but the biggest part of the story is that's how you do it. Your human approach is you know somebody, be it somebody in your class, church leader, whoever, you know, whatever group you're part of, you know somebody.
I mean, my last job, well, I guess this one [laughter], I had the interview before I ever gave them a resume. Like, that was a formality, right? It's all about who you know.
VIVIAN: So, to kind of get a little bit of an overview, the general consensus of what I'm getting is tailor the effort to the audience. If I am spending my time creating cover letters and resumes for a company that's going to review my resume with AI and that's the only way it will likely ever be seen, let an AI do it, because that doesn't matter. If no one ever reads it, then it doesn't matter if it's handcrafted to look perfect and to feel authentic and to perfectly describe my experience in relationship to this organization.
But if I can make a connection, again, that's the human connection that transcends all of it. Because, I mean, I was talking to Balaji just this morning, actually, and he said something a little bit interesting, that his perspective is that humanness, like the ability to be human, is just going to continue to get more and more and more valuable as technology advances and becomes more and more human-like because that humanness becomes that key essential aspect.
And so, like, I don't know. You guys have talked a lot about networking, about finding these connections, a lot about, like, ways to optimize things. But, I don't know, my overall takeaway from it is, at the end of the day, I could hyper-optimize my resume to be perfect, and then I might get one job that may work out, or I can find somebody that I know. And the success rate and the effective success rate of that is going to be way more efficient than any sort of, like, mass application process.
DAVE: And likely to be a job you like, because you'll be able to...we always tell people you need to interview the company. And if you're talking with people and it's somebody that...it doesn't have to be...we keep saying somebody you know. Somebody you've met, and you've just, like, again, like, lunch with somebody is enough of a note. Met them twice at a URUG is enough of a note. But you've had the chance to actually interview their company, and it pays off in both directions.
KYLE: Even if you don't know them, introduce yourself. I've seen people do that, too. They're applying, I mean, personally, I think it's more worth your while rather than writing a cover letter. Look at who's in the department that you're looking to go into, and introduce yourself on LinkedIn. Try and reach out. See if anybody will respond, and, all of a sudden, you've got a connection.
WILL: Yeah. I mean, I suppose, like, you know what I mean? In, like, sort of a general sense, rather than, like, don't care, right? I would scale your effort into, like, an application process with, you know, your expected value of the thing, right? Like, if you've got a warm lead, right? You know, your old college roommate is on, and they're a project manager at some startup, and, like, they're like, "Hey, you should apply for this job," put a little extra in, put a little more into that, because, like, the odds of somebody beating that are pretty high. And, well, obviously, you know, you don't want to burn your contact, right? And if it's just sort of some, like, LinkedIn open to work, you know, cattle call, then we're probably looking closer to the bare minimum, right? And so, I mean, like, that'd be just to generalize that theory a little bit.
MIKE: Honestly, we might want to save more questions for next time, because it's...I think we've come back to the same conclusion. It is this people, this human aspect that's more important. No matter all this technology we have, that's usually what works best. Use the technology if you can, but, man, that's the lowest valuable, the least value way of going about it.
WILL: Yep. And I'm going to sign off with this: If you're listening to this right now, go and add everybody that you've ever worked with and everybody on this podcast. If you're listening to the sound of my voice, add me on LinkedIn. I'll add you back. I don't care [laughter]. Like, everybody on this podcast, add everybody. Add all of them. All of them. It costs you nothing. Everybody you went to college with, everybody, like, just look around. Stand up from your desk and look around the office, and every face you see, add them on LinkedIn. Do it now.
Why are you still sitting [laughter]? I'm joking. I'm joking.
KYLE: [inaudible 1:00:44] [laughter].
DAVE: Yeah. Yep.
MIKE: With that, until next time on the Acima Development Podcast.
This episode of the Acima Development Podcast centers on organizational complexity, using NASA's root cause analysis as a jumping-off point. Mike opens by contrasting the military's rigid hierarchy with the failed free-form communes of the 1970s, arguing that both extremes reveal something about what makes human structures work. NASA's finding was that technical complexity should not be matched with organizational complexity; simplicity is the path to success in both. Dave reinforces this with a Microsoft study showing that bug severity correlates with organizational distance between the people involved, growing exponentially with each layer of hierarchy separating them. The group discusses product-vertical "pod" teams, where cross-functional members own an entire stack and success is tied to customer outcomes, versus discipline-based teams like a dedicated database group. Justin describes guilds, a parallel structure where specialists across teams meet and share expertise, as essential for keeping vertical teams from becoming siloed, and Mike ties this to the Team Topologies framework of four fundamental team types with clearly defined, unambiguous roles.
The conversation shifts to how expertise is actually developed, with strong consensus that software engineering is fundamentally an apprenticeship trade. Will argues that engineering has always worked this way and attempts to replace mentorship with formal structures fail. Vivian, an intern on the call, agrees, noting that simply sitting in meetings watching senior engineers reason through decisions has taught her things she otherwise wouldn't learn for a decade. The group critiques code review as a mentoring tool (too late in the process) and advocates for pair programming instead. Vivian draws an analogy to the brain: intelligence comes not from the number of neurons but from the depth of their interconnections, so 10,000 employees who never talk to each other are just one person 10,000 times over. This leads to discussion of how leadership scales, with Kyle and Will emphasizing that approachable leaders matter, that "open door policies" can be declared but trust must be earned, and that policies made without consulting end users (like tool purchases) create dysfunction.
The final stretch becomes practical career advice prompted by Vivian's questions about building trust as a newcomer. The panel's answers converge on consistency: show up, do the work, take responsibility when things go wrong, and never pretend to know something you don't. Will offers his "secret silver bullet" for getting help from anyone: make a genuine effort first, document what you tried, and ask a specific question, and people will do backflips to help you. Dave adds the notebook rule (never ask the same question twice) and a rough heuristic for when to stop grinding on a problem alone: value your time like a senior's, and escalate when an hour of their help outweighs hours of your spinning. On Vivian's tendency to hyperfocus, Will and Dave both encourage her to "let it cook," arguing that obsession at her career stage builds lasting skill, as long as she's engaged and learning rather than stuck. The episode closes by looping back to psychological safety as the throughline connecting organizational design, trust, and team success.
Transcript:
MIKE: Hello and welcome to another episode of the Acima Development Podcast. I'm Mike, and I am hosting again today. With me, I have Dave Brady, Thomas Wilcox, Justin Ellis, Vivian Moore, and Kyle Archer.
And I'm going to start by sharing, not a personal story, but one I read. Actually, I'm going to share two things. Well, first, a little bit of context before my story. We've been talking for several weeks now, several episodes. We're going to continue on our final, final episode talking about the NASA root cause analysis that they did─I think it was back in 2012─talking about things that led to some institutional failures that they needed to pivot from.
And I'm going to start by talking a little about the military. I've not served in the military, and, frankly, I haven't wanted to [chuckles]. That's a tough thing to do, you know, all of the risk that's associated with it, all of the risks of PTSD, and so on. But I give respect to anybody who has served, because, you know, that's a real thing.
The one thing that the military does do is they have a very rigid hierarchy, so everybody always knows who they report to, and that's pretty universal, I think, in militaries, you know, successful militaries. They have this rigid hierarchy. And so, there's this question: why do they have such a rigid hierarchy?
And rather than answering that question, I'm going to share another thing that I read, and it's brief. I read...I don't know if story is the right word. I did an analysis of counter-cultural communes back in their heyday, back in 1970s. The opposite of the military hierarchy, right? As far from it as you can get.
And in one of these particular communes, there was one where they said, "We're not going to have anybody married to each other, free love, you know, and everybody just be with whoever they want." And they interviewed somebody, and he said, "There is nothing more lonely than every night having to find a new partner," you know, like, begging them to spend the night with you. And I thought that was fascinating.
In one case, you know, you have this rigid hierarchy with all the constraints and all of the potential downsides that come with that. It's hard to be in an environment with such a rigid hierarchy. I report to this person; I do what I'm told. But on the flip side, if you have no hierarchy and no commitments, it can be even more crushing. And almost all of those societies have fallen apart. They were experimental, and they fall apart because there is that lack of commitment. And the social structures just deteriorate, you know, you don't see very many.
There do exist some communes, but most of those experiments that happened in, like, '60s and '70s have fallen apart, and they don't exist anymore, or exist only at extremely small scale. And the ones that did tend to exist are those that look very hierarchical, much like monasteries, where you have people with a strong dedication to a cause, and, you know, single-minded, and not that kind of social conflict.
Once again, they've ended up looking more like the military, and that's fascinating. It says something about what is effective in running human structures. Also, my heart broke for that poor guy, the loneliness. He said, "There's nothing more lonely..." and you picture just kind of begging for somebody to care enough about you, and you have to do that every day.
Coming back [chuckles] to this NASA study --
JUSTIN: I was thinking of how you're going to relate that, you know, begging for somebody to relate to you at work, or something like that [laughs].
MIKE: No. So, NASA's final element in the root cause analysis...and I'm going to read this just straight verbatim from their report, because I think it's interesting.
They said, "The final root cause is closely related to the fourth and is driven by the technical and organizational complexity. Space systems are highly complex and very sensitive to small uncertainties. This complexity requires in-depth penetration and understanding, where small technical glitches result in major problems. The technical complexity leads one to think that the technical complexity must be matched with organizational complexity."
I'm going to pause there for a second. So, they have this innate technical complexity, right? You're building a rocket. It's complicated. And you might think, "Okay, so my organizational structure needs to be complicated." You have 20 bosses, right? And I go back to what they said, "We know that when managing the organizational complexity becomes as large as or larger effort than managing the technical, then the product success is in question. Simplicity is the pathway to product success, both technically and organizationally."
Now, we tend to focus a lot on technical simplicity. Well, we know the simplest solution that's almost certainly the best. But we don't always think about the organizational simplicity.
DAVE: Mike, I love that you hit on this. Last week, when I hosted when you were out, one of the things that I mentioned is that, at Microsoft, they did an in-depth longitudinal thing on, like, what's likely to cause a bug and how severe and difficult is it to fix? And the dominant variable was how far away in the organization the person who fixes it is from the person who wrote it, or the two modules that have to interact.
And they measured it by how many layers up the hierarchy you have to go to get to the common person to take you back down to approve this other person working on your thing, and it's exponential. Somebody on your team, pretty straightforward. Somebody on the next team over, kind of difficult. Somebody in the next department, very, very hard. And if it's in another business unit, forget about it. You might as well just open a ticket and pray.
MIKE: You're saying if you don't have that person that you know and trust and can rely on, things fall apart.
DAVE: Mm-hmm.
MIKE: It's fascinating to me how much human aspect there is to software engineering. We have talked about technical stuff. Sometimes we've gone deep into unit testing here, right? But here we come back, and this is going to be a very human-focused episode. We're talking about that idea of organizational complexity driving, as they said at NASA, product success. Product success is in question when your organizational complexity grows.
So, Dave, you mentioned the study at Microsoft [chuckles].
DAVE: Yeah, it's straight-up complexity and the hierarchy. And if you think about hierarchy like a snowflake or a star graph, it becomes fractal, right? You've got these two great big branches, and then lots of little medium branches, and then each one of those has fine branches. And by the time you're down four levels, it's just, like, fine little hairs. And when one little hair over here has a bug in some software written by a little hair over there [laughs] and you have to get the head brain completely involved in...
At CMM, we had that one route between engineering and the database team. You had to go to the C-suite because the DBA was...sorry, the VP...what are they called? CTO, that's the word, was a founder and owned part of the company. And so, the CEO had to be polite. He couldn't just go in and say, "Fix this, or give them what they need." It was like, "Well, I've got to go have an argument." So, you had to go all the way to the CEO, and it definitely impacted things. There was a lot of ceremony between engineering and database because there had to be. They were basically in a different company. They had completely different objectives.
JUSTIN: Wow. So, when you talk about organization, and I remember when I was at big organizations, it was organized like that, that there was a database team. There was a front-end team or a back-end team. And they often didn't play well together because they, frankly, had different objectives, and those objectives weren't necessarily product or customer-centric.
Some places I saw, they were like, "Oh, we're going to make, you know, our whole goal is to make our database normalized," or something like that, or optimize space, or optimize cost, or something like that. And that didn't seem to work out very well in those cases. There was a lot of bureaucracy, and that's where you're going up the chain and coming back down the other chain and trying to communicate and things like that.
Whereas other places I've seen that were very successful, and maybe this contributed to their success, but they owned a product or owned a feature in the product of the company. And their success was determined by how successful that product was. And that product was a vertical stack: frontend, backend, database, you know, even all the way back to backup, and everything else like that.
And so, all these people with different disciplines were all moving towards the same product, and their success was based on that product. And there was a leader who was leading that group: frontend, backend, database. And that person kind of had to have everything in their mind because they owned the entire stack.
And it seemed like those types of situations were less bureaucratic. You know, you could get answers from one person about that product. And so, the C-level, the CEO suite, they could go to that one person and say, "Hey, what's up with your product?" And that person knew exactly what was up with their product because they were focused on that product's success.
MIKE: That's interesting. We're actually doing some efforts at structuring our department in engineering to map into customer outcomes here at Upbound, and not just the same level across the whole Upbound group, for much of the same reason. We want to be focused on what matters to our customers, customers broadly. We work with customers and partners. Both of them are our customers.
And the goal is to align those experiences, make it simpler. You go to one person, "Hey, what's going on with, you know, customer acquisition and, you know, with the customer application process? Is that working right?" You got one group that's all focused on that. That's what they care about. And we're very early in the process, but that's the goal.
I'm curious because I think that every set of lines you draw in an organization, you know, every way you divide it up is a different set of walls that isolate people from each other as well. Every inclusion is also an exclusion, so, you know, there's trade-offs with all of those. And it sounds like you saw a lot of success with that, Justin?
JUSTIN: Yeah, and I'm glad you mentioned those walls because you do want to take advantage of all the database people, like, working together in some sort of unified architecture. Like, you don't necessarily want, you know, different databases used in different parts of the organization because then you have unnecessary complexity that doesn't necessarily cross team borders. But sometimes you do want different...I don't know. You want that discussion. You want all the database guys talking to each other.
And I think...I can't remember if Acima had this, but it was the guild structure, basically. All the database guys were part of a guild, and they got together once, you know, twice a month, or something like that, where they would talk about what was going on in the database. And there was a front-end guild, and all the front-end guys would get together and talk about, you know, the different architecture decisions. "Okay, today we're going to start using this new TypeScript library, and we want to use it across the organization, and here are the benefits, and here are the minuses," and things like that.
And so, there was, like, this...not necessarily shadow but another structure that wasn't your manager structure, but it was a kind of professional structure where you can get together and talk about those things that...And you could present at it, and you can make proposals. And it was actually really kind of a fun way to be part of the company that was, like, outside of your product and, you know, grow professionally. So, it was kind of a cool side channel.
MIKE: It does sound fun. You know, it's great to be part of a thing. "Hey, I'm learning. I'm doing stuff together." Do you feel like...Well, and I think it's almost inevitably true. Do you feel like the structure and maintenance of the database suffered in that kind of environment compared to one where you have your...how do I put this? I'm thinking about the Soup Nazi from Seinfeld. You got that guy for the database, you know.
JUSTIN: [laughs]
MIKE: I don't want to make a Nazi comparison because...Nazis.
JUSTIN: [laughs]
DAVE: It's a...kids, that word did not used to be problematic [laughter]. It wasn't as awful. It used to be humorous [laughter]. The word used to not be problematic, not the actual people.
MIKE: It's like, I think it's just pretty much always been a problem with Nazis [laughs].
DAVE: Mm-hmm. Fair. Fair.
MIKE: That idea of, you know, having somebody who militantly protects their domain means that's protected. But [chuckles]-–
JUSTIN: In my experience, it does depend on the leadership of those groups, you know, and depending on if they're a very strict leader or some...I don't know, it depends on their focus, right?
But the most success that I've seen is the second one, where you have groups focused on a product vertical and a guild that exists on the side, because it puts your focus on, hey, we need to, like, do stuff that makes the company money because we all want to get paid at the end of the day. And the better we can make the product, the more likely we'll get paid at the end of the day. Certainly not a guarantee, right [laughs]?
But you get that focus of, you know, hey, what I'm doing has actual effect to customers and to the bottom line of the company. And I think that is actually a really good motivator for people. And that motivates me certainly, is like, hey, what I'm doing is making life better for my customers. It is resulting in a better bottom line for the company. And, hopefully, I will be recognized by my leadership team and, you know, paid better for X, Y, and Z reason.
But, you know, that is a big motivator for me, and I think it's a big motivator for a lot of people, where they can do something that causes a direct effect. And you can see that, in my experience, if you're part of a product vertical and you're all pushing together as a team.
MIKE: Absolutely. I know I feel that way. It's different from saying, "Yeah, I think that my work helps customers," than saying, "I helped Jenny," right [laughter]? I talked to that...I know there's that person that I know. There's a person that I helped. And even if you're not working individually with those customers, you're talking to people who are talking to the customers, right, if you're focused that way. And it does, it changes the way you think about your job. I agree from my personal perspective.
JUSTIN: Yeah. So, going back to your complexity, initial complexity proposal, where NASA found that, like, the more simplified structure, I don't know which is better in terms of that. Is the product vertical more simple, or is, you know, all the database guys together more simple?
I think the product vertical is more...I don't know. It could get more complex because you can get siloed really easily on the product vertical. And you could go away from, like, what the company decides in terms of technology. And so, you got to be really careful, if you are going that product vertical focus, to have a strong guild system that you are all in agreement, or at least you're having very good discussions about what technology you want to implement at that company.
MIKE: Fascinatingly, I was having a conversation about team structure earlier today, and I pulled up a quote from a book about it. And I still got it up on my phone [chuckles]. Team Topologies. I love the book. It's been...I read it several years ago, and I go back to it all the time because, working with an engineering group, you think a lot about, well, how do we organize, and what are the pros and cons?
We could probably do a bunch of episodes talking about the content of that book. But, in particular, they talked about the kinds of team types that tend to form. And they call out that there's four kinds of teams that tend to effectively form, and they say something about how those should align. And this goes to this organizational complexity. I think it's probably worth reading.
They say the four fundamental team topologies: streamlined: that is, like, these verticals we're talking about. Enabling: that is a team that's dedicated to helping out the other teams. Complicated subsystem: it's when you have a team that, you know, you've got that really messy old system [laughs] that you just have to manage independently because only a few people have the expertise. And platform: platform are thoughts that, you know, we have something that supports everybody else. We're the foundation everybody else builds on.
They say those four types should act as magnets for all team types. All teams should move toward one of these four magnetic poles. That is, we should prefer these types and aim to adopt the purpose, role, responsibility, and interaction behavior of these fundamental types for every team in our organization.
Simplifying the types of teams to just these four helps to reduce ambiguity within the organization. As was identified by Zhao Luo and colleagues in research published in 2018, "Reduced ambiguity around organizational roles is a key part of success in modern organizational design." See, that ties directly to what we're talking about here.
A large or mid-sized organization is likely to have one or more teams of each fundamental topology. Multiple stream-aligned teams are the starting point, but an organization may also have several platform teams, a few enabling teams for different purposes, perhaps one addressing CI/CD and a second addressing infrastructure architecture, and if strictly necessary, one or two complicated subsystem teams.
That's what Team Topologies has to say about this. I would like to call out, you say, "Well, is that right?" Based on the research, and they said, "Well, yeah, you should have something like those vertical teams, and some other teams that support that. But the important thing is that you have clearly..." So, I'm going to repeat their quote here: "Reduced ambiguity around organizational roles is a key part of success in modern organization design."
So, you clarify what the team is responsible for. You make sure that you have supportive teams supporting those verticals that are supporting the customer. And you make sure everybody knows what their role is, so it's unambiguous. That's their recommendation for reducing organizational complexity. So, there's a take. Thoughts?
DAVE: I have a thought, but it's a really weird fractal jump into software design, so I want to give people a chance to talk about teams for a minute, because it's going to be, "And then Dave took the podcast and departed the text [laughs]."
MIKE: Well, you haven't read the book, but the fundamental premise of the book is that you cannot separate organization from software design.
DAVE: Mm. Okay. Fair. Fair. Well, let's dive into it then a little bit. One of the things that I've been on about lately is something that I originally got from Katrina Owen and Sandi Metz, which is that good refactoring will reorganize your software around ideas. And then when you read the code, you can literally watch the idea moving through the codebase, which is great.
But by default, we tend to refactor to structure. We pull out, oh, this code is identical to that code. Let's extract it and put it in a method. Well, you just split an idea in half, and you put one half here, and you put the other half over there. And these two things get to use their half of that idea, but they get to share this idea, and it's lost. You've lost the idea.
Sandi Metz makes the radically heretical claim that nobody knows what maintainability is. Nobody has a good metric. You come up with a good metric for this, and you will eat rich in the software community forever because we still haven't solved it. And there's good money being made attempting it.
But she uses kind of an unmeasurable heuristic, which is understandability. If I look at a piece of code and I can quickly understand it, if I look at a line of code and the lines around it, can I see what's different between these lines of code? Can I see what's similar? I talk about this with, like, RSpec, you know, like, block of code, you know, a spec, a spec, a spec, a spec. I love having a lot of repetitive code that you can look at and go, "Oh, this part is all the same. That is the one piece that's changing," and, "Oh, look, that's the name of the spec."
Refactoring to understandability is hard to do if you refactor your code to platform, to hierarchy, if you basically say, "I'm going to write the database layer." And the reason we end up saying...this is where it gets really heretical on my part. If you have requirements coming in, somebody went out and talked to the customer, or to the CEO, and they said, "We want to do this." Then they went off and had a meeting with one of the tech leads, and they engineered up a solution to it. And then that gets handed down to a team lead, and it gets divided up into Jira tickets.
And it finally gets to the developer, and the developer says, "Well, what about this?" And nobody knows, right? It's a waterfall, right? This waterfall design thing. And the developer says, "Well, what about this?" And the developer was the one who spotted that we're building the wrong thing, and we don't know it yet because we're downstream waterfall.
What is my defense as a developer when that happens? I'm going to step back and say, "Well, I'm going to write a generic database adapter that will solve however business wants to do it, and now I'm writing a platform." And so, years and years ago, I told people, "Don't write a framework. If you think you're writing a framework, you have had way too much caffeine, and you are probably solving the general case of a problem that you don't know the solution to your specific case yet. Go solve your specific case."
And the specific case, that's your vertical line, that's your stream-aligned teams. I'm going to say the word pod, because that's a buzzword here at Acima right now. Cross-functional teams, you know, there should be somebody in your pod, in your working group, who can say, "Yes," to any problem that team has. "I need to fix this in the database." "Got it. I have the access to do this." "I need more network bandwidth." "Got it. I'm the DevOps guy in this pod."
Make it so that there's nothing that comes up that everyone can say, "No," but nobody can say, "Yes." Make it so that people can say, "Yes." That pod is a team structure, and it is the exact opposite, in my opinion, of a platform team. It's almost like a vertical slice. Give me one person from each layer of the cake, and now I'm going to send them off. And you guys have passwords to everything you need. Go make some business happen.
That was a long ramble down a kind of a weird thing, but that's what team structure to platform lived in my brain.
MIKE: And you didn't say that there shouldn't be platform teams.
DAVE: No. Yeah, there actually should be. Just every team in the company should not be a platform team. That would be one thing. There's four types. Please don't take this and say, "Well, pick the one you like, and now everybody's in that team." That would be a terrible decision.
MIKE: No, there's strong emphasis there that you should have most of your work organized around customer outcomes. And then you have dedicated people toward the systems that need to support that, you know, as support, right? As support for the main work that you're trying to get done.
WILL: I'll tell you, like, I've worked at...I worked someplace where they had implemented something very similar to that, and I like it in general, right? Because, like, right now, I've spent, like, literally my entire day, like, knocking heads with people trying to be like, "Hey, I need end-to-end ownership of this thing. We can't just pass it off and say, like, "Well, my part works, so I'm going to write a ticket for this team, you know [chuckles]. And I'm going to write a ticket for this team, and I'm going to make this other team validate it. And I don't know or care whether, like, the actual soup-to-nuts process gets done. I'm just going to do my end and bail." And I've been twisting arms and hurting feelings all day long.
But one thing that I did notice where...I worked someplace else where they did these domain teams, right? And they had a representative from each of the subgroups, right? So, you could solve this domain problem, soup to nuts, and everybody could do it. And one of the issues that we ran into, and just, you know, what I thought of, because I like the idea, but one of the issues that you have to deal with is, it's really hard for a lot of developers to succeed in that environment because you are an army of one.
If you're working on a platform team, you've got other people who know the platform, and you could reach out. And you can ping-pong around, you know, you could bounce ideas off them. Like, "Hey, have you seen this?" or that sort of, like, collective knowledge around a platform team. But, like, if you are a mobile app guy, right? And I've got a web front-end guy, and a DevOps guy, and a web back-end guy, and a mobile app guy, and we're all doing our thing. Well, if you don't have the answer to the problem flat out in your head, if you're an army of one, you're in a really tough spot.
And if you are a junior developer looking for mentorship and you just sort of get parachuted in with a bunch of senior devs, where it's just like, "Hey, buddy. Where's my backend? Hey, how you doing? This thing's not scaling very well. We're getting a lot of outages. What's going on?" And you're sort of like, "I don't know," you know. And so, that sort of development becomes really difficult because you are...I mean, like, I love the idea of a pod, right?
I love the idea of bringing together the people you need to deliver a customer outcome, right? That I love. But you're never going to be out of the platform model because you're always going to have people. You're always going to have senior platform engineers that ought to be reviewing your code, right? I, as a pod member, I don't know, looks good to me. Ship it. It gave me back a 200. Let's ship this thing. Let's go. And, you know, the platform might have more refined [laughs] and subtle requirements for a good outcome than, like, I don't know, man, 200, let's roll [laughs].
MIKE: One thing interesting that Justin brought up before you were able to join, Will, is the idea of guilds that he's seen in a previous company he was at, where the people who had a specific focus, like all the database people, would get together in this guild and meet periodically, so they could swap expertise. They're the person you can reach out to, so you don't have that army of one problem so much.
JUSTIN: Yeah. And related to that, there was the guild channel, and I think this was actually more important than the periodic meetings, is, like, there was an active guild channel on Slack, or whatever communication channel you have. And people were chatting there every single day, multiple times a day, and they were helping each other out.
And it goes back to...there's kind of two types of leadership that you have in a company. You have your managerial leadership, and then you have the technical leadership. And the technical leadership exists in a guild where you have guild leaders who are ensuring that your guild is healthy and that there's communication and that they reach out to people who are struggling. And I would hope that a company that has that sort of structure, if they're trying to understand how things are actually getting solved, that they would look at both of those channels, and they would understand that the guild leadership is almost as important as the product leadership.
WILL: I mean, in the end, somebody's got to review code, you know. Like, you got to have a code review, and so you have to have, like, I don't know. I hesitate to use a code review as a sort of, like, leadership, team building, development sort of exercise. Like, it's fine, and it's good, and it's necessary. But it's not really a substitution for, like, okay, I am going to mentor my junior developers via code review; that's a pretty...I don't know, not -- [inaudible 30:14]
JUSTIN: Yeah, I --
MIKE: It's a little late in the process. It's late in the process, right? If you're mentoring after the code's already written, then you've missed an opportunity.
DAVE: There's a soapbox that I've been railing at with our team, which is we've got some faults with our agile process. Our standup meetings run really, really long because people are just belaboring all of the status that they're currently in. "I worked on this, and I'm having this problem. What do you guys think about this?"
And I'm like, "Tell me you're not pair programming without telling me you're not pair programming." Because if you were pair programming, everybody would know what you were doing, and all you would need to report in standup is your coordination. "I'm going to touch this module today, and I'm going to deploy it." And somebody else can go, "Oh, I'm touching that." That's standup. "Yesterday I did this. Today I'm doing this, and I have a blocker with this." And that's it. You don't need status.
You can look at so many things...and, like, mentoring the new folk, if you're not pairing, then you're mentoring them in the least efficient way possible, which is code review. It's, you know, "Go spend days writing this; check in some code, and then we'll tell you what you did wrong, and you can cycle back." Versus, "Sit next to me, and let me show you how to peel a clean shaving off of this piece of wood," you know. I switched from code to woodworking, but you get what I mean.
MIKE: Yeah. Well, you say woodworking. You switched to talking about an apprenticeship-type structure for teaching.
DAVE: Yeah. Yeah.
JUSTIN: So, my first real job, the day I turned 16, I actually got hired by a woodworking shop. And, man, I was at the bottom of the totem pole. I was sanding every single day. I'd come home with, like, sawdust everywhere, and it really sucked. But I was working with people and learning, you know, hands-on with people, literally hands-on. And it was a good experience with that regard because you really got to know that other guy. And, like, you started to learn how to fit pieces together and what good wood looked like and how to stain. There was so much staining, oh man [laughs].
But I wish we could go back to...that kind of pair apprenticeship is applicable across all sorts of industries, especially the software development industry.
WILL: I believe...it is my, like, sincerely held belief that the software industry has been like is now, and always will be intrinsically...engineering in general, right? Engineering in general is that way. It's always been that way. You can study on your own. You can have formal coursework, like, all that stuff, right, to prepare you for your apprenticeship. But it's going to happen like that, no matter what.
And I think all the attempts, I've seen many, to, like, make it be another way have all failed dismally. And sort of if you always...I mean, this is a core belief of mine, a heuristic, if you will, but like...what do you call it? What's the word? What do I want to say? Sorry, I completely derailed me. [crosstalk 33:37]
MIKE: Apprenticeship, blacksmithing --
WILL: But, like, accept it, right? Well, I mean, just accept it. Accept it, and make it work. Like, accept it, and make it go. I don't think there's another way around it. If you embrace it, you can do it efficiently. And if you do it the stupid way and you try to pretend it's some other thing, then you're stupid. You're doomed to failure. You're just going to repeat and repeat and repeat forever.
MIKE: My experience has been deeply aligned with that. I've seen lots of empirical evidence, in that I've seen a number of engineers, very successful engineers, come from non-traditional backgrounds that worked closely with senior engineers who were mentors, learned the ropes through that experience, you know, watching and learning together in partnership with others, and they got really good and went on to be extremely successful.
I've worked with people with PhDs who their career never really took off. They've got all of that technical background, but it didn't go very far. And I've heard that lots of times. That apprenticeship aspect in the industry is absolutely true. That's deeply aligned with my personal experience.
I'm curious, so Vivian is here with an internship. And I'm curious, Vivian, if you don't mind, you know, how much are you learning from, like, shadowing, working with other people, you know, over the last...what's it been? Couple weeks now? Two, three weeks?
VIVIAN: Yeah, it's been about that long. Oh, sorry.
MIKE: Yeah, no, go ahead. What's your thoughts about this idea of apprenticeship and what you learn there?
VIVIAN: I distinctly wish that apprenticeships were more common, more accepted, and a more natural way of moving through an industry. I think that, especially recently with the kind of addition of AI and the effects that it has had on kind of entry-level programmers' ability to be effective and actually contribute towards an organization, I think that the entire industry would benefit from apprenticeships being used.
But especially, I think that lower-level engineers could gain valuable experience that they wouldn't be able to learn in their careers until a decade in, unless they have that mentor to go to to ask questions, to watch do things and see them do something different in a way that they never would have thought to do because they don't have a decade of experience in this industry.
The amount of things that I've learned just from sitting in meetings and sitting there like a fly on the wall, watching things happen and seeing these discussions happen, and understanding, like, the thought process that these senior engineers have of why they're making the decisions that they're making, that has been invaluable to me, even beyond the amount of, like, value that it brings by having someone with a lot more experience that I can go to to ask a question while I'm in the middle of working on it. That close tie between someone with more experience and less experience, I don't see a way of ever replacing that, and it is incredibly invaluable.
MIKE: There you go. Thank you [laughs].
DAVE: I like her. We should keep her.
VIVIAN: See, that's what I'm saying.
MIKE: [laughs]
JUSTIN: You know, we're hiring an intern. If you get tired of over there, you sound like an awesome person [laughter].
DAVE: Hey [laughter].
MIKE: Mine [laughter]. So, you know, we've been talking about organizational complexity, and we've gotten...We took a little bit of a detour here [inaudible 36:56] apprenticeship. I think that there's connections here though, right? We said that knowing who to talk to matters, and even just sitting next to somebody and watching them do their work may teach you more than a formal hierarchical structure, you know, a formal teaching program to teach you stuff. Because that line of communication is so critically important. It seems like it all connects here. It's just that, yeah, you want to get things done? Make sure somebody knows who to talk to. Make sure that that's easy.
VIVIAN: I --
MIKE: Go ahead.
VIVIAN: This feels like an analogy to the human brain, and the way that it functions, and the ways that neurons on their own are more complex than most kind of virtual neurons that we create for a lot of machine learning applications.
So, they are more independent than a lot of times we give them credit for. But at the same time, the quantity of neurons in a person's brain is not even close to a good measure of intelligence. The one thing that can be considered a good measure of intelligence when it comes to neurons is the amount and depth of those connections to the other neurons around them.
DAVE: Interconnections.
VIVIAN: The interconnectedness of the brain is what produces the intelligence itself. It's not the individual power of each neuron or the number of neurons that are operating. So, you may be operating an organization that's a brain of 10,000 people, but if those 10,000 people never talk to each other, it's 10,000 individual people. And there's no power behind it because they can't do the work of 10,000 people meshed together.
DAVE: You have 1 person 10,000 times.
VIVIAN: Yeah, exactly.
WILL: I mean, in the end, like, I spend so much more time figuring out what to do, and how to do it, and who to talk to, and how to horse trade, like, to get my stuff fixed and all that. I mean, the actual programming development, like, quote, unquote "engineering" that I do on a daily or even a yearly basis is ridiculously low. Maybe I shouldn't say that out loud. I should be [inaudible 39:00]
MIKE: It's why AI only boosts, like, 20% on your productivity, because that code writing is only 20% of the job.
JUSTIN: Yeah. So, we're doing spec-driven development, and it's literally writing specs. And, you know, we're following a template. We have architectural context and everything. And we fill out this spec, and it goes off and does the thing, come back the next morning and check to see if it worked. And the day is spent writing specs.
WILL: Interesting. I don't know. At this point, mostly I feel like an old jazz musician, I mean, in that, like, I just sort of sit down at the piano, and I'm like, "Just start humming, and I got you," you know what I mean? "Just count me in. It'll be fine [laughter]," which is how product, you know, where I'm at, sort of likes to develop their specifications and their features. Anyway, so it's just, like, it's been okay. Let's not pretend that this score is anything but a suggestion, and let's jam [chuckles].
MIKE: The most popular book of scores for jazz musicians is called The Real Book, which is a joke, because it's actually a fake book [chuckles]. It's for faking that you know what you're talking about [chuckles]. We have to fake our way through things a lot, and you get there by having practiced a lot [chuckles] on what you're doing, which you might have learned through an apprenticeship kind of program, sitting at the feet of elder musicians.
WILL: Man, I wish. They just...I've got a good poker face, with just handy stuff, and I'm like, "Yeah, okay. Sure, sure [laughter]. [inaudible 40:55] [laughter]." I don't know what to say, you know, fake it til you make it.
MIKE: I don't think the world knows what percentage of software engineering revolves around Google queries and poking around to figure out [chuckles] what you're doing.
WILL: I mean, just the ability to just sort of sit in a chair and wait until it works.
DAVE: And remember, if you're going to learn to code from Stack Overflow, the answers. Go off of the answers, not the questions [laughter].
WILL: If you're going to learn how to code off Stack Overflow, you got to remember...I mean, and this doesn't even matter anymore because I think we sacrificed Stack Overflow for --
MIKE: We did.
JUSTIN: Yeah, I don't think there's [crosstalk 41:44]...I don't know how many questions they have right now per day, but I am certain it is, like, several orders of magnitude less than, you know, 10 years ago.
MIKE: I saw some stats recently, and it's absolutely collapsed.
WILL: It's crazy. Well, it's always the second answer, or it used to always be the second answer [laughter]. The first answer was the guy who had a lot of free time. And then the second answer was the guy who came in and was like, "What did you write down? What are you doing [laughter]?"
DAVE: Stack Overflow heavily incentivized being first, and there were people that, like, routinely would just camp. And so, they would see a question come, and they would put an answer on it, and it was literally just to be the first one in there. And yeah, it...the second answer [laughs] is the right one. Second mouse gets the cheese.
VIVIAN: Not to mention the effect of the most effective way to get the information you need on the internet is to first state it wrongly [laughter] because then the person who knows you're wrong comes in and corrects you because they cannot stand that you're wrong.
WILL: It's tough. Like, there's definitely a major cognitive distortion that, I think, we as a society are working through, in that, like, social media has empowered people with a lot of time on their hands to have an outsized impact on the narrative. And if you imagine your own life, what the kind of person who sits around and posts on Reddit all day would look like, and how reliable a narrator or life advisor that person might be, you might run into...you might have to sort of have some fairly serious questions about, like, where we're getting our information from and what that's doing to our thought process. This is my old man shakes fist at cloud moment [laughter].
MIKE: So, bringing it back to organizational complexity, we've talked a lot about the...[chuckles] yeah, about the...well, and it's...we went deep into this idea of, well, you were talking about expertise and where it comes from, how we learn. But we got down on that detour because of how critical it is to have somebody to learn from.
And if you're in vertical teams, you might not have somebody else directly on your team who knows what they're doing. So, this was reinforcing the critical importance of those guilds, some sort of mechanism by which you can collaborate with people who know what they're talking about, and grow and learn together, that may be independent of the formal team structure. And that's something that I don't know that I had thought through. I'm almost certain that I hadn't thought through recently, at least about how important that is. And, I don't know, I've got some takeaways.
Well, any other thoughts about what we should do to, you know, organize simply so that we don't fall into failures because the organizational structure's too complex?
KYLE: One thing that's been on my mind as we've been talking about a lot of this is policies within the organizational structure and who's making decisions where. Because we have cases in the organization, say at a platform level, where you'll be told to use X, Y, Z tool. A small subset of people will have said, "This is the tool that we need to use," without talking to the rest of the org. It turns out this tool does not work for the rest of the org. But it's the tool that we have chosen, and that's what we're moving forward with.
And how often that happens, especially as an org gets larger, like, that happens way more often than it really should, right? And we don't have these guilds, I guess, that maybe would solve this, if that would solve it. But I'm saying policy because they're not required to talk down. They're not required to communicate with the end user to determine what it is that they actually need.
They get a spec or a requirement that says, we need X, Y, Z tool. And they go out, and, you know, the first vendor that takes them out golfing is the one that we end up with, you know? And that feels like it's rarely the tool that would make us most efficient. And if we talk to everybody in the trenches, we might find out that another tool, and maybe even an open-source tool, or, you know, whatever happens to be in your field, is the more appropriate tool for the situation.
MIKE: Interestingly, our new CTO¬¬─I say new; he's been here a couple of months─he's been involved in some of the conversations recently around sourcing products like that. And his approach is generally to assign somebody, you know, to go talk to three vendors and bring in somebody from the team that is going to have to work with them. And I love that [chuckles]. I love that, because it gets that perspective. It enforces the comparison, right?
And he also tends to have a pretty short, you know, "And get back to me in a couple of days," so that you get that quick feedback loop. You don't end up stewing on it for too long, and then you can, you know, regroup kind of the agile approach. Get feedback quickly, and then maybe there's going to be follow-up questions. "Well, okay, so this is what you found. You know, based on that information, do you need to get more feedback and go do another couple of loops?" But always has somebody who is closer to the end user, right?
So, they were looking at some security tools, and he brought in somebody from engineering, so not from the security team, somebody who's going to actually have to work with it and have them talk to the vendor. Love that. Very helpful. And it goes along with that idea you're saying, that you should try to bridge those gaps, actually get the end user and the vendor closer together so they can see where those problems are or advantages.
VIVIAN: So, that makes me curious. I mean, I feel like that's a very, very good idea, not only because you get that end user close to the provider and the person, like, what the person's going to be interacting with, close to what they're interacting with, but it also helps to eliminate some of the bias that comes with, like, yeah, being taken out to golf to get shown what the best product is.
But I'm curious, as an organization grows and the distance between the leadership that needs to be making these end decisions...because they will affect the whole company and the end users of this product...when there gets to be more and more and more kind of middle managers between these people, how do you connect the right person at the top with the right person at the bottom, or vice versa? How do you bridge that gap as an organization grows and the number of connections grows exponentially relative to the number of people?
KYLE: One thing that I would say is your leadership needs to be comfortable to talk to, comfortable to communicate with. Because I've gone through both, you know, leaders that I felt comfortable communicating with and those that didn't. I won't say who, but I've been under leadership where I really felt as though if I wasn't his direct report, he did not want to communicate with me, and he had no reason to. And I feel like that will kind of break down as you grow larger; that breaks down really fast. So, having a good leader.
And as Mike has brought up, the new CTO, he's very much, you know, I want to communicate with the trench people as much as he wants to communicate with his direct reports. So, I think that will allow us to expand and adopt some of these new ideas like Mike just proposed, or said that we're doing something.
MIKE: Yeah, yeah absolutely.
WILL: I don't know. I mean, the whole...the concept, right, of, like, the management of managers, right? I mean, that's an enormous...I think it's an enormous task, and I, you know, it's more art than science. And, like, there's no...I don't know that there's a scalable way to continue doing that, I mean, you know what I mean? Like, how do you make that work? It's an art, and there's a reason that really capable senior leadership gets paid so much and is so hard to find, but everybody's got a manager.
MIKE: Yeah [laughs].
WILL: I don't know, above my pay grade, in all honesty. I have opinions about it, and I think, if you want to scale it, I don't know, like, it's pretty far afield from my, like, my direct experience. But, I think, if you want to scale it, I think disempowering managers is the biggest issue because, like, in all honesty, I think the biggest thing that...where things come unglued most often, in my view, is because "I said so," right? Which, if you're signing the check, then okay, that sounds like a pretty good reason to me. But the processes all break down.
But if you had one of these leaders where you didn't necessarily have to be in his org...I think I know who you're talking about. We don't need to get into specifics, but, like, this person was an unpleasant person to work with. People want to be good at their job. People want to have an impact on their job. People want to make money at their job. People want to be experts. They want to contribute. They want to have impact.
And if you have a leadership that's good at driving those things and you, you know, say, "Hey, if you go out and you work on these things that are doing good for the company," you know what I mean? And that affects your direct compensation. That affects your, you know, ability to advance. That affects, you know, like, all these things, you know what I mean? And you say, like, "Well..."
But the managers, hey, you got to convince them to work with you, right? And it's not just, like, you were assigned. You're my chattel, you know? I own you, and you, and you, and you, and you, and if you want to work here, you're putting up with my crap, you know, I don't know. It's a [inaudible 52:10] I mean, just like the pod model, right? Everything, you know, it's a good idea, and it breaks down in a different way.
But, I think, that sort of, like, where you see bad leadership, it's people you don't want to work with. And if you give people an out, if you give people an off-ramp to get out from under somebody who sucks, they're going to take it. And if I'm the CEO and I see, like, nobody wants to work with this VP, like, this VP completely sucks. He's bleeding team members everywhere. Are they a leader, or are they just another manager?
MIKE: Yeah. I'm going to go --
VIVIAN: The common thread that I'm kind of seeing between at least those two responses is, like, there needs to be a level of not only comfort, but trust, especially between, like, a worker and a manager, or a manager and an even higher manager, however that structure works.
Because, I don't know, going back to that brain analogy, if you're sending signals to this other neuron and that other neuron is just doing nothing with it, or doing something with it and it's destroying your brain or destroying the system overall, or not communicating back, or ever getting to a point where you are able to see the results of the effort and time and work that you're putting in, you lose that trust, and then you stop sending those signals that are necessary for the brain to function.
And correct me if I'm wrong, but this kind of, I don't know, every single connection that builds the organizational complexity or organizational efficiency and effectiveness without building complexity, requires that trust in that communication boundary.
MIKE: It's not just complexity. It's cohesion.
VIVIAN: Cohesion, mm-hmm.
MIKE: Yeah, cohesion. You know, we've talked a lot before on the podcast, in the past, about psychological safety, because research has shown it's the most important indicator of team success. Here we are landing again [laughs].
Well, we started with organizational complexity, make your people feel...make, not just feel, make your people safe. You know, make it a place that people can actually talk to you. And if you're not doing that, you're not going to succeed. Yep.
You talked about how you do that as manager of managers. I had somebody tell me once, and you've likely heard something like this before, but it's really stuck with me. When you're considering a romantic partner, look at how they treat their pets.
DAVE: I've heard this expressed as the waiter rule. See how your date treats the waiter, because you don't have to be nice to the waiter, right? You can treat the waiter any way you want, and same thing with pets, right? And pets are even more, because you have a duty of care.
MIKE: Yeah. And my wife, I say, when we went to her parents' house, she, like, got down on the floor and was playing with the dog, wrestling with the dog on the ground because they were such good friends. That says something, right [chuckles]?
There is somebody who loves interacting with somebody that they could mistreat because they're in a position of power to do so, and instead, they're saying that they're just having fun together, right? That says something. And I think that, as a manager of managers, you have to make a similar sort of evaluation. Now, how do they treat their pets? Are they going to provide safety?
WILL: Fair enough. I, you know what I mean, I guess, like, you know, one thing, and I will push back...I think it was Matt, you know, where he's like, "I keep an open door policy, you know, and, like, and everybody can come and say anything they need to to me, like, because I'll listen to anything." And I pushed back on him, and I did it on the podcast, so I'll call him out, you know.
But, like, the issue is that so many managers and managers of managers will mandate that thing. They'll say, like, "Hey, guys, open door policy, right? Come to me with anything. If there's bad news, I want the bad news, you know, like, you got to let me know so we can work it out. No judgments, you know what I mean? Open door, open book. Like, bring it to me because I can take it." And I'd be happy to give people the benefit of the doubt on that, that when they say it, they mean it.
But the issue that you run into, very few people will tell you this, that's not yours to give. You can, I can be assigned to you as a manager. I can be assigned to you, and you're going to say, like, "This is what I need you to do. This is what I need you to not do. This is the criteria for, like, you know, your success. This is what I want you to do every day." But my feelings of safety and trust to you must be earned, and that is absolutely unequivocal, you know what I mean? But you could say that, and you could mean it, and it might even be true, although that can't just be assumed, you know?
You can't just take some stranger, right, that I have no personal relationship with; I was assigned to him, or her, you know, whoever, right? I was assigned to them, and they say, "You can trust me with bad news." And if I take the wrong step, that could be, like, you know, I could lose my house, right? And they say, "You trust me." Nobody's going to tell you─very few people.
You'd have to be pretty dumb with a pretty big mouth, like me [chuckles], to say, like, "Hey, man, time out." You got to earn that. That's not for you. That's for me. I could give it to you, or I can withhold it. And the smart money is "We'll see," you know? Like, that's the smart play, and I've been very successful. We'll see [laughs], you know? It's been a winner for me.
MIKE: And you can earn that trust by actually earning it. Like you say, somebody comes and gives you the bad news, and you don't kill the messenger. Somebody comes in and has a problem, and you help them fix it. You know, those are the...and you actively go out looking for problems to fix and actively go out looking for those bad messages because it's important for you to know them. And you actively don't kill the messenger when you learn them.
WILL: Yeah --
MIKE: Like, you can go and earn that trust by doing that.
WILL: Yeah, and you ask people for feedback. You have to go out and, like, the first few times you ask anybody for feedback, it might not be completely, like, rainbows and kittens good news. Like, you're going to have to go out and find that, right? So, I mean, like, something's always going to blow up, and you're going to have a couple at-bats just for free because something's going to catch fire somewhere.
But if I never hear from you, if things aren't going bad and you'd be like, "Tell me anything," I'd be like, that's not a relationship. A relationship is when something goes bad, you know, then okay, then I can talk to you and, hopefully, you're all right. But, I mean, like...anyway, I mean, that's just how this is how things go. And that's why these sorts of things are an art, not a science.
VIVIAN: So, I have a bit of an advice question from all of the senior engineers here. How does someone relatively new to a company go about building that trust? I came into this company as an intern, and I can only imagine that most engineers want to do their job and get their work done and not micromanage an intern who doesn't know the ropes of an organization yet.
But I've found that almost every single person I've interacted with has been nothing but kind, and patient, and accepting with me. But I really struggle to know how much that means that they trust me to do my work versus they are babying me because I'm brand new to an organization, and they don't want to hurt my feelings, or push me too hard, or not give me enough freedom.
So, how do you go about building that trust when you're both trying to learn how to trust the people around you, to know that they are doing good work, that you can trust their advice, that you can listen to what they say, but also that when you do your work, you know that they will listen to what you have said? Even if they don't fully know why you've done it, they're willing to, at the very least, hear you out and trust you enough that you can do work.
MIKE: You know, I said a minute ago the manager earns trust by proactively going out and solving problems for you and not making a big deal when you get a message you don't like. If you go out and proactively solve somebody else's problems, they're going to like you [chuckles]. You know, like, "Oh, you went and fixed that bug, and I didn't even hear about it until it was fixed in production. Wow."
I had somebody once tell me about, they gave the idea of change in the pocket. They'd heard it from their manager. Every time that one of those things gets fixed they didn't have to deal with, it was like a coin going into their pocket. And then when they broke something, they had to go tell their manager, "I broke something." They're like, "It's okay. You got a lot of change in the pocket." Like, "What?" "Oh yeah, well, all of these good things you've done have added up." And so, you know, it goes against that balance, and the balance is pretty high, so it's okay.
DAVE: I literally this week told someone...somebody did something that cost me some effort, and they were like, "I'm so sorry," and I'm like, "Dude, it's fine. You have a huge balance with me." Like, literally in those terms, like the emotional bank account kind of thing. It's like, yeah.
WILL: I mean, honestly, like, more than anything, it's just consistently showing up and doing your best work. Like, they're going to give you work to do, and you do that work. And then, you know what I mean, and things are going to go wrong, because they will always. And when something goes wrong, you show up, and you give your full and forthright effort on fixing the thing. If it's your problem, figuring out what went wrong; if it's not your problem, finding help if you need it, right? Just taking responsibility. It's like, okay, this thing happened, okay, was it my fault? Then just show up. Just show up consistently.
It is absolutely unmistakable when you have somebody who is going to do that consistently day in and day out. Pay attention, you know? Like, just do the work. I don't know why it's so...no, I do know why. I do know why. There's lots of good reasons for why it's difficult to find people who will do the work consistently. But, like, if you just show up and get things done, talk to everybody and get it done, you're going to do fine, like, honestly. I mean, that's it.
You know, where I feel people fall off, where I've seen people fall off, is either lack of...well, the effort wanes because, like, it's just psychologically difficult to keep on, like, fighting this fight every day, day after day, week after week, month after month, year after year, decade after decade. Like, it's hard to remain psychologically committed to the work.
And you have to be psychologically committed to the work. You can't be 50% in, you know? You could be an accountant and be like, "I'm not passionate about these taxes." I think you could probably successfully file somebody's returns even if you're listening to a podcast. But if you're in this thing, you have to be all the way in. And so, like, that's one thing.
And then the other thing is, like, when things do go bad, you get weird, you know? Like, you get weird. I'm dealing with...I'm right now fighting with some people who have gotten weird. And their things are not working, and we are having a lot of trouble coming to an agreement on how they're going to get fixed, and people are going to start losing a lot of money.
Big people are starting to become aware of this thing, and because everything is digital, there's a paper trail of them being very weird for a very long time. And they are going to have a very big problem because, like me, they are consultants, and consultants don't have all that much grace. And so, accountability, accountability, and consistency. Work hard, be accountable, communicate. I don't know, I mean, like, I wish I had a more, like, I don't know, like, poetic answer, but really, that's really all you got to do.
VIVIAN: Honestly, it's kind of comforting to hear that there isn't some, like, secret sauce that everyone knows that I don't know. Just being consistent and showing up will pay dividends, not only in the trust that I can build, but also in the kind of connection and organization that I can help to be a part of and create.
KYLE: I'll just say one that's a pet peeve for me is, don't tell me you know something that you don't. If you don't know it, ask. I've run into situations where I've been working with an engineer that I was under the impression that they knew about a subject, only to find out that they don't. And that is a problem in the sense of, like, I can adjust how I'm approaching the situation if I know that you're not aware of something. Like, it's not a problem for me to teach you. It's a problem that I thought you knew this, and you're pretending like you do. Yeah, just a pet peeve here, but that's one of them.
VIVIAN: No, that makes a lot of sense, yeah [laughs].
MIKE: Well, there's kinds of dishonesty. Somebody saying, "Yeah, I know how to do this," it makes it impossible to actually get them up to speed. If they said, "I don't know this," great. Let's solve that problem. If you are dishonest about that, then it completely destroys the ability to progress.
KYLE: I would say to that, as a, you know, mid-level, senior level, you know, we...at least I would think that most people aren't going to assume what you do and do not know. So, it's kind of on you to say what you don't know. And there's no problem in asking because I don't think there's any senior that's going to look at an intern or a junior and be like, "No, that's a stupid question. Why are you asking that?" No, we want those questions.
DAVE: And the seniors, too. I think I've shared this story on here, but I was in a meeting a year or two ago with Eddy Lopez. And we got to something...there was any questions, and I asked a question. And it was a boneheaded, stupid question. And, like, Eddy turned to me, big eyes, and he goes, "You are fearless about asking stupid questions." And then the person next to me said, "I'd actually like to know the answer, too." And I'm like, yeah. Yeah, I love a good, stupid question. I love a good, stupid question."
WILL: Yeah, I mean, embrace your stupidity. Like, I'm really honestly, like, I'm a beginner in this, like, sort of AI stuff. I'm actually getting fairly effective. But, like, if I had approached it from a position of, like, "Oh, I've been doing this for 30 years, I know how to do this stuff," like, I would be nowhere. So, I'm just like, "Okay, let's try some stuff," you know?
I mean, you'll hear me, you know, hit up, you know, Dave. I have another buddy, Darin. And we're big, big AI guys. And I'm like, what about this? What about this? Can you do some of that? And how does this work, you know? Because I don't know. I don't know. I'm just brand new. I'm an old C programmer for crying out loud. That's what I know about. [inaudible 01:07:09] one of these function pointers? I bet not.
MIKE: [laughs]
DAVE: Oh, yeah. Love me some PFs.
WILL: But I will say that, like, you know what I mean, and, like, the secret silver bullet, you will never, ever, ever, ever, ever have a hard time getting an answer to your question from anybody in the world, ever. And they will happily answer whatever question you have if you just do this one secret thing, which is to say, make a concerted effort, you know, relative to who you're asking, right? Relative to who you're asking.
But, I mean, like, if you're just like, "I've got a problem. I can't figure it out," spend 15 minutes thinking about it, trying some stuff out, documenting your approach and your specific question, you know? And anybody on your team, you could get an answer from the CEO, I swear to God.
Like, I don't know your CEO, but I bet you if it was a question that was relevant to him and you spent 15 minutes thinking about this thing and you emailed him, he would email you back. I bet you $1,000, you know? If you were sending to the right person and you put some effort into, like, figuring it out yourself and you're like, "This is what I did; this is my question; this is what I need from you," like, [snaps finger] you'd get what you were asking for, and everybody's like that.
And where people run into issues is when they are...they just sort of, like, they just immediately give up and throw the white flag. And people will get frustrated with you then, you know? But if you did the effort, I swear to God, everybody, they will do backflips for you. You'd be stunned.
MIKE: We talked about that first answer on Stack Overflow. Like, it's catnip to engineers to see a problem that somebody's gone halfway through and they couldn't figure out the solution for. Like, they can't help themselves.
DAVE: It's like setting down an unsolved Rubik's cube. It draws attention.
MIKE: [laughs]
VIVIAN: That's very interesting. I'm kind of curious about how to find that balance. I have a strong tendency to dive really deep into trying to solve a problem to the point where it's all I can think about. And I'll look at the time, and it will have been five hours since I last talked to anyone or checked my messages or looked at the time.
And I sometimes struggle to identify that time where I'm like, "Okay, I could spend the next five hours trying to solve this, but I know somebody that knows how to explain it to me so that I'll never need to spend five hours on this problem or a similar problem ever again." How...I don't know if this is too theoretical of a question, but how do you actually create that self-discipline to pull out?
WILL: Don't.
MIKE: Yeah. I've got a very pragmatic answer to that. I say two to three hours. That's flexible.
WILL: I have an even more pragmatic one. Don't. Let it cook. Let it cook. Just cook.
DAVE: That attitude --
WILL: If you find yourself at midnight, you know what I mean, like, working on a problem, and you're still engaged, and you're locked in, and you're focused, let it ride. Let it ride. Like, because, yeah, okay, especially where you are in your career, right, especially where you are in your career, like, just cook. Get in the weeds. Get weird with it, you know what I mean? Like, do any kind of crazy thing that you think of because, like, you are learning things you're going to take with you forever, so just keep it going.
Where I wave the white flag maybe quicker, faster, but less often, is because, generally speaking, I have production quotas that I need to meet, and people are starting to crawl up my leg. And I cannot spend a week on a problem in a way that I might have happily done another time [laughs], you know? Prod is down. Did you know that prod is still down? Maybe let's fix that, you know? And so, I can't indulge those things. But I'd expect you as an intern to get weird with it.
DAVE: I will second that with a slight measure, which is, yeah, let yourself get obsessed, especially at this stage in your career and with this warning: That might cost you your job, but it will make your career. You can bank a career on that level of obsession.
WILL: If you've got standups, and you've got people, and, like, you have people like, "Hey, Viv, how's it going? You still working on that thing? Can you show me what..." you know what I mean? Like, there's people in your standup, especially as an intern, they will redirect you. And you'll sort of start to get a feeling for, like, "Oh, I spent a little bit too much time on that," you know? But, like, there are people whose explicit job is to be like, "Hey, so let's walk through this. Let's pair through this thing," you know what I mean? They will redirect you. I mean, like -–
MIKE: Yeah, true.
WILL: Up until...I'm trying to think of a time where you can't get away with that anymore because, like, again, you know what I mean? Like, things, like, people get up my leg, you know, if I'm taking too long on something that ought to be resolved.
MIKE: Well, if you're actually engaged and learning stuff, that's productive work. The reason I said a couple hours is if you're stuck, that when you get stuck, and you don't know what else to do, and, you know, you're disengaged, and you don't ask because you don't want to ask anybody, that's where you get in a lot of trouble.
DAVE: Right. I had a really good manager when I was a junior who helped me with this because I was very much the obsessed type. He had come in in the morning at, you know, 8:00 in the morning, and I was at my desk, and he says, "You're in early." And I'm like, "I haven't been home yet," you know? I've been here since yesterday. And he liked that. You know, he's like, "You need to have some work-life balance, but it looks good on my productivity sheet."
But he pulled me aside and said something really, really profound. And this goes to asking questions. He gave me two pieces of advice. One, carry a notebook and write down any answer you get because your coworkers will pay...and the reason for the notebook isn't for writing down. It's so that you never, ever ask the same question twice. Your coworkers will hear you ask the same question over and over and over again, and they will flip the bozo bit on you, and they'll withdraw. That respect that Will was talking about, they'll pull that back in.
But the other thing is, you do have to value your time, and if you're obsessed and you're locked in, keep chasing that rabbit, man. That builds Vivian 2.0, man.
VIVIAN: [chuckles]
DAVE: That's always going to be pushing you down the road. That's a superpower. Don't cure it. But if you are stuck and spinning your wheels and you don't know where to go, and you're just kind of beating your head against, you know, document pages that you can't find the answer to, value your time the same as a senior developer's. If you spending an hour on this can get you through it and will save a senior developer an hour helping you, don't waste their time. Just go spend the hour and get us the...but if you've been down this eight hours and you're still stuck, and a senior's hour of time can get you out of that eight hours, the value call for the company is go bother the senior developer.
And this is psychological safety. Trust that the senior developer sees that ratio as well. We do. You know, we know which people are live ones and which people are help vampires. And if you're a live one and you've, you know, like, like Will said, you show up with, you know, a pile of broken pieces and half-built solutions that don't work, and it keeps falling down here, here, and here, that's when somebody can pull you aside and say, "Ah, let me show you this."
Or, like we talked about at the top of the call, just watching somebody do this, you can have a senior just kind of look over and look at the pile of parts and go, "Yeah, it does that." And you realize, I shouldn't have been solving this problem in the first place. I've been trying to prove P equals NP, and okay, yeah, move on.
KYLE: Also, ask the question, "Get me out of my meeting," then I can spend time helping you and get out of a non-essential meeting [laughter].
DAVE: Yeah. Public calendars are a weapon. I love it [chuckles].
VIVIAN: I'll be honest, I have used that one once so far at this internship [chuckles].
DAVE: Nice. Nice.
VIVIAN: Oh my gosh.
KYLE: One thing I was thinking is double dip when you can, too. And what I mean by that is solve it yourself. Spend the time to solve it yourself, but don't call that the end. Go to your senior. Go to your, you know, person with the knowledge base and say, "Hey, I solved it this way. Should I have done anything different?" Have a discussion with them and see, like, what the better approach was, or, you know, the differences.
VIVIAN: I don't know how this will change as, like, my career progresses and I get into higher levels, but I've been really enjoying the code review process a lot more than I initially expected to. Because I love, like, getting someone who just has way more institutional knowledge and way more knowledge about programming in general to take a look at my code and say, "Oh, why are you doing it this way?" And I'll have to explain it. And in that explaining, the thought process occurs of, like, "Oh, this is why I'm doing it this way. Oh, and I could have done it better this other way." Or they'll look at the code that I'm writing and go, "Oh, why didn't you do it this way?" And, suddenly, that kind of reopens my brain just in the code review process to get something into prod.
WILL: Yeah, you'll get over that [laughter].
DAVE: Yep, yep. You'll get jaded. Yep.
WILL: Yep. I don't know. Sometimes you'll find yourself in situations where you're dealing with just how people want things to be, and it's sixes. And it's easier to just give them their way than it is...which is a substantial investment of non-productive activity on your part, just because somebody really likes, you know, all their widgets sorted in this particular way. It's easier to just give them their way and move on. That can be frustrating when you're operating under resource constraints yourself.
MIKE: So, engage with the people who tell you why. There'll be people who say, "No, it's got to be this way," and there'll be people who say, "Well, if you did it this way, it would be easier because it would do this and this and this. Here's an idea for you. Here's some example code." That's a very different kind of answer.
VIVIAN: Yeah. Okay. Well, I feel like I've taken it way off of the organizational complexity task and into personal advice for Vivian podcast.
DAVE: This is what...no, this is absolutely going to be published, and there are going to be people that are going to tell us, "I am so glad we had this chat." Like, legitimately, these are...you have not been...unfortunately, Vivian, you have not figured out how to ask stupid questions. I'm going to have to ask you to work on that.
VIVIAN: Okay. I'll work on that.
DAVE: Yeah, yeah.
MIKE: I think that's a good place to tie the bow on this one. I think we're going to have a podcast on organizational complexity and advice for interns [laughter].
DAVE: The thing is, if we did a podcast called Advice for Interns, it would be crickets, yeah.
MIKE: Thank you, everybody. And until next time on the Acima Development podcast.
This episode of the Acima Developers Podcast continues a series on a NASA paper about engineering failures, focusing this time on decentralization of authority. The core question Dave poses is where breaking things down and simplifying goes wrong, and the group lands on a shared answer: complexity doesn't disappear when you split systems or teams apart, it migrates into the seams between them. Will frames organizations as multi-threaded programs, arguing that distributed work is inherently less efficient than a single mind holding everything, and that the real failure is that nobody budgets for that inefficiency. Teams plan sprints as if they operate at maximum efficiency, then get blocked waiting on other teams like threads waiting on a mutex.
The conversation moves into war stories about organizational structure. Kyle describes how Acima's DevOps team, once embedded with engineering under one budget, now sits in a separate department requiring alignment several layers up before work can happen. Dave cites a Microsoft study finding that bug difficulty scales dramatically with org-chart distance between the person who needs a fix and the person who can make it, and recounts a previous employer where carving out a DBA team concentrated all database authority in one group while the blame stayed distributed everywhere else. Will's proposed remedy is cross-functional "tiger teams": pull a dedicated body from every needed discipline (DBA, DevOps, security) onto one team for the life of the project, deliberately over-allocating rather than pretending ticket queues between silos are free. He argues for institutional checks and balances over exhortations to "speak truth to power," since pressure, layoffs, and optimistic scheduling make honesty structurally difficult.
The back half turns philosophical, sparked by Vivian's observation that startups work like mesh networks (everyone connected, nothing precious to break) while enterprises are hierarchies protecting a "golden goose." That leads to a warm tangent on being kind to your future self: Dave's story of leaving network diagrams inside junction boxes that paid off 15 years later, Vivian's "second brain" note-taking practice, and the YAGNI principle of trusting tomorrow-you to solve tomorrow's problems with better information. Will counters with a lament about corporate chat retention policies deleting institutional knowledge after three months, which he calls "leadership malpractice." Vivian closes with a fitting meta-observation: without Mike there to keep them on track, the decentralized conversation never actually reached a conclusion about decentralization, inadvertently proving why some centralized accountability matters.
Transcript:
DAVE: Hello and welcome to the Acima Developers Podcast. I'm your host today, Dave Brady. We have got a really good panel today. We're still talking about the NASA disaster stuff.
Today on the panel, we've got Will Archer. We've got Eddy Lopez. We've got Vivian Moore. We've got Kyle Archer. We've got Will's dead camera connection from his previous dropped call, and we've got Ramses Bateman on the call.
So, we've been talking about this NASA paper, if you've been following along at home. You can get this on the Googs or wherever: Elements of Engineering Excellence by J.C. Blair, R.S. Ryan, and L.A...Oh boy, Shutzenhofer, from AI Signal Research in...if you find that, Google for that, you'll find this.
And, basically, it was...they prepared this paper for NASA telling them, "This is why your stuff keeps breaking, and you keep killing people when they try to go into orbit or come back from it." And we've already talked about, like, normalization of deviance and, I don't know, some other stuff. We did a whole show a couple of times, and I kept making it a forum for AI. So, I'm hosting today, so I'm going to be asking questions and trying not to turn it into the Dave show.
We're really excited to have Vivian on the show. Vivian, you're one of our interns for the summer. Do you want to say hi to the audience, just tell them who you are?
VIVIAN: Yeah, sure. So, I'm Vivian Moore. I'm going to be one of the software engineering interns for the summer. I've been actually listening to the podcast for the last, like, couple months or so, kind of getting prepared for the internship, getting to know everybody. So, it's kind of a weird experience being on here now that I'm actually at the company, but it's very fun.
DAVE: Right on. Welcome.
So, Mike always starts with a story, and I usually have 20 of them on hand, but all stories that I have are going to go immediately onto a tangent. So, I kind of wanted to just kind of open the floor a little bit. Mike commented to me...Mike's running late today. He might make it. I'm hoping he does make it. But he sent me the meme from Office Space of, like, when you have eight bosses, nobody's actually in charge when you answer to eight people, right? It's the bystander problem, bystander effect where, you know, if a crisis is happening and there's eight people standing there with their cell phones, and you say, "Call 911," nobody calls 911 because everyone assumes that everyone else will do it.
Paper hones in a little more sharply on this, that it has almost more to do with siloing teams and isolating teams from each other and stopping them from communicating in between each other. And, in the pre-call, I mentioned that the devil kind of lives in that space.
I've seen so many software ideologies come down the pipe over the past decade or two, where people have, you know, like, "I've got the answer, and this is going to solve everything. All you have to do is simplify the piece that you're working on." And they show you this really simplified thing, and it looks good. It looks clean. It's obviously true, but it doesn't solve the complexity. It just pushes the complexity outside the thing you're actually working on. And if the other team on the other side of the wall is pushing that complexity back at you, that complexity now lives in between.
Honestly, that was my big grief with, like, the way Java architected things out by default. I mean, like, Java's great. It's a good plan. But the naive approach to the way Java breaks everything down into tiny, tiny, tiny, tiny, tiny, tiny...Like, I can remember people saying, "You should not have a class longer than 25 lines of code, and you should not have a method longer than 5 lines." And I wish I could write software like that. I know people who can, and, I tell you what, they are very, very good at finding the file they need or the seven files that they need because they're all over the disk, right? Which arguably is probably better than trying to sift through a 3,000-line chunk of code, which actually might be an allegory for this organizational problem.
But my issue is that you start having problems with how the modules get picked up and put together, because complexity starts to live in the seams, not the units, but the integrations between them. What do you guys think about, like, decentralization, breaking stuff down? We all know simplifying things is good, but where does it go bad? If we just take simplifying and breaking things down as an objective good with absolutely no qualifications, where does it go wrong?
WILL: So, whenever I think about, like, sort of, like, these sort of, like, distributed processes, right, like, across engineering organizations, individual engineers, like, I always think about a multi-threaded program, right? So, one of the things that we've seen over the past, like, you know, however many years, like, really since we broke out of the 486, you know, line of computing is more and more and more cores, right? More and more cores: 18 cores, 16 cores, 32 cores, lots and lots of cores.
Well, the problem with lots of cores is there's lots of threads. And one of the sort of dirty, little secrets of, like, this sort of, like, nitty-gritty systems-level programming is your efficiency, as you have these multi-threaded processes, intrinsically, like...And I think this is mathematically provable. It always goes down relative to a linear A to B to C to D to E to F, all the way to Z, like, one at a time, right, maximally efficient. But, you know, then you only have one person working one way.
For people who've experienced, like, the joy of creating a system, or a service, or a program, or a protocol soup to nuts, and it all lives in your head─you made all the decisions; you did every single thing; you know where all the bodies are buried─you will never in your life experience efficiency like that. But you're still only one person of flesh and bone. You only have 24 hours in the day, just the same as everybody else. And if the enterprise is going to scale, you're going to have to scale out to different people. And that comes with inherent inefficiency.
If you did it right...and writing multi-threaded programs is astonishingly complex if you've had the burden. A lot of people don't really do it. It's pretty rare for people to write hardcore, nitty-gritty, multi-threaded programs anymore. All of that heavy lifting has been done by, like, really, really, really, really smart people at several different layers, you know, like, to the database, to the operating system, to the sort of, like...Anyway, the frameworks, all that stuff has been done, but it's astonishingly hard to do.
We have, you know, general organizational principles to do it, and they're all pretty good. They're all easier said than done, but, like, where I feel like things break down foundationally is we don't allocate for the inefficiencies inherent in the system, like, that's the problem.
Every team schedules themselves. They plan their prod. They plan their releases. They plan their sprints, their production quotas, you know what I mean, their timelines for this maximally efficient thing. And it's always wrong because you're always going to have to, like, you know, take the lock. I'm waiting on this mutex, you know? I'm waiting on a barrier, right, which is, like, the release calendar, and all the threads have to wait at this, you know, this barrier. You know, going back to, like, all these, sort of, like, nitty-gritty parallel processing terms.
And so, you don't allocate for that, but it's always going to happen because every team is just like, "Well, how fast can you get it done?" And we all want to say, "Yes." We all want to say, "Yes, I'll get it done. No problem. I can get it done in a sprint. I can get it done in two sprints." And you could, but you can't, and you never will.
And capacity planning doesn't take into account, like, well, you know what? I'm going to spend 20% of my time fielding outages and service requests from other teams that interface with me. And, like, did I blow it? Ah, maybe, maybe not, you know. But we're going to have this inefficiency. And it's baked into the system, but, like, human psychology is such that you always want to downplay it. That's my read on it.
DAVE: Yeah, no, 100%. There are some things that when you put them together, they're too big to hold in one head, right? Like, we have software that customers use to onboard leases, literally to apply for a lease, and we give them to you. We have another team that just services the leases for our existing customers, and they are separate teams. They have an entire business domain between them, and our products have to talk.
I want to say, when Acima started out, like, in twenty-teens, early teens, right? This was all under one roof. It was just one gigantic app. I could be wrong about that. But it was one gigantic monolith. I know our merchant portal app has been calving services. It's like a glacier constantly calving sub-bergs out there.
And you've hit the nail on the head that you can't hold all of MP and all of LMS in your head at the same time as a functioning integrated unit. So, let's build an API interface between them. Now we've decentralized these two things a little bit. We've decoupled them, but there's an inefficiency that now hits really hard, which is that the interface into LMS has to be robust and generic enough that somebody who doesn't know the innards of LMS can use it, and vice versa. When they call back into us to look something up, our interface has to be generic and robust.
And when you're all in one spot, you can run into this thing where it's like, "I don't have to solve the general case. I know I'm solving this case, this case, and this case, and everything else can go hang. Who cares?" But if you make this a blind public API and you publish it on GitHub and say, "You can use my library," now you do actually have to, like, assert your domain and say, "I'm handling this case, this case, and this case, but not that case."
And half the time, if it's a public project, somebody will go, "Well, why not? Why don't you just..." right? And then they hand you a, you know, an issue that's going to be, you know, a 7,000-line pull request or whatever. And that's the answer to why I didn't just. But that's inefficient because that doesn't do anything for my three base cases. I'm done. I've got what I need. My itch is scratched, right? Screw you guys. I got mine. But if I want to decentralize, if I don't want to take on all the business of managing leases after they've been created and, you know, servicing lease lifetime, then I have to support a decentralized team, and that has to give them the necessary communication.
KYLE: You kind of hit on what I was thinking in a little bit different way there, Dave. I was sitting here thinking, like, you see the problems with decentralization evolve as you're going from more of a startup to an enterprise level, in my mind. And that, you know, is that where you've got one boss and three developers to begin with. And there's a lot of efficiency. There's a lot of, you know, speed that can happen with that.
But as you age through the company, you know, now all of a sudden, you need to extend this out to, you now need DevOps; you now need IT; you now need QA. You've got all these independent groups with independent goals. And then it goes further and further, you know, and it's stuff like you have people reporting to different departments with different budgeting, you know, different things entirely. And it's getting those two budgets aligned, those two initiatives aligned for the one goal is no longer an easy thing.
I can even remember here, you know, at Acima, DevOps was in engineering. We were next to the engineers. We had the same budget. We had the same goals. Well, those are two different departments now, and we have to get the people above us aligned before any of us down in the trenches that are doing the work can start doing and communicating together to get anything going. There's so much overhead, so much, you know...we've got too many cores, you know what I mean? Like, exactly what Will was saying. There's too many cores going on, and that has created a lot of chatter.
DAVE: Mm-hmm. And it's layers, right? It's cores of cores of cores, right? We've got threads running in cores, in multiple CPUs, on multiple machines, in multiple clusters, in multiple data centers. And this is the [SP] pitch: My thread needs some data out of that thread, but I have to go up to the core, up to the CPU, up to the da-da-da. I literally have to hop over the internet to another data center and then back down the same, you know, ladder.
There was a fantastic paper that came out of Microsoft. They analyzed all of their bugs in Windows. They had plenty of data to work with, right? [laughter] We had a lot of fun poking at that. And they rated, like, how hard a bug was to fix. They, like, "Let's categorize this data. What do we see from it?" And they said, "Far more, by far and away, the biggest thing that would add story points and difficulty to a bug fix, and also something that would be most likely to cause a bug, is when the thing you need and the person who needs it are separated, not by distance, but by layers of the org chart."
If it's my stuff, I just fix it while I'm typing in my file. If it's somebody on my team, I have to, you know, poke you or go stand by your desk or open a Slack call. But if I have to go up to my team lead and then up to the department head, and then up to the engineering manager to go back down to a different department head, to a different team lead, da, da, da, every time you go up a layer, the likelihood of a problem and the severity of the fix...the severity of the problem and the difficulty of the fix, like, squares. So, give me a bug that's 10 times harder in my own code to fix. It's so much easier than a simple one-point story that is 3 layers away, because that's a 16x multiplier on everything that you do.
I thought that was really, really interesting. And I lived the thing that you're talking about. I lived this at a previous employer where they divvied the company up into horizontal slices. And it's so easy to justify this on a budget. You know, it's like, "Hey, if we put all of DevOps into one pool, and they just serviced all the DevOps needs for the entire company, that would be so efficient because we could buy them one big Grafana license." And it looks really, really good on paper.
But now you've got this org chart where this entire layer services this entire layer. It looks great on a PowerPoint slide until you realize that I'm over here on this end, and you're over here on that end. And when I go down a layer, I'm going sideways across five layers. And I might talk to a DevOps person who has no idea what my problem domain is. And that DevOps person has a different budget. Like, you actually said budget issues, right? You're having the...You know, my laptop is starting to bulge because the battery is going to explode any day now, but I'm waiting in line because IT, they're cracking the whip over them because they have to get the Keycloak integration locked down into our, you know, da, da, da, da, da, right? It's the different masters.
The organization that I was referencing before, I won't name names, we had the database team. We had everybody who was embedded. You know, as a developer, I worked everything from the JavaScript all the way to the database, to the SQL stored procedures, all the way up and down. And they sliced out the database layer and said that we now have a DBA team.
And it went wrong in exactly the worst way, exactly the way that you could predict if you understood one social thing. And the social thing is the CTO of the company was a co-founder. He had put in millions of dollars, which means the CEO outranked him on paper by a few dollars and cents, but the CEO could not go to the CTO and say, "Fix your stuff, or you're fired." Okay? The CTO would just be like, "Screw you. We're... I'm saving money."
So, what happened is all of the authority to do anything moved, in the database, moved to the database team. But all of the blame for anything that goes wrong in the database went everywhere else. So, if I took out the database, it was my fault. But if I needed a column added to the database, I had to go, you know, beg and plead from the DBA team, and it was that Microsoft up five layers all the way to the C-suite. You had to go all the way to the C-suite to get a budget authorization/a command of just do the stupid thing, please, from engineering to data. And it crucified us in exactly the ways that you would think.
In a gigantic enterprise organization where everything is sorted, and everything is turnkey, sure, I think maybe it works. But I've mostly only ever worked in startups, and startups, lately, startups moving into the enterprise. Like, Acima is kind of in that boat as we kind of enterprise-ify. But we still have...we have a lot of startup in our back pocket. And that startup means that we have our three base cases and nothing else is supported.
And if we separate the layers, ugh, the data team or the DevOps team, they're now backfilling all these other use cases because our parent company wants them or because our sister company wants them. We don't need it. Who cares, right? It's...We got ours, but we can't say, "Screw you," because you over in DevOps, Kyle, you are answering to a different master, and, hopefully, we stay in business.
WILL: No problem. I mean, and what I think of when I hear these things, right, which I've heard, like, so many times, and the way I think about it, at least, is, like, well, what do you do about it, right? And you have these sort of, like, nobody wants to talk about the inherent inefficiency baked into the system to get a thing done, right?
And so, from my perspective, like, what I think is you just sort of, like, when you are trying to develop a feature that crosses a bunch of different teams, you make a new team, and everybody on the team, right, like, you have a representative for, like, everything that you're going to need to do, and you can give those guys back when you're done with them. But, like, you actually, like, really, like, you take this inefficiency, and you say, like, "All right, well, I'll need to roll this thing over," right? And so, I need somebody to manage the lease, and I need somebody to, you know, somebody from the lease, whatever, underwriting team, you know what I mean?
And, like, I will get a DevOps person to, like, make sure that this stuff is together. And, like, I'm going to get everybody. Like, I will assemble, like, the Avengers team, and it's going to live as long as this thing is alive. And you're going to do that, but, like, politically, right, like, how many VPs need to sign off on this thing?
DAVE: All of them.
WILL: All of them. All of them, right? And so, it becomes really easy to do this thing. And it's one of those things where there's an argument that can be made that, like, yeah, like, enterprise moves so slow. If you come from a startup background and you see the pace of feature development in an enterprise, right? It's glacial, glacial.
In all honesty, right, it isn't, like, it's glacial because, like, you're doing so much, you know? It isn't that way. I've, you know, worked for a few enterprises, including but in no way excepting Acima. And, like, you know, the actual work that you're doing, right, like, if I had carte blanche and all the resources I need, I could probably do my coding for a month in a day, and I don't know that I'd need the whole day. But I can't do it. And maybe that's just the nature of it.
DAVE: Maybe.
WILL: And I suppose, like, the thesis that I've got is that by really accepting and embracing this model where we are going to over-allocate, I'm going to get a body from every team that needs to have a body contributing to this thing. And we're going to give everybody everything you need so that we're not...I don't have to put a ticket in to another team. Like, I just go in, and you're on my team until we get this thing shipped, and we're done, and we embrace the suck with both hands.
And we say, like, "No, this is what it's going to take." And you could get more things done faster rather than sort of overscheduling this blocking to get, you know, a container allocated, right, to do the thing. But every time I do it, every time I want to hit that thread, I got to take this overscheduled mutex, and I'm blocked until the thing comes out.
It reminds me of, like, in the battle days when we had spinning metal disc drives. When you over-allocate your drive, they had a little light, right, that would go on and off. You could see it every time the drive did a seek, and it would kind of chatter, you know, as the, like, kind of physical spinning head on the disc. And you could tell that when you had exceeded the capacity of your RAM because it would start paging out virtual memory onto that disc because you didn't have any memory.
So, it's like, okay, well, I'm going to page this one out. I'm going to page this one in. And that disc would start chattering, and when that disc started chattering, when the disc started talking to you, what it was saying was, [laughter] you're done. Nothing is getting done from this point forward. You need to...somebody's going to have to get voted off the island if you want any of these processes to finish. You're done now.
DAVE: The sound that spooks anybody over the age of 30 on this call is [vocalization]. That's your hard drive going.
WILL: Yeah. But the hard drive was never going to...the operating system was never going to tell you that. It was never going to...you were just going to feel it, and if you happened to be present, and you weren't, like, you know, tunneling into some server somewhere, then you were going to see it, and you'd be like, oh, no. Oh, they're planning my...they're [inaudible 23:25] process. Somebody's going to have to walk now.
DAVE: I talk about law versus lore. Lore is the stuff that they won't teach you in CS class, right? And I remember, I wanted to get into Linux. I was a Windows developer, C++, and I wanted to get into Linux and open-source development. So, to teach myself Linux, I took an old computer, and I installed it. This is back when you put it in a...You know, Linux came on five floppy disks, because it would fit on that. And I remember typing away and hearing [vocalization], and then a pause, and then a pause, and then [vocalization].
And what it was is it was, I would say, a bot. It was like a Perl script somewhere on the internet trying to password, trying to brute force a password. Three password attempts and then a three-second timeout [vocalization], and then a three-second timeout. And you could hear it. I could hear it because the computer was under my desk. I was physically adjacent to it. As soon as they moved everything into data centers, I'm like, "But how will you hear the paging attack [laughs]?"
Will, when you were talking about getting groups of people, do you mean, like, cross-functional teams, like tiger teams?
WILL: Yeah.
DAVE: Where, we're going to put a DBA on your team, so now your team has the authority to make a database change? We're going to put a DevOps guy in it, so you have the authority to punch a hole in the firewall right now and just get your stuff done.
WILL: Yeah. That's exactly what I mean. That is exactly what I mean. Because, again, I feel like we mostly talk about human problems, rarely technical problems. And I always look for, sort of, like, founding fathers checks and balances, style, organizational principles that you can use to combat these sort of inherent psychological weaknesses in planning, in leadership, in resource allocation, in willingness to have difficult conversations.
You can always say, like, "Well, you just got to be honest." And it's like, yeah, man, that's great, but we're on our third round of layoffs at, you know, big co. Everybody's head is on the chopping block, and you're going to go in and tell your VP, "We're not going to ship this thing," you know? You're an intern, you know, who is two or three weeks into your first internship job in a terrible job market. And it's just like, if I do a good job here, you know, I might, you know, have a chair when the music stops, and I graduate, you know?
And if somebody has just given me a completely ludicrous timeline, am I going to say no, you know? There's a lot of people who feel some pressure, especially in the year of our Lord 20 on 26. And when you increase the pressure, which can happen for any number of reasons, you know, the ability of people to speak truth to power is not something that I think we could just say, like, "Well, do the right thing," because the CEO, in one of those giant all-hands meetings, said, "You can trust me with the bad news." And they meant it, but, like, also it's kind of a cop-out way to run leadership.
So, you introduce these sort of institutional checks and balances so that you eliminate the ability for, like, managers and, you know, all these people to say, "Trust me, bro." Because the optimistic scheduling winds up being, you know, you get your least resource-overloaded team sort of setting the pace and the expectations for the organization as a whole, because they said they could do it in two weeks, and they can, hopefully. But, like, what about your security compliance officer who's six months back? If they're a blocker, they need to be a blocker for a reason, but they're overscheduled.
You have a team, and it's like, "Hey, that security guy's on the team. He's on the team until it's done." And then you go to the security guy, or you go to the security guy's boss, and you say, like, "Hey, I need him for two weeks," and he says, "You could go kick rocks," you know? And that's how it is, right? So, you just sort of, like, organizational principles so that you can't pick your own pocket. Because the temptation is always going to be there. It's never going to go away.
We're always... I've gone to work every single day of my career for decades now under some sort of pressure, internal or external, either because I want to do more, or I'm overscheduled, or I'm just an anxious person. I want to do everything I can, and usually the call is coming from inside the building, but, like, I don't know. Who among us is different? So, yeah.
DAVE: I think it's also...I think of, like, mentality. I can't remember which business book I read years ago. It might have been The Dip. It might have been one of the Seth Godin ones. But he talked about how...the author talked about how, in a startup, you need mavericks. You need boat rockers because we don't have a golden goose. We don't have a cash flow stream. Your job is we're going to turn a whole bunch of hungry programmers loose in the desert. And we're going to ride hard, stay late. We're going to sleep rough. And we're going to catch a goose, and we're going to bring it back. An enterprise company is the opposite. They have a golden goose, and your number one priority is don't hurt the goose, right?
I joke that Merchant Portal is one of the rougher codebases I've ever been on. So, somebody was like, "Man, you're so negative about this." And I'm like, "No. No, I'm not." And I had to go back and explain. It's like, I'm not negative on this. We are a multi-billion-dollar company. This codebase is our cash cow. That is freaking awesome. And so, it's a target-rich environment, certainly, but it's not just, like, messed-up code for the sake of having messed-up code. It's code that made us money, made us money, made us money.
The developers that wrote it rode hard, stayed late, slept in the desert in a sleeping bag, and now we're all in our air-conditioned apartments with, you know, healthcare and, you know, a human resources department, and don't mess up the goose, right? So, our job now is kind of to preen the golden goose and get it, you know, keep it fluffed and safe and healthy.
The thing I found interesting from the book is they talked about when you move from startup to enterprise, there's this pogrom phase, where you have to get rid of the martyrs...sorry, you have to get rid of the mavericks, not the martyrs. You turn them into martyrs. You have to get rid of the mavericks because they, by instinct, want to rock the boat, and rocking the boat is how you tip the golden goose out into the shark-infested waters. That's a weird metaphor.
Yeah. So, it's like, I think sometimes you have people that want to get rid of the mavericks, but then if you go too far with that, you end up with everyone reports to eight bosses, and now nobody's in charge of the project. The only thing that got distributed was blame, and authority has been jealously guarded, and everybody's covering their own butt behind.
VIVIAN: Okay, I just wanted to jump in here. From everything I've heard you guys talk about, as much more experienced engineers than I am, it feels like there's kind of, like, a distributed graph system that we're building. Like, in a standard enterprise environment, there's, like, one CEO that is in charge of the CTO that kind of distributes authority down from there.
And so, at the end of the day, like, whoever is in charge of making the decision is the one responsible for it. But what that results in is a lot of isolation between teams, a lot of disconnect, a kind of inability to move forward and make decisions and actually kind of, I don't know, as you guys are talking about, innovate and make big decisions. And in a startup environment, it's more like a mesh network where every single thing is connected to every single other thing. Each engineer knows every single other engineer, knows what they're good at, knows what they're bad at, knows where they can jump in, knows what they can do.
And because they're not protecting anything, they can, as is the common phrase, like, move fast and break things, because there's nothing to break. When you break something, it just breaks what you're building. It doesn't break the golden goose, that if you lose, you lose the company.
So, I guess I'm kind of curious how you find the ability to continue to innovate, to continue to create these mesh networks that actually drive innovation, that drive this progress forward, while maintaining a level of accountability to where each person knows who they need to be accountable to and why they need to be accountable to that person, while still being able to talk to anyone to get the things done that they need to get done at the end of the day.
WILL: Amy, and, I think, foundationally, you don't. It's a different animal. It doesn't work the same way. You don't write multi-threaded versus single-threaded code the same way. You don't do it. It doesn't work that way, like, I mean, and that's it.
So, what do you build, right? We could talk for an episode or two about, like, taking a majestic monolith, right, a unitary program, right, which Acima was, I've seen it, you know. And then sort of slicing off sub-processes, slicing off business units, breaking it up into pieces that still work, right, and kind of wing walking your way through that as things expand, and leadership changes, and you get bigger and bigger and bigger. All those things are just engineering and people engineering challenges. You know, you need to do things.
And I think another aspect of, like, we talked about, like, this mesh network, right, where everybody can talk to everybody else. One of the things, I mean, in terms of, like, a mesh network of people is, like, one of the most important things you accumulate as you gain tenure within a company is a network of connections to people who will actually do anything [laughs].
And a horrible heuristic, which nonetheless I have found to be true, is the number of people who actually work in any company is the square root of the number of employees of that company. And everybody knows who they are. And one of the things that a manager, a good manager, at the very least, will do is utilize those people to the best of their ability without burning them out because we all know somebody who, you know, if you've got a problem, you call them, and they're going to make something happen for you.
But, you know, everybody knows that person, and they will just burn them out. They will use them up. And, you know, you always want to be one of those people. You know, I'll say, like, you know, as an old senior dev, right, like, try very hard to become known as one of those people because they will always eat. But, I mean, managing that, managing that load, right, managing this sort of, like, this one node in your network that is just getting absolutely dogpiled with requests is, you know, an organizational challenge.
DAVE: I think Kent Beck talked about how to, like you were talking about, Vivian, of taking organizational, like, a mesh. You've got this...We want all the ability to move through it without the kind of the defects of it, right, the bogging down, without just immediately going to centralization and command.
And the interesting thing I find is that if you go read, like, Extreme Programming Explained...this is a book from, like, 2001, like, it's a really, really old book. But he gives, like, 11 principles of extreme programming. And they are all based on taking something and dialing it up to 11. And they all, in my opinion, address these exact problems. It's not just waterfall. It's the organization that is meshed in with the waterfall. So, the, you know, constant communication.
Extreme programming says, on-site customer. An on-site customer literally means the person who wants the software is on the cross-functional team with you 40 hours a week. They sit in the bullpen with you so that you never have that thing of, "Do you think they want this?" It's like, "I don't know. Gary's sitting right there. Ask him," right?
YAGNI, the principle of YAGNI, which is, You Ain't Going Need It. It's the build the three base cases that you need, and then don't worry about the other base cases. Now, if you're building a public API, you have a little bit more of a going to need it than that. But if you're building internally, Kent put it really, really well: trust tomorrow you to deal with tomorrow you's problems. Don't try to solve tomorrow you's problems today.
And, I think, it was Sandi Metz who put the best postscript to that ever, which is, because tomorrow you always has more information than you, and tying tomorrow you's hands is not doing yourself any favors. Find ways to be kind to future you by leaving yourself the ability to change choices without locking yourself...And we like to lock things down and tidy them up and say, it's clean. And then tomorrow you find out that, well, we put this in the wrong place. Really wish we hadn't welded it. Really wish we had just, you know, set...because we didn't need to. We just needed to set it down on the end of the table. That's [inaudible 36:39]
WILL: I don't know, man. I don't know about tomorrow Will and what he's going to be up to, but past Will was a real piece of shit [laughter], and he screwed me over and over and over again.
DAVE: Dude, like, legitimately, legitimately in the last couple of years, I made it a conscious principle of try to be kind to future Dave. I am not kidding. I have experienced a couple of times this year the sensation of, "Well, past me was kind of a nice guy [laughs]." Like, legitimately, like, feeling happy that past me...I did have to start to hold grace for myself though, because you'll get a bug. It's like, go write this, and then you go to that spot, and it's just a blank file. Like, who never wrote this? Oh, I never wrote that.
Well, what moron wrote this without this one...Oh, past me said you ain't gonna need it, and here it is 9, you know, 9 months or 19 months later, and we haven't needed it until now. We might have gone for another year. We might have gone until forever without actually needing this. So, we tend to judge very harshly.
What was the...it's very difficult to predict the future accurately, and, unfortunately, it's very easy to hold someone accountable for not predicting the future correctly. And when I look back at past me, it's the same thing, right? It's like, past me was a jerk. I'm trying to get kinder to future me, and as I do, over time, past me is turning out to be less of a jerk. And that's kind of weird.
VIVIAN: I mean, this is a bit of a tangent, but I've found that, especially when it comes to, like, schoolwork and things like that, the more I treat future me as an entirely separate person, that I'm, like, holding myself accountable, too, like, "Oh, future me is going to be really mad if I don't take care of this," it almost adds a level of responsibility that makes doing things a lot easier.
Like, it reduces that friction of like, "Oh, do I really have to do this? Do I want to do this right now?" It's like, "No, I'm accountable to future me because future me is going to have three more things on her plate that she's not going to be able to have time to do the things that I want to do now." So, that habit has helped a lot in recent years, and especially with, like, taking notes to make sure that when future me comes to look at what past me has done, there's actually a record of it so that I can reference it and move forward growing rather than shrinking.
DAVE: Yeah. I rewired a basement years and years and years ago for a friend, and they sold the house. And one of the things that we did, sorry, not the...Sorry, networking. We pulled network cable. And we were pulling it through the walls, and that meant we were going to switch boxes in various places. And there were spots where we were literally going through the junction boxes in the walls.
So, in every single junction box, I printed out a copy of the network diagram and circled, "This is this node," like a "you are here." I folded it up and put it in the junction box, not next to the wire so that it would...I wasn't putting tinder against the spark gap. But I put it into every single junction box, and I forgot all about it. 15 years later, I bumped into the person who bought the house from me, and they were like, "Thank you. We had a problem go wrong with the network [laughter]. And I pulled the junction cover, and there was a map of the network in there." And it's like, yeah, it absolutely solved it.
It was kind of a pain to do it at the time, but, like, I realized that...I'm not going to...you know, it's...Here's a weird off-skill from this as well. When you get good at writing things down, folding them up, and putting them where future you will find it, you give yourself permission to un-remember that thing. You give yourself permission to forget that thing, and that clears up space.
There is a...Eddy...we were in a meeting years ago, or a year ago, and I asked a really stupid question. And Eddy was like, "Man, you are fearless about asking stupid questions." Like, yeah, it's because I'm real stupid. But it's because I actively pick things to forget. It is a hugely powerful career skill to correctly identify the things in the future that future you will need you to have done in the past, without over-anticipating and trying to, like, complete all the things because that's wasteful.
If anybody is listening to this and wants to pick one thing to improve on, on being nice to future you, don't lock the door unless you are doing security work. Don't weld down the interface and require seven layers of ceremony if you don't actually need it. It feels nice and complete to tidy up. It feels like...in RSpec, drying everything up into your lets up at the top of the file feels nice and tidy, but it's a garden path. You're being led down a garden path because future you's going to get there.
You go to the bottom of the RSpec file, and you're like, "What are all these variables, and what are their roles? Gosh, if I had just left them, you know, in every single example, left them duplicated, then I would be able to read this one example and understand the entire context." My map of the network would be in every junction box in that case. Identifying that is more art than science and possibly more black art than regular art. But it's a huge career booster when you can manage that because it frees up your capacity. It frees up CPU power in the future you because you don't force future you to carry that cognitive load.
VIVIAN: There's a concept in note-taking that I've kind of grown to enjoy quite a bit called, like, a second brain. It's fairly common. And there's a...the YouTube channel, I think, is called, like, No Boilerplate. But he talks about essentially something that you've actually talked about, too, like, if you do it more than, I think it's twice, automate it in some capacity.
And the idea is that with a second brain, you're essentially creating this system where anytime you're thinking of or needing to keep track of something more than twice, you write it down, and then that's future you's job to look it up. But it's always on the responsibility of past you to have written it down. And so, over time, you can build trust in yourself, and in, like, the past you, so that you know that when you write something down, future you is going to have forgotten it and still be able to access that information because you can trust yourself.
And again, like, it frees up so much cognitive load to not be needing to be trying to remember every single piece of a schedule, every single piece of a note from a meeting that was three years ago that has that key piece of information that you need. That kind of decrease of cognitive load frees up so much of the other focus that you need to do.
DAVE: I love that you said trusting yourself, learning to trust yourself, where you go through that discipline. Once you've been on the other end of it, and the thing that you've received from past you has been the exact map you needed in the exact junction box when you needed it, you just get this, "Ah," feeling.
And then the next time you're in that situation, and you're writing something down, you write it down, and that little niggling worry in the back of your head of, "Do I need to remember this?" goes, "No. No, I don't." And just, "Ah." You can feel the CPU cores in your brain drop ten percent because you literally just let it go, and it frees up cognitive load. It frees up CPU power for the problem that you actually have to deal with today.
MIKE: I really rely on that pretty heavily with my to-do lists. Because if you have to remember it all in your head, that takes a ton of effort, and you're going to forget something, and it's stressful all the time. You write it down, you know, you can get to it. You know the place to go. You're good.
WILL: I've got a lot of... I have a lot of notes. I bought an e-paper tablet just to write notes out longhand. But I very rarely read it [laughs], you know, like, I just write it down. Like, occasionally, you know, occasionally, what I am notorious for doing, and one of my deep abiding hatreds of Microsoft Teams and, like, sort of, like, corporate lawyer retention policies, is that, like, I am currently working for a large organization where their lawyers have extorted the engineering team to have their chats expire at about, I think, three months, which is leadership malpractice on a apocalyptic level.
There's so many things that, like, you just... It was, like, a message. [inaudible 44:56] message from that team lead [chuckles] where, like, he was saying that you got to put this phone number in for this system. And it has to be a palindrome because somebody was the weirdo [SP] artist, and that's just...he really liked palindromes, but, like, if you do it like that, then this system, you know what I mean?
And it's just these pure weirdo things that, like, I will remember to write down on the magic notebook. But, like, it's not easily indexable like you hope Slack and maybe Teams because right now, as we were speaking, I was referencing a conversation, which fortunately has not gotten down the memory hole, between us, our head of QA, a third-party vendor, tier three, I don't know, customer service, or whatever's above customer service team, and, like, a project manager who quit two weeks ago.
And I was like, "God, it's somewhere in there." And it's about why we can do this thing in this particular pre-production environment, but not staging and not development, but, like, because of the configuration of this third-party vendor's service. And I just knew it was in there somewhere, and I dug it up, and I added the guy to the chat, but, like, I'm not going to lie to you, like, this happened in May. In August, it's all gone. Tears in the [inaudible 46:33], baby. It's gone [laughter].
DAVE: Yeah. It's gone. Yeah. There was this sci-fi thing that I read or watched years ago about protecting your IP. The accountants and the lawyers would be so happy if when you leave the building at 5:00 PM, if we could force you to leave the laptop and everything you know about the project behind the door. Like, we'd just erase your mind on the way out the door, right?
And when you talk about the...I understand what the lawyer was thinking. The lawyer was thinking, "If we get subpoenaed, we don't want things going back forever," right? I get it. That lawyer does need to be slapped, though, because literally going through and cauterizing the organizational knowledge base to do that, that thing about going out the door and having your brain wiped, this was not aspirational. This was a dystopian warning [laughs].
WILL: Oh, yeah. Yeah, yeah. Well, that's the plot of Severance, right? The Apple TV original TM. Get some sponsors in here [laughs].
DAVE: I love it. I love it. We have ranged pretty far afield today. We got anything else we want to cover on this? It's been an interesting ride.
VIVIAN: I mean, I just wanted to make a point of something that I kind of noticed was a little bit funny. The topic that we were going to be discussing today was decentralization of authority. And without Mike to kind of keep us on track, to hold us accountable, we had a wonderful conversation, but did we end up talking about or coming to a conclusion about decentralization of authority?
DAVE: Right.
VIVIAN: No. So, it kind of proves the point of why, at the end of the day...
MIKE: [inaudible 48:03]
VIVIAN: Although you can accomplish really great things with a lot of decentralization, if you're trying to accomplish one specific thing, that might fail if there's no one to kind of keep you on track.
DAVE: I love that.
VIVIAN: To keep things moving forward.
DAVE: Yeah. Did we have the wrong podcast, though, right? Did we...We didn't have the podcast we thought we would, but did we end up having the podcast we found out we needed, right? Be the podcast they deserve, not the podcast they need. No, it's the other way around, isn't it? So, yeah, get the quote right.
WILL: So, I really...I mean, I feel bad that I didn't get my rant on extreme programming in.
DAVE: Mm-hmm. I didn't intentionally beat you to the post on that one, but I'm with you, brother. Solidarity.
WILL: I mean, I mean, I'm just saying it's a solution to all these problems.
DAVE: Mm-hmm. Mm-hmm. That actually might be a good place to wrap up then. Thank you, guys, for coming out. Thank all y'all, and thank you for listening.
And this has been the Acima Development Podcast.
This Acima Development Podcast episode connects a NASA document on critical failures to the modern temptation to over-rely on AI. The hosts open with examples of reward-function failures: a reinforcement learning agent in the game Coast Runners that racked up points by endlessly circling a lagoon instead of finishing the race, and an early GPT model that compulsively opened a calculator because its reward was dialed too high. These illustrate NASA's identified root cause of major incidents: lack of critical thinking and over-reliance on computers, processes, and procedures. The recurring metaphor is "playing with your head up," meaning you execute a strategy efficiently while continuously re-evaluating whether it's still the right one rather than blindly trusting the process.
The middle of the conversation explores how much to trust AI, particularly for code review. The consensus rejects all-or-nothing thinking: AI is another tool, like a human reviewer, and catches different things, so the best approach is using both. Dave shares wins (AI flagging a subtle nil-returning bug he'd have missed) and failures (AI ripping out a carefully crafted Arel query to jam in raw SQL, or re-implementing a feature the team had deliberately killed). The takeaway is that AI can't run unattended because it lacks institutional lore, and an experienced human needs to stay in the loop, especially for high-stakes changes. Low-risk, templatized work like certain database migrations are good candidates to let AI handle, applying risk frameworks (mitigate, eliminate, transfer, accept) to decide where it can safely "cut its teeth."
The discussion broadens into how skills evolve as technology shifts. Using analogies of welders replaced by robots, miners re-skilling into adjacent jobs, and the Pony Express being rapidly outpaced by the telegraph and railroads, the hosts argue that giving up certain "critical thinking taxes" (like manually smelting iron, or writing low-level compilers) frees people to build at higher levels, while foundational skills survive in niche "ranching" spaces like microcontroller programming. The unifying through line is proprioception, a visceral, grounded feel for what you're doing. Even as AI tools do more, you can't fully let go of the reins, because grounding in the underlying craft is what lets you sense when something is going wrong.
Transcript:
MIKE: Hello, and welcome to another episode of the Acima Development Podcast. I'm Mike, and I am hosting again today. With me today, I have Eddy. I've got Dave.
DAVE: Howdy, howdy.
MIKE: Ramses and Kyle. It's possible we'll have a little bit of ebb and flow in who we've got here, as we sometimes do, which is great. We have a group here that can go into a good topic.
Just a heads-up: we're going to talk a little bit more about the NASA document we've been talking about on the last couple of episodes. So, as usual, I'm going to start with some background story. This one, I'm going to go more on the technical side.
There's a great example, and you can look it up. It refers to a game called Coast Runners. So, there's a video game called Coast Runners. And some years ago, they built a number of reinforcement learning agents to play it. This is some of the early days of, "Hey, we can actually build AI that can play games, and it's working." Because there were a lot of years where they couldn't make that work very well, and it's kind of some of the first steps, "Hey, this is actually working." It led to some of the breakthroughs where we saw, you know, amazing things happen a few years back in video game playing, you know, including beating human players in a variety of games.
Well, this game, you know, it's not chess; it's not Go. It's a video game where you drive a boat, and you try to win the race. And they built the environment, and they built these reinforcement learning agents to play it. And not surprisingly, the reward function was to maximize your score. And here's where things get interesting. The score in Coast Runners is partially determined by winning the game, but it's also determined by how many of these little boxes that pop up that you can hit along the way. And what the agents learned to do is there's a lagoon that you can go into on the side. And...well, I'm going to read this. This is taken directly from a write-up that OpenAI did on it. It said: "The reinforcement learning agent finds an isolated lagoon where it can turn in a large circle and repeatedly knock over three targets, timing its movements so as to always knock over the targets just as they repopulate.
Despite repeatedly catching on fire, crashing into other boats, and going the wrong way on the track, our agent manages to achieve a higher score using this strategy than is possible by completing the course in the normal way. Our agent achieves a score on average 20% higher than that achieved by human players, [laughs]" which is to say, computers will do exactly what you tell them to do.
DAVE: A follow-on that. In one of the...I think it was ChatGPT, in their training —the early ChatGPTs —if you'd ask a math problem, it would get it wrong because it was just doing words and tokens, right? It wasn't actually conceptualizing the numbers. And one of the things that they started reinforcing was open the calculator. If you get a math problem, open the calculator, type in some numbers, and add it, and when you hit equals, you get a cookie, right? We're going to increase your reward satisfaction score.
And they shipped this thing, and I want to say it was GPT-3, 20 to 40% of queries that you sent, no matter what they were about, it would silently open up the calculator, add one plus one, and hit equals, and give itself a cookie, and then go back and resume [chuckles] and answer your question.
They had prioritized use the calculator instead of confidently guessing off of the words, right? Because AI's confidently [inaudible 03:42] the wrong way. They had dialed the reward up so high that it liked playing with the calculator. And it just reminded me of being a junior in, like, geography class playing [inaudible 03:51], you know, waiting for math class. Bless his little heart.
MIKE: Exactly, well, yes. And the reward will be maximized. So, NASA looked at some of the failures that they had had, and we've been talking about this document they published about a decade ago. And they said that one of the root causes of the key failures that they've had in some of the major incidents was lack of critical thinking. They refer to it as over-reliance on procedures and computer codes.
And there's a longer paragraph; I'll quote from this as well. Sorry for reading the quotes, but they're well-written [chuckles]. It's worth giving some context. "The third root cause is closely related to second. It is the lack of critical thinking with an over-reliance on computers, processes, and procedures. Computers, processes, and procedures are necessary but can never take the place of the human mind. In the end, all of our resources, tools, et cetera, are there as an aid to the human mind and the human creativity and decision-making."
Now, this is fascinating a decade later, where we have AI becoming so much more successful, and we're outsourcing a lot more of our thinking. And if we rely on that without supervision, we get what we ask for. You know, like King Midas [chuckles], we may get exactly what we ask for and regret it for the rest of our lives.
DAVE: There's a fun takeaway to this as well. I'm going to take this immediately and go meta, which is that, like, when I was in college, somebody barked at me and said, "Dude, you need to learn to play with your head up. Did you not ever play basketball?" And I'm like...I was not athletic at all. I don't even know...That's the round one, right?
MIKE: [laughs]
DAVE: And, yeah, right? So, he said, "When you play basketball, what they teach you is don't look at the ball. You have to play with your head up because you have to be watching. You have to be looking at the net. You have to be looking at the other players. You've got to be able to dribble the ball without watching the ball."
That stuck with me because he wasn't talking about dribbling the ball. He was talking about I was writing a piece of software that nobody in the room wanted. And one of the executives who had direct control over my paycheck directly did not want, and it was costing me dramatically politically. And so, basically, it was like, read the room, right?
But the phrase "play with your head up" has stuck with me as, once you've decided on a strategy, you should execute the strategy efficiently. But you constantly have to be evaluating: Is this still the right strategy? When you find a good idea, it's very valuable; but also valuable is when you get rid of an idea that's no longer valuable —that's no longer earning its keep.
And if we say, "Don't use AI because you won't learn nothing," we are at risk of falling into the very same trap that we just outlined because, for instance...I told this story on a previous podcast. I needed an auto clicker in Linux, and I know how to write one, kinda. I mean, I know the principles. I know how to write a timer. I know how to send input to the mouse. I know how to capture keystrokes. I know conceptually that I'm going to need to intercept a keystroke in another application, and that's going to require elevated privileges because that's technically a keylogger, right? And I don't know how to do any of that in Linux.
15 years ago, I could tell you from memory how to do it in Windows because I was a Win32 programmer for years and years. But I never learned how to do it under Windows, under the X11 system. I went to an AI, and I said, "I'm just playing Minecraft, dude. I want to AFK and grind some resources. I just want an auto-clicker, and apt search on open-source repo isn't turning up an auto-clicker. Write me a clicker." And the AI went off for two hours, came back, says, "Here's your auto-clicker." I looked at the code and went, "Yep, that's an auto-clicker," and I've never looked at it since.
Have I just cheated myself out of the knowledge of making an auto-clicker? Yes. Yes, I have. In the same way that every machinist working on an aircraft fuselage is cheating himself out of the fundamental knowledge of how to smelt red earth into iron to forge the...right? That's great if that's what you want to do, but if you're trying to do this other thing, pick and choose your battles and know when to do and when not to do.
I posted in our AI horrors channel a while back that I vibe-coded an AI chatbot, and it did it─Eddy, you'll love this─in React, and I didn't take the time to learn React. I'm like, "I'll just trust the AI to do it." And it ran great for about two weeks, and then the app slowed down, and slowed down, and slowed down. And every time I would hit a keystroke, it was redrawing. It was re-rendering the entire chat in order to catch some live event. It was doing the most inefficient way possible, literally re-rendering every character on a, you know, in a chat that's getting longer, and longer, and longer, and longer, right? And it would just grind the server to a halt, and I found myself unable to debug it, and I ended up scrapping the project.
I chalked it up to a lesson learned about don't vibe code something if you need to understand it. I don't regret doing it, but I definitely...I paid for that one. That one I should have gone to the AI and said, "Teach me how to do this, or tell me where to go learn it." And it's which strategy is right right now, and which strategy...We don't want to just blanket choose this is what we are going to do; this is what we're not going to do. You have to play with your head up because the right course of action is going to change from second to second during the game.
That's my speech. Yeah, so [chuckles] didn't mean to go all soapboxy on it, but yeah.
MIKE: Well, it matters; it's directly correlated here. Because we're talking about...you say playing with your head up. If you're looking at the ball, you'll miss it, and that's the ability to not look around has increased here because we've got tools that can do some of that for you. So, you can get yourself further in the hole [chuckles].
You know, you can go out...you know the old cartoons where somebody would run off the edge of the cliff, and they don't fall until they look around [chuckles]? You get a lot farther off the cliff before you look around here, and the consequences can go up higher. Because it looks like it works, and, boy, there's just so much potential to go wrong because there's so much potential to accomplish things, right? We've been given these big tools that do lots of cool, powerful things. It could sound like I'm being gloomy, but what I'm saying is, wow, we've been given superpowers, and what next?
DAVE: Yeah, absolutely. This actually harkens very strongly back to the very first thing we talked about last week or two episodes ago, or one episode ago, sorry, two weeks ago, which was the normalization of deviance, right, where you get stuck on the process. We are launching a space shuttle. Don't bother me about it's too cold. Don't bother me about some engineer actually knows for a fact that those O-rings are going to freeze and leak, and we're going to shut that guy up and shut him down so that he doesn't have time to think all the way through to, "This is going to leak fuel onto the engines and blow up and kill everyone," right? They shut him down, shut him up, and we didn't bother...
So, it's like, we normalize this. We're going to adhere to the process. Stay on target, stay on target, stay on target. Sometimes you need that. You know, when you have to take that hill, even if you're suffering, you know, catastrophic losses, you have to follow the process. You have to execute it. Artists will sometimes say, you know, trust the process. When you're trying to paint a grassy hill and the first thing you're supposed to do is pick up purple, and, whaat, trust the process. It'll be there. Trust the process, right?
But trusting the process is playing with your head up. Normalizing deviance is trusting...no, is blindly trusting process to keep you safe, and expecting the process to be the sentient overlord that it isn't, that it absolutely isn't.
MIKE: So, we've been talking very meta here, very high level because it matters, right? This is kind of a philosophical question that is going to continue to haunt us here for a while. I'll propose a practical example. How much do you trust AI reviewers to review your code?
DAVE: You're not going to like my answer. It's increasing every day.
MIKE: Well, I didn't say I was...That's a legitimate answer.
DAVE: Yeah. I guess what I would say is, so, I actually led a discussion a couple of weeks ago in our AI channel, where I said, "It's too quiet in here. Let's start a fight." And I said, "We're going to be vibe coding in prod by next year," and it started a fight. Because when you hear vibe code, you automatically think blindly trusting the process of something that's going to be catastrophic, and it's going to da, da, da.
And, like, we have a codebase that is complex and has enough lore and enough decisions that changed midstream and weren't well documented that you need a human, not just, like, the complexity of human thought, but literally, you need somebody who was here 10 years ago and watched the campaigns, who's been to see the elephant a couple of times. And Claude doesn't have that, Copilot doesn't have that.
And so, Claude will come back and say, "Oh yeah, this code doesn't match this code over here because it's missing this feature. Let me go add that feature." And we're like, "No, no, no, no, no, we killed that feature in 2021. But we weren't able to remove the code because we were at a dead run, and we were committed to it. It's technical debt. We want to get rid of it." And the AI, if we blindly let it go, will go re-implement that feature that we decided we didn't want.
That said, I'm getting reviews now, at least one a week, where there will be a detail that I never would have caught. Like, I've had two now where, like, the...and this might be a terrible habit, but when I review code, I read the PR, the diff on GitHub. And I try to think about it, but I don't mentally catalog, like, "Oh, this is one example of a query. There are 37 other queries in the file. How does this fit shoulder to shoulder with the rest of these?"
Claude is the one that will come back and say, "Uh, by the way, there are five queries that share the same shape, and this one over here is now missing this...you just deleted an instance variable, and this one over here uses that instance variable. And because it's an instance variable and not a method call, it's going to silently return nil, and you're going to end up with a blank spot on your UI," because it was in one of the view files, right? And I never would have caught that. And we scooped it up; we put it in.
So, yeah, I don't know. I wouldn't trust an AI to review a sweeping scope architectural change, you know, pivoting the entire codebase from, you know, Postgres to SQL Server, or something like that. I wouldn't just throw that at an AI. But, like, a tiny self-contained change where...Oh, I genuinely don't think database migrations should be touched by humans. I would be willing to, like, if I had budget for two weeks on that, most database migrations, I'll put it that way.
Adding a column, adding an index, some backfills, backfilling old data, you know, that kind of stuff we have templates that we use on our team. We have a standard template for this is the column we're adding; this is the type. We have a standard up migration. We don't use change. We have a standard up, a standard down migration. They have to work a certain way. This is all templatized. There is no reason...I'm almost offended; I'm almost to the point where I'm offended that humans are wasting their lives and their attention on this.
You should go into Jira, right, or whatever ticketing system you're using, and write up a ticket of, "I want favorite color added to merchants. It should be a string, free form, 50 characters. We might make it an enum, but we're not doing it right now. Go. And we don't need it indexed." And the only review...in my opinion, you could teach an AI reliably to make it to the point that another engineer reviews the Jira ticket and says, "Yes, this is good," and half an hour later, it's in production. I don't see any reason we can't build a system that is trustable at every single step.
Would I let it backfill? Probably not. And would I let it write that PR today? No. No, I would hand-hold, walk it through the entire system. And I would also have it add all the missing pieces that we don't have that...because we, as humans, we deploy something, and then we watch all the things afterwards to make sure everything is still stable, right? Those aren't automated. Those should be automated. Not just automated, but they should be watched by an AI that's thinking about, you know, threshold and alarms, and will go find a human when stuff's panicking. And if it can't find a human, might even be, you know, if it was an auto deploy, might even be empowered to auto rollback.
KYLE: I think about this one a little bit different just because...and maybe people won't like my comment here. But at the end of the day, it's another tool, and at any company that you're at, so are you. Like, you're just a tool, right? And so, in my mind, do I want AI reviewing my code? Yeah. But do I also want Dave reviewing my code? Yeah. Both of them are going to find different things.
DAVE: Absolutely.
KYLE: I don't want the pendulum on one end or the other. I want both. I want the best of both worlds, right? And auto-deploying, like, I would be a bit leery there, but, you know, if the AI reviewed it, and Dave reviewed it, and it looked good there, yeah, sure, send it out. But I think we get hung up on that; it's an all-or-nothing. And when we get into these conversations about, like, "Hey, AI will be reviewing your code," cool. If it's in addition to, that's wonderful.
And, I mean, there are going to be specific use cases where, I mean, my eyes are going to look at a DB migration and just glaze over, and I'm useless there, you know? AI is great there, and we probably should rely on it there. But it's like introducing different skill sets, right? Like, sometimes you want a DevOps engineer on your review, sometimes you want QA on your review, sometimes you want devs, IT, you know, depending on what you're doing. It's just another mindset. So, at least that's my take on it. Use all of it. We're all tools. It's a tool.
DAVE: Yeah, yeah. And playing with your head up, right? We're not just going to say, "Oh, we're going to make the AI do all the stuff and stop thinking about it." That's literally what we just said don't do, right?
KYLE: Yeah.
DAVE: You're absolutely right.
We had a PR come through this week where one of our most senior engineers and one of our smartest contractors...the contractor couldn't figure out how to do a really difficult subquery without going straight to SQL. And so, they put a heredoc in there and jammed in a bunch of SQL. And Adam, one of our...Adam [SP]Loper is one of our devs, and he can shatter a human mind at 50 paces. You do not want to step to him in mental combat. I love him. He pushed back, and he said, "We should be able to do this. We don't want to put SQL in the code."
And the developer that authored that PR was like, "But what about, you know..." And so, Adam helped him write the Arel query, not ActiveRecord. He actually had to go to Arel, A-R-E-L, to build it out. And they had to push back and forth on how big the subquery was and the cardinality, and, you know, it got into, like, the kinesthetics.
I ran Claude over it, and Claude said, "Oh yeah, here." And it ripped out the Arel and said, "Here's a heredoc with the SQL embedded." I'm like, "No, no, no. You are helping in exactly the wrong direction," right?
MIKE: [laughs]
DAVE: Yeah.
KYLE: Right.
DAVE: So, yeah, you absolutely cannot let this run unattended, right? And if you find anybody that's managed programmers, just ask them if they've ever had a very enthusiastic, very brilliant junior programmer who's going to show up at 8:00 and plow rows and then go home at 5:00. And if you set that developer down next to the freeway, if you don't watch them, you'll come in in the morning, and they have plowed the freeway, you know, for their eight hours of back and forth. AI's just doing that faster. We're just seeing it on fast-forward, basically.
I don't want to turn this into the AI show because...or maybe we do, I don't know. But specifically, we're talking about, like, the NASA critical failures, and I'm mapping it more, for me, mapping it more into human space of, like, what are the mistakes that we make? Because you can make these errors outside of software. You can make these mistakes in your marriage. You can make them in your career. And that, again, trust Dave to take it meta. I like to see what is the shape of this error? Does this shape fit anywhere else? And what does that teach us?
KYLE: Well, it's just, I mean, it's just one of those cases for critical thinking, right, that we run into, which is we're...Be it AI or something else, we're handing off our critical thinking. And when you do that, you can run into issues; you run into holes.
DAVE: It's the, what part of the critical thinking is going away, right? It's I want to give the AI, like, if I'm a blacksmith, I want to give it the knowledge of how to smelt the ore into usable iron. Like, I don't want to solve that problem anymore. I want to be thinking about building civilization.
I've used the analogy in the past of, like, "Is AI coming for my job?" And I said, "Guys, it's 1970, and we are welders in Detroit, and they are wheeling robots off the truck. Yes, this is going to change our job." One or two of those welders stayed back at the plant and learned how to run the robots, and they had to do all the critical thinking. And they had to get smarter at their own job in order to stay on top of that. But the actual job of, like, stacking dimes, which is what it looks like when you give a really good arc weld down between two pieces, the melted metal looks like a stack of dimes, a robot can do that better than a human. And I'd rather have the robot do it. It's going to build a safer car.
But the humanity, yes, I want to give up the critical thinking of how do we stack those dimes? No, that's not even right either; that's not even right either because the 98 welders that lost their jobs, they hated the robots, right? They put them out of a job, and they were miserable. But then follow up with those people six months later, and they are now running welding on other teams in areas where the robots can't get to. They are welding upside down on the bottom of broken water tanks.
It's like, we put these guys out of a job, but what we did is we freed up 98 skilled welders to build more civilization in other areas. Because the critical thinking that they gave up was also a tax that they had to pay, and the only thing they got out of that tax was stacked dimes on a car frame. And now they're, you know, fixing battleships and water tanks. You know, it's...
This is a personal thing for me because my dad worked as a miner, and they shut down the mine, his uranium mine, in the 1980s. Nuclear got frozen out, and it put him out of a job. And six months later, he was working construction, and two years later, he was a water operator. And those weren't just random re-skill, start over your career. It absolutely was, "I'm going to move adjacently from this related job to this related job, and then from this job to this job." And it wasn't just who do you know, and where do you go work? It was entirely what are the skills that you learned from this that apply here that nobody else has because they didn't have 26 years of blowing up rock a mile underground and mucking it out?
MIKE: Definitely gotten deep into the AI. How does this idea apply in non-AI context? That is, can we think of examples where we've relied too much on the process and fallen really short?
DAVE: Or where the process was outdated, and we didn't inspect the process.
MIKE: Ooh, yeah. I think a lot of...When I was preparing for the podcast today, I was thinking back to a time when I wrote some code I was proud of. I wrote the unit test. It worked perfectly. I went to do some manual testing, and I realized this code works perfectly at doing the wrong thing. It doesn't meet the requirements. I had followed the process to the T, and I had tested it. Everything worked perfectly, and it was wrong. If I had done some earlier [chuckles] manual testing or, you know, something to break out of my thought process, pairing, for example, I could have saved myself some time. Well, a little example.
Do you have all examples of times that you have depended on the process and ended up regretting it?
DAVE: We were talking in the bullpen today about having product people in refinement. And I was arguing vehemently that not only should we have the product people, basically, we should have all the way up to the customer. Like, Agile programming, you take any principle, and you dial it to 11, right? And their idea for the most effective way to make sure you're always building the right thing and the thing the customer wants is to have the customer in the room. You dial that to 11. You have the on-site customer. You literally have the person who's going to use the software sitting with the engineers 40 hours a week, like, literally on site.
And then big companies came in, and they wanted to keep doing Waterfall, so they canonized it. They said, "We're going to firewall this. We're going to have a product manager, product owner, and that person is going to now play telephone and hide the fact that it's Waterfall because this person's going to gather, you know, data from the customer, and then come be the product owner in-house."
That can be an efficiency saving. Some people will want you to write software, and they don't want to come sit with you. They don't want to give an entire employee over to you just to be the authority on a...especially if you've been hired by a one-man shop, that's, you know, that person's the chief cook and bottle washer. They can't be your onsite customer 24/7. So, product owner is a good idea, but if you fall into Waterfall, you can end up with the customer talks to the product owner once, and then the product owner develops some stuff once and hands it off to the team leads, and the team leads come up with a design.
That's the 1980s SDLC that, you know, provably has a track record of failing 90% of the time. Because by the time you are in the debugging phase, you're going to find errors in the design phase. You're going to find out, like you said, Mike, we built the wrong thing. We have successfully put the ladder on...We've climbed the ladder and realized the ladder was leaning on the wrong wall the entire time.
So, yeah, this is a process that we're using here. Our parent company uses a very formal, time-honored software development process that I don't particularly love. It's not super Agile. It's a lot more...what's the word? It's like Scrum rather than Agile, like, Scrum with a dollar sign and a TM at the end of it. Like, it's very enterprise, very process.
MIKE: [laughs]
DAVE: I'm trying not to get myself fired if...No, I'm kidding. I'm kidding. But it's very formalized, and it's very process-oriented, and it's good for working at scale. It's good for having all your product owners talking about the same thing. But at Acima, we were always small and kind of startup-y, and every team was doing kind of its own thing. And you kind of get apples to oranges, and standardizing that is kind of miserable, and you can make the mistake of standardizing the wrong things.
Pulling in this kind of Waterfall-y bit that works at scale, if all the other pieces at scale are present, if all the other bricks in the wall are there, then you can use the same kind of mortar. But if you grab this one brick that isn't well-supported because we don't have the other pieces of that scale in, it's a terrible idea. So, that was one for me.
We ended up with a design decision of, like, how are we going to deal with this particular lease in a specific condition? And the answer was actually something that should have gone all the way back to the customer. Because the decision that we made kind of turned it into, are you viewing the entire purchasing process this way, or slightly this way? And it was a perspective shift all the way back to there. And yeah, we were at risk of building the wrong thing.
MIKE: I've certainly been in that situation before. I worked previously at a company where we built a content management system for mostly small publishers, newspapers, magazines, that sort of thing. And I remember meeting with somebody from the newspaper, and he said, "Yeah, I can tell you're not newspaper people." I'm like, "Why?" He said, "Well, the way that you label these fields is totally different from what newspaper people would say. They don't call it the author; they call it the byline [chuckles]," and so on. Like, "Oh yeah, well, we can change that."
But we were building this interface that was foreign to them because we hadn't spent time just sitting together and saying, "Okay, you know, how do you build out an article?" We'd built what we had imagined that they would want, and it generally worked, but it was a mismatch. It was a mismatch.
And it led to also us building some things that they didn't really need. Like, we improved the interface for them to build the content, what they really needed was, you know, what they needed to do was make money [chuckles]. And so, they needed better ways to make money from their advertisers. I could go way into this, you know, what killed newspapers was not the internet. It was Craigslist because it took all their classified ad revenue away.
DAVE: This is another thing that came from the bullpen. I'll ask...so Ramses knows the answer to this one. The Pony Express lasted for a surprising amount of time. Does anybody know offhand how long the Pony Express was around?
MIKE: Randomly, I've looked this up within the last two weeks.
DAVE: Okay. So, you know it wasn't, like, 100 years, right, or 50 years. It's the other end, right? It was, like, 18 months. It wasn't because it was inefficient or wrong. I mean, it was a brilliant idea. It was an idea so good that it changed the way people thought.
If you were in Kansas City, San Francisco was a foreign country. You heard about it on the news. If you wanted to send a letter to San Francisco, you sent it south to Louisiana to get on a boat to go out of the Gulf down around South America. You would find out, hey, how's my investment doing in the Gold Rush? You wouldn't hear back until next year. And all of a sudden, the Pony Express comes along and says, "Hey, the Mormons have settled out in the middle of this deadly, deadly desert. I think we can actually make a shot across this desert. And by whipping the horses hard, we can not only get across there, but we can do it quickly. We can get a round-trip message, like, what, 10 days?" Something like that?
MIKE: Something like that, yeah.
DAVE: Maybe less. It was a matter of weeks, what used to take a year.
And now you got people in Kansas City that can hedge fund arbitrage and say, "Hey, I will sell you futures on your shipment coming back at this rate, and it's going to sound really, really good to you because you don't know whether they struck it rich. But I do because I've heard, and you're not going to find out for another 10 months."
And it was so good that everyone...all of a sudden, America was...it wasn't two countries separated by Mexico anymore or the Utah territory, this deadly desert, right? It was now all of a sudden one cohesive whole. And we could actually communicate from, you know, Kansas to San Francisco the way we could communicate Kansas to New York, contiguously. And then with the Gold Rush and the hedge fund arbitrage and all of that going on, people said, "This is absolutely brilliant. Let's build railroads and a telegraph." So, basically, the Pony Express was so good it immediately attracted a much better competitor. The Pony Express revolutionized it, and then immediately got out-revolutionized by someone coming behind it.
MIKE: So, we've had a lot of ideas thrown out here of varying connection to the original thought, which is that you need to pay attention to what you're doing and have a...I'm thinking words like tactile or visceral that suggest you have some human sensory connection that you can have a gut response to what you're doing. You can't be sipping drinks out on the beach thinking that the system's working. You have to be engaged, and there is temptation with the tools we have out there, because they're so good, to disengage. But you see, this is over a decade ago that NASA wrote this document. It's not a new temptation.
When things start working well, you think, "Oh, they're fine." Or in the Pony Express era, you think, "Hey, we've solved this problem. We're ahead of this. And we took out a lot of debt to do it [laughs] because it was a great idea," which it was, but now, you know, they're in trouble. You know, it's not a new problem. You have to be engaged with your work, and have, as we've talked about for the last couple of episodes, be actively engaged with your work.
You have to strive because you can't possibly know everything. But you want to have...So, you talked a lot about blacksmithing, and, like, it helps if you've smelted iron once, right [chuckles]?
DAVE: Yeah. Yeah.
MIKE: It just gives you a different perspective on things. This idea of, you know, doing things without having that grounded connection to it is the through line here. You have to get grounded somehow and keep that. Even though your grip is going to be less because these tools are going to do so much more for you, you can't let go.
DAVE: That's actually a good way to view it. I love that, like, the kinesthetic proprioception, there we go, is the sense of your own self, right, and especially kinesthetically. You've been...you said visceral.
MIKE: Simone Biles, she has amazing proprioception. She knows where she is, but she's doing flips through the air [laughs].
DAVE: Yeah, yeah exactly. You need to have a feel for, like, where am I? When I'm developing software, I do. I have a feel for, like, how close to done am I? How close to my next problem am I, or, you know, this thing that I'm trying to solve, how close is this to working and testable? And what can I touch, and when and where?
And all of that are...Kyle, you mentioned, like, we give up some of our critical thinking. All of this is under an umbrella of, what are we doing? And what we're doing is we're shipping software and...Well, okay, but go up a level. What are we doing? We're a leasing company. Okay, but what are we doing? We're a company. We're making money, right? At some level, you go up, and when you're at that level, the things underneath you might be interchangeable skills.
We've all had engineers that, well, I mentioned at the top of the thing. I had an AI write a React website, and it was catastrophically bad. It absolutely...I believed the overpromise, and I paid for it. Executives have had big layoffs last year because AI promised them some things that it's turning out AI couldn't deliver. This is that. They vibe-coded their org chart, and now they're getting caught in the shorts for it. And, you know, we sit back, and we eat popcorn, and we go, "Oh, this is very satisfying." And I'm like, this is the same pattern. And those executives need to smarten up and go, "Okay, what part of my critical thinking did I give away that I shouldn't have? And what part can I keep?"
And at every level, that proprioception is what's enabling me to feel things. Like a database migration, that, to me, feels like something that is nothing but paperwork 9 times out of 10. And I can look at...and I would need a human, and I need to be that human. I can look at a database migration and tell you if it's a candidate for a rubber-stamp AI to paperwork it, right? You know, adding a column, adding an index.
Maybe dropping an index I wouldn't trust an AI to do. This is a little trickier. Actually, yeah, you get the idea. Some backfills, you know, oh, we're going to backfill the leases table. We've got millions and millions and millions of those. No. I want to fine-tooth comb that all the way along.
But I needed to add an index to one of our tiny tables. The table had fewer than 100 records in it, and it wasn't something that got accessed by the customer main. It was something that got accessed on the admin side. I could have dropped that table, and the company wouldn't have lost money. Some people would eventually have been upset, but, like, it wasn't catastrophic. That would have been a fantastic candidate to let an AI cut its teeth on because it could flame out.
You literally could just choose accept out of your META, you Mitigate, Eliminate, Transfer, Accept, the four things you can do with a risk. We could just choose accept. If the AI catches fire, we let it burn down, and we eat popcorn, and we roast marshmallows. And you can't do that with the leases table, right? We do that with the leases table; we're all looking for work. Make sure the AIs can help us write a shiny resume.
Yeah. What are you doing at a high level? How do the skills that you use, how do they feel in your hand? And that, I think, is something that I'm just now realizing, as I'm talking through that, that is kind of what I'm feeling is, an experienced person needs to be over the AI, and that a junior programmer might not have that internalized skillset. And they're going to give away some critical thinking that they shouldn't have, and they're going to learn some lessons from that.
There were some big events early on in my career that I did not learn enough of the lesson from, and I had to learn it again, right? And that's going to happen to a lot of junior programmers. And 10 years from now, the guys that are interns that are coming into AI straight, they're not going to know Scheme. They're not going to know how to, you know, write a Forth compiler, you know, in assembly language to bootstrap a C compiler so that the C compiler can write itself so that they can then develop a new CPU. That's a skill that they'll be able to just go to Google and say or go to AI and say, "Write me a Forth compiler for it." And that's fine. That's smelting iron.
And there will be a YouTube channel of one crazy guy out there going, "Let me show you the old ways of this. We had the ones, and we had the zeros, and that's all we had, and we liked it," right? And, you know, he's out in his blacksmith garage, you know, typing away on a Tandy TRS-80, you know? That'll be there.
And the stuff that's not essential to our job, the smelting iron, is going to be a couple of people. There are still farriers. There are still people who heat iron and bend it around an anvil, and we need them for now [chuckles]. But there are other things that we don't need from, you know, we don't have leeches and [inaudible 37:59]
MIKE: But it's mostly for show.
DAVE: It is a little [inaudible 38:01] yeah.
MIKE: Like, people, even 100 years ago, horses were a major part of the economy.
DAVE: Yeah, that's true. That's true.
MIKE: Today, they are something that you have because –-
DAVE: It's almost a luxury.
MIKE: Yeah. It's a luxury.
DAVE: Or you're in a very poor area, yeah.
MIKE: Yeah, exactly. Like, in many economies, like United States, there are rare cases where you would actually need that.
Well, so my grandmother, when she was young, they didn't have an automobile. They used horses to get around. And now things have changed. Horses are very different because the technology has fundamentally shifted. Likewise, there are still zeros and ones. We have just built compilers that do that part better than we do.
DAVE: Wow! Okay, sorry, just huge epiphany. Huge epiphany. So, I grew up in ranch country. I grew up down in southern Utah. I grew up in Moab before it was a famous place, and that was uranium mine out of town, and then everyone else was ranching. And the ranches were up in the mountains, and so a lot of the terrain was, you know, at 30 degrees. And a horse was the only four-wheel drive vehicle that could get, like, you couldn't get up there on a four-wheeler, you know, unless somebody built you a road. You needed a horse.
So, there were blacksmiths who made shoes for the horses, and the horses needed those shoes fitted. You don't buy a size nine wide. You have it fitted to the horse. So, that skill is still there, and those people are still there. But ranching is getting less and less common, and I just realized where the ranching is in software. And it's still there, and we're still seeing it, and it's in micro computing, micro devices.
You want to write your own keyboard controller? That's going to have a little teeny-tiny Arduino on it. Now, the Arduino that you kids are using that's called a Pi, and it's got a 1080p port, and you can put 32 gigs of RAM in it, and it's a computer.
MIKE: [laughs]
DAVE: But the Arduino still exists. There's still a version, I think, of the Teensy USB out there that only has a megabyte of RAM. You can still buy a PIC chip, a programmable IC. This side of the millennium, I saw a guy program a video poker game on a PIC chip that only had 64 bytes of RAM.
Poker, there's 52 cards in the deck, so he's only got 12 bytes of RAM for your score, you know, how much money you have, how many cards are in your hand, what your current bet is, in eight bytes of RAM. And that's fun. It's delightful to do that. And 40 years ago, that's what Steve Wozniak was doing, and he was inventing the future in a garage with a soldering iron by figuring out how to lay those 8 bytes in just the right way and abuse them. And we only need these 6 bits of the byte, so I'm hiding some extra data up in here because I've got 5 of them. That's 10 extra bytes that I could be, you know, 10 extra bits that I could be using. There's wonder, and there's delight in that. And now, yeah, it's kind of ranching.
But we're not going to launch a GeForce 4090 and a 10-pound PC into orbit and fire it at Jupiter if we can launch a 100-gram computing device that has enough compute to do everything that it needs to do and not be an AI platform or general computing, right? And anything you can do in a general purpose can be optimized down to a for-purpose thing, and that's where ranching is today. That's where the old skills are still necessary. And they're not old. They are still current, and they're still valid, and you have to be good at them.
If you watch the show Forged in Fire, those guys are bladesmiths, and, for them, pushing iron is very definitely an art. And watching them is actually beautiful and entertaining to do it.
MIKE: My son actually does blacksmithing.
DAVE: Oh, right on.
MIKE: He makes knives and other implements. He doesn't have to. It's not a viable career path. They say a blacksmith can make anything but money, you know, like, except in, like, farriers, like you say. There's special cases where it's needed.
Well, I think that's probably a good place to tie things up. We've talked about, you know, the need to stay grounded, how the underlying stuff doesn't go away. And knowing it is still useful because it gives you an edge that allows you to keep that connection.
And if you miss that chance, explore that stuff a little bit, you know, try something at a low level. It'll change the way you think. And keep connected because you let go of the reins [chuckles]. Use some horse there, you know, talk there. It might go well for a little while, but eventually you'll be in trouble.
DAVE: Yeah, you're going to go where the horse wants. And you better hope the horse wants to go where you want to go. I like that.
MIKE: Okay. Until next time on the Acima Development Podcast.
This episode of the Acima Development Podcast centers on "normalization of deviance" — the pattern where small anomalies get repeatedly ignored until they cause catastrophic failures. Mike opens with the Space Shuttle Challenger disaster as the anchoring example: engineers warned that cold O-rings could fail, but their concerns were drowned out by schedule pressure and accumulated tolerance for small deviations. The crew connects this to the Columbia disaster years later, where the same organizational lesson went unlearned, and to NASA's own "Elements of Engineering Excellence" report, which lists not questioning anomalies as a major root cause behind their biggest failures.
The conversation then wrestles with the tension between safety culture and velocity. Will pushes back on pure risk-aversion, arguing that heavy regulation has real costs and that tech's "move fast and break things" ethos has produced enormous value. Dave introduces the META framework (Mitigate, Eliminate, Transfer, or Accept) and contrasts NASA's culture with SpaceX, which celebrates blowing up unmanned rockets because the risk was already accepted and the explosion yields data. Mike reinforces this with an analogy from his kid's rocket-themed birthday party, where different risk levels (model rockets, sugar rockets, thermite) warranted very different safety boundaries — treating everything as maximum-risk would have obscured where the real dangers actually lived. The group lands on a key reframe: rather than trying to control everything, build a monitoring culture that instruments heavily, tests to failure, and pays attention to the signal inside the noise.
The final stretch applies these ideas to current software practice, including AI-assisted development. Matt and Dave debate whether vibe coding will dominate production code soon, with everyone agreeing humans must remain accountable for what ships. Will gives concrete examples of normalized deviance developers live with daily: thousands of ignored compiler warnings (some of which are genuinely dangerous), bloated mobile web performance, and test suites nobody expects to run clean. He notes AI could finally make the ditch-digging cleanup work economically viable. Mike closes by tying it back to the opening theme: entropy is the default, letting things slide is easy, but flipping the culture toward actively watching the data is what prevents small deviations from becoming the next tragedy.
Transcript:
MIKE: Hello, and welcome to another episode of the Acima Development Podcast. I'm Mike, and I'm hosting again today. With me, we've got Will Archer, Dave Brady, and Kyle Archer.
DAVE: Howdy, howdy.
MIKE: I'm going to start with a story, as I typically do, actually two stories, but one funny and one not at all funny. I'll start with the funny one.
My wife, when she was in her late teens, decided to drive with her sister to college. She wasn't going to college yet, but she decided to road trip with her sister to college. And they made sure the car was good the day before, had been doing some maintenance, and they cracked the case of the cooling fan for the engine.
WILL: Oh.
MIKE: So, when the fan was running, it was bumping against this cracked part of the case, so you can imagine the sound of that, not good, right? They actually took it to a mechanic and got kind of a loose sign-off that, "Yeah, well, this isn't going to make the car break, but it's going to sound terrible, and you should get it fixed soon," like, "Okay."
And they drove cross-country [chuckles] with that thing rubbing the whole way. And what they did is they just turned up the radio, so full volume, full road trip. They drove for, like, two days [laughs] with the volume cranked up, just ignoring it. And she's told the story for years. It's funny, you know, everybody in the family laughs. You can just imagine, just turn up the volume, and the problem goes away. That is one way to make a problem go away.
The other story is related to what we talked about in our last episode. And we're going to continue with the topic we talked about in the last episode, which is the Space Shuttle Challenger disaster, which happened in, let me check my dates here, '86. I believe this happened in...
DAVE: '86?
MIKE: '86. That's the date that I was remembering, so 1986. There it is: 1986. So, I actually looked this up. I read about it on Wikipedia. As a kid, I remember watching this [chuckles] in school, and it was, you know, horrifying. So, they had O-rings around the booster engines that, you know, like rubber or rubber-like material. And they had had record cold, I guess, at the launch pad the night before. And that cold caused the O-rings to, you know, shrink and stiffen. And so, in the launch, they lost integrity, so air started getting into the fuel. Eventually, that caused a catastrophic explosion, and the entire spacecraft disintegrated.
I remember the horror of seeing those booster engines just randomly wandering, and there was not anything left of the main craft. It was a tragedy, you know, a horrible tragedy. Anybody who was around that time remembers. I was talking to somebody else like, "Oh, that was our JFK moment," you know? Everybody remembers that. Where were you when that happened?
And it turns out the engineers had warned this might happen, and they were ignored, because there was enough noise in the data. They're like, "Oh yeah, well, there are so many things that can go wrong [inaudible 03:18]
DAVE: Not just ignored, though, right? They were told to stay in their lane.
MIKE: I think that's right.
DAVE: If I recall. Yeah. They were told to be quiet, yeah.
MIKE: The interesting thing about that is there was another space shuttle disaster some 20 years later or so, where Columbia broke up in re-entry. And the diagnosis afterward essentially said, "We didn't learn from the last time." There were likely problems that were pointed out by engineers, and there was just so much pressure to make this thing work that the concerns were ignored, and people died as a result.
And in the document that we started talking about in our last episode, which is...certainly you can look it up yourself. It's titled...this was published by NASA titled Elements of Engineering Excellence. It was published in 2012. They made a list of root causes behind the major problems that NASA had had over the decades previous.
Last time, we talked in depth about the importance of hands-on experience, that unless you have people who really have, you know, kind of gotten their hands in the work and understand it deeply, then you're going to miss stuff.
The second point is what they call normalization of deviances. They also refer to it as not questioning anomalies. I'll quote from the report, "As was evidenced in the Challenger failure, we see deviations, and they're not quite normal, but seem to have no major consequence. After seeing these deviations a few times, we accept them as normal and ignore them. The result is a major failure where the deviation becomes catastrophic."
So, that's our main topic for today, is the importance of questioning those anomalies and being able to see that signal inside, you know, a bunch of noise, because there's always noise [crosstalk 05:18]
DAVE: There are a couple of interesting extrapolations on that as well.
WILL: So, I have some thoughts about, like, sort of, like, these sort of, like, normalization of deviances and ways that it can go wrong. But, like, I suppose, like, and maybe this is just my priors, but I'm very much a believer in, like, a move fast and break things sort of ethos. Like, I'm familiar with heavily regulated, heavily controlled industries where, rightly or wrongly, there are high stakes, people die, right? And let me tell you right now that there is a cost to that. There's a substantial cost to that.
And I do think that technology, in general, is pretty out of control in terms of, like, accountability, right? I mean, if you look at, like, you need a license to braid hair [laughter]. But, like, I didn't even need to graduate high school to do the job I'm doing. I just needed to convince somebody to give me a shot and then not get fired for long enough, and then you're in.
DAVE: And we're writing software that handles people's money for them.
WILL: Yeah, to the tune of billions of dollars, you know? And it's just like, "Yeah, you know, he sounded like he knew what he was doing. Let's roll," you know, which is fun. I think that's wrong. But, I mean, you can't argue with results of the industry that we've been in, right? And I think there's benefits there, and there's a lot of stuff where, yeah, you can let it slide until it blows up. You could do that. That's a strategy. It's a valid [crosstalk 06:56]
MIKE: Well, and not only is it a strategy. It's a critical one.
WILL: Initially, right?
MIKE: Yeah. Well, absolutely. And even in regular life, you can't pay attention to everything. Attempting to do so would not end well, right? Our brain is very good at removing extraneous information. You can't pay attention to everything. So, you have to prioritize what you actually give attention to, and that better be the important stuff.
DAVE: There's a rule in insurance, which is if you can afford to replace it, don't buy the insurance. But if you can't afford to replace it, don't even ask how likely it is that you're going to lose it. You have to get the insurance.
The entire science of risk assessment is getting people to stop thinking about reducing the likelihood of a catastrophic fault and dealing with the case of when it is catastrophic, right?
It's like, if you're going to take, "Oh, this hash collision can happen one in 10,000 times, and it will bankrupt the company," and I turn the PR back to you, and you say, "Okay, well, I've reduced the likelihood to one in a million times, and it still bankrupts the company." No, I'm not going to approve that PR. You're trying to reduce the likelihood of something that will end us all, when what I need you to do is mitigate it, right? The META rules for...M-E-T-A: Mitigate, Eliminate, Transfer, or Accept on any given risk, right? And if we can't accept it, then you have to mitigate, eliminate, or transfer.
And the thing about the Challenger discovery that I love is that it's mirrored by SpaceX. One of their first unmanned rockets went up, and they start cheering. It gets off the launch pad; they're screaming; they're going nuts, and then it explodes. And somebody opens champagne, and they keep screaming and cheering. And the reason...it was unmanned. There were no people on it.
And they were normalizing science. They were saying, "This is successful collection of a data point, and the risk that we assumed was entirely mitigated." Once it was off the launch pad, they said it was all icing on the cake. This is absolutely 100%. We accept this risk. We are in the black for days, right? We can burn this rocket, and it's fine. And they used that to normalize that and create a culture of psychological safety, and let's move forward with this.
But the normalization of deviance is kind of based on this weird thing where humans tend to reach for confirmation, and when you're trying to prove that a rule doesn't hold, the only thing you care about is exceptions to the rule. Is there a thing that violates the rule? Like, I can't remember the name of the rule. There's a really cool psychological test that I learned last week, where you set out four cards with numbers and letters on them. I'll dig it up for later in the call if it's relevant.
But the important thing is, you're like...if I tell you every person in this bar that's drinking alcohol must be over 21 and I ask you, "Tell me if that's true or not," you know that if somebody is over 21, you don't need to know what they're drinking. And if somebody's drinking a soda pop, you don't need to know how old they are, right? But we go for that. You're like, "You're 35. Are you drinking a beer?" That's the confirmation case.
You need to be looking for counter cases. Are you underage and drinking booze? If you're drinking booze, are you underage, right? Those are the counter cases, and that's the only thing you care about when you're trying to prove a negative. When you normalize deviance, you are throwing away the counter cases and grabbing confirmation, confirmation. And, eventually, your META rule, you end up accepting a risk, and it's catastrophic.
WILL: So, one of the things that I worry about, right, is this sort of psychological need for control, right, and, like, people's psychological need for control on emergent systems that are nearly impossible to fully model inside your brain. And, like, all of us can think of examples of really catastrophic failures. We've all blown things up. We have blown the rocket up. And we've blown the rocket up to a degree where it's like, "Hey, you know, we could lose the company. This company could not be a company, and we could all have to work somewhere else very soon." We could all think of those.
And so, the question becomes, right, is there a productive safety culture that can really eliminate, like, really, like, look at these deviances to a specific and scoped way where you can get a level of certainty where the juice is worth the squeeze, and you're not just sort of navel-gazing and being, you know, petty and fooling yourself, right? You're trading velocity for the illusion of control.
DAVE: Right. The fun police or the policy wonks on your team they just want to slow things down. It's the foolish consistency is the hobgoblin of little minds. It's like, we followed every checkbox, and we did absolutely nothing wrong, and that's why the company went out of business, because we didn't make any money. Yeah. You're focused on the wrong things.
WILL: Well, yeah, absolutely, absolutely. And it's just, like, killing your productivity, killing your velocity. Like, I have run into this in many respects where people will...One thing that I've seen go dangerously awry is people's focus on shallow indicators of code quality, you know? Like, where every [inaudible 12:31]
DAVE: 82% C0 code coverage.
WILL: Yeah. Well, no, I mean, I don't know. Like, where you'll go through, like, three rounds of code reviews with no substantive architectural improvements, right, like lateral moves because people don't know what's important and what's not important, but they definitely want to put their stink on it, you know?
MIKE: Well, I've been thinking a lot about this importance thing that you brought up, because it matters. So, I've been thinking about analogies. So, I haven't thought about this in a while. When my oldest was around eight, we threw a rocket-themed birthday party, and we had fun. Everybody who came made little model rockets and launched them. And we made what they call rocket candy. You mix sugar and potassium...
DAVE: The sugar rocket fuel, potassium nitrate?
MIKE: Yeah, potassium nitrate.
DAVE: Sugar rockets, yeah.
MIKE: Yep. And we made some of that, and lit it on fire and watched a big fire [laughs]. And we made some thermite.
WILL: Oh! [laughs]
DAVE: I want to go to your birthday parties. Holy crap.
MIKE: [laughs] And lit that with magnesium ribbon, and that was fun having molten metal [laughs] in the backyard. And I'll tell you, so the model rockets, heavily controlled. Those things have been, like, standardized for I don't know how many decades. They made them. They put their own engines in those, and they were able to launch them, and everybody laughed, and it was great. So, you know, eight-year-olds wandering around with rockets in their hands.
The rocket candy, everybody was at least 10 feet away, right [chuckles]? It was enough. And the thermite, everybody was at least, like, 30 feet away, right [chuckles]? They were on the other side of the property. They could see it, and they could see the fire. There was no eight-year-old anywhere close because not safe.
And there were very different rules applied for each of those different risk levels, and that was important to identify because if I had tried to make all the eight-year-olds obey those thermite rules with their rockets, they'd have had no fun, and they wouldn't have known where the boundaries really were, right? They wouldn't have known, "Well, I actually need to be careful of this," because you're treating me like this rocket is super dangerous that's not actually that dangerous, you know, there's a lot of controls around it. And so, I don't really know where the boundaries are. And it was really important to establish ahead of time, well, what's sensitive and what's not? Because that allowed us to pay attention to the things that really mattered.
MATT: Thermite and firearms, favorite games.
DAVE: Thermite and firearms, yep. There's an interesting...It's not the reverse of it. It's not even the countercase. It's an agreement in it in, like, the shadow of it, which is kind of going back to, like, the Challenger disaster. We had normalization of deviance, and it isn't that cold O-rings kill people. It isn't that foam is falling off a shuttlecraft is deadly. It's the bigger thing hiding behind it that you're normalizing this thing. But if you're not paying attention to this, what over here is even bigger that you're not paying attention to?
And I had two things...I learned this lesson really well because I had it happen in two places kind of at the same time, information that came in. One of them was I was in a fast food restaurant. We walked in, and it wasn't busy. All the tables were filthy, and it wasn't, like, immediately after the lunch rush. And the guy that I was with, he's like, "No, we're going." And he turned around. I'm like, "But, I mean, what are you talking about? The food here is pretty good." And he says, "No, come."
So, we went to another restaurant, and I'm like, "What was all that about?" And he says, "Here's the thing. If they're not cleaning the tables out in front where we can see them, what do you think the kitchen is like where we can't see it?" I'm like, "Oh, that's a really good point."
That same week, I had started a new job at a company in Salt Lake City that...they're like a Groupon clone. They were doing financial, you know, manipulation stuff, like, batching together coupons and stuff and deals for people. And the CTO had a really cool hip line of like, "We move fast, and we break things, and we only fix it... We don't polish rivets here. We only fix things good enough to ship it." And I'm like, "All right. Yeah, let's get some money. We're a startup. Let's absolutely do that."
So, I hired on, and my first day, he's like, "Okay, cool. We need to get you on the board for the pager." I'm like, "Okay, well, pager isn't my big thing." And I said, "How often does the pager go off?" He says, "Oh yeah, you're going to need the pager every night because every night, you have to reboot the server at two o'clock in the morning." "Your production server goes down every single night, and you consider this business as normal?" "Oh, yeah, it's totally fine."
I turned in my badge. I walked out. I quit the same day, the first day. And the pager was the thing that did it. And it wasn't the pager; it was, if this is okay to you, you've just told me a lot of things about how much you value my good night's sleep and my value as a person, but also everything else in the company.
And when I found out a year later that the CEO was on trial for financial fraud, I'm like, "This surprises me exactly not at all." Like, everything in that company was move fast and take what you want, and hope you don't get caught.
WILL: You already said Utah startup.
DAVE: Yeah, yeah, exactly. Utah startups, yeah, yeah. Tell people --
MATT: I feel like [inaudible 17:48]
MIKE: Acima was a Utah startup [laughs].
MATT: I feel like we've crossed paths 20 years ago. And I'm sure of it now. Because I built one of those companies that was doing the same thing back at the same time.
DAVE: Oh wow.
MATT: That was acquired by the other company.
DAVE: Oh, interesting. We'll have to talk offline.
MATT: And the founder also happened to end up, I believe, in prison. So, yes.
DAVE: Yeah. We'll have to talk afterwards. That's fantastic. That's the other fun thing is that the Ruby community is a small, small world. Yeah.
MATT: Going back to what Mike said and then you extended on, it's constraints, right? And then we're talking about architecture. And while things may not fail on their own, when you put them together in a systems architecture, and then you apply pressures, that's when you start to see failures, right, of those constraints. And I think a lot of people overlook that architecture as a whole and are losing sight because they can't see the forest above the trees, or through the trees rather.
WILL: Yeah, but, you know, I guess, like, and I don't know whether we'll be able to, like, resolve this to, like, a satisfying conclusion. But, like, people talk about, like, the Challenger disaster. This guy was talking about, right, I think it was foam, right? Like, foam was falling down and damaging the heating tiles, right? And that compromised the thermal integrity of the space shuttle, which made it blow up, right? And they told him, "Hey, shut up. Shut up. We got to get this thing in the air, you know, whatever. The project's got to go," right?
And so, we all, because of the tragedy and the benefit of our beautiful hindsight vision, we're all like, "Oh, well, obviously, this man is a hero. These evil, greedy executives were the villains," you know what I mean? And one guy wears the white hat, one guy wears the black hat, and then boom, [claps], put a bow on it, ship it. But, I mean, just because, like, I am the way I am, I always think about like, okay, well, how many times did the executives say, "Shut up. It's fine," and they were right? You know? You know what I mean?
And, like, I don't have that information, but I do know for certain that there is this temptation for all of us to assume that if we just do everything well enough, it's not going to blow up in our faces because we can have it under control.
MIKE: So, I want to take that and combine it with the SpaceX example that was brought up before. It's going to blow up. If we're talking about rockets, yeah, they're going to blow up. You're not going to start a rocket company or, you know, a government rocket program that's not going to have a lot of things blow up. If you go into it with that mindset and start blowing things up on purpose, say, "Yeah, I'm going to have blow...these things are going to blow up," it changes your approach to the problem versus saying, "I'm going to try to control absolutely everything so that nothing will blow up."
MATT: Try to test your failures. Push the [inaudible 21:15].
MIKE: Exactly. Yeah, fail on purpose. Learn from it.
MATT: Yes. And I'm wondering, you know, and I don't, again, like you just stated, Will, I don't have the information. However, that foam may have very well passed temperature testing, right? However, you add velocity to that ; did they test it at velocity? Because things get fed more oxygen. They ignite more quickly, and, exponentially, things go bad.
So, it also illustrates test your edge cases, right? That's an important thing. You can't always predict edge case. But as Mike just stated, you need to try, right? Try to determine your failures. Try to test those failures, and you're going to have much better success than just saying, "Okay, no, we want to make it perfect. Here's our MVP. This is best-case scenario. Everything's successful. Let's send it."
MIKE: So, the testing to failure is very different from testing that it meets certain parameters. If you test, "Oh yeah, it didn't fail within these parameters," and the failure point was, like, 1% away from that, you have no idea whether it's 1% away or you've got, you know, tons of headroom, you know.
MATT: That's right. Test it till it breaks.
MIKE: Yeah. And that approach, that change in mindset, that very fundamental change in mindset is a big deal. And it's kind of the difference between waterfall-style software development and agile development is, in one case, you try to control everything and inevitably fail [laughs]. And the other approach you say, "I can't control everything, so I'm not going to try to. Instead, I'm going to take an alternative approach where I build a small prototype, test it out, and go into a loop so that I know far more about the process as I'm going on." So, you plan...You're still planning, but you're doing just-in-time planning rather than attempting to cover all your variables before you could possibly know all the details.
WILL: Yeah. And, like, some stuff you got to test in prod. That's one of the things, I mean, like we talk about, like, SpaceX, right? I believe, you know, we return to the analogy, right, where they blew up that rocket, and they were so happy about it. Like, they were pretty sure that rocket was going to blow up. They didn't want the rocket to blow up. I don't think they were trying to blow the rocket up. They were trying really hard to not blow the rocket up. But even still, they were like, "There's no way fucking way this makes it all the way," you know? And so, when it got off the launch pad or whatever and it blew up, you know, on the first stage decoupling, and it was just like, "That's a great win."
I think that's...they've embraced, like, you know, the futility of the illusion of control, where, like, you just can't test a rocket on the ground. You can't do it.
MATT: No. And you can't predict everything. I mean, let's face it, this is reality. There is no way to predict every variable. And, you know, some of us on this call witnessed Challenger. You know, I remember sitting with a group of children watching it live on TV and then watching it happen, and I will never forget it. I can picture everyone next to me, their face, the reaction, you know, similar to 9/11, same thing. But you can't predict everything. But you can force failure.
MIKE: So, if you set up a [inaudible 24:56]
DAVE: Yeah. So, at the end of the day, we're all testing in prod every single day.
MIKE: Well, yeah. But if you accept a culture of monitoring where you are looking at the anomalies and paying attention to them [laughs] and doing something about it, this is kind of where we launched this conversation, right? Then rather than trying to...It's the opposite of control, right? You assume, I don't have control, so I'm going to watch everything I possibly can to see when things start going out of bounds. So, you develop a monitoring culture rather than a control culture. And I think that's a big deal.
Like, we talked about SpaceX. I'm sure they had all kinds of instruments on that rocket that blew up to figure out what went wrong in every possible way [chuckles]. They didn't know what would go wrong, but they knew something would, so they instrumented that thing to death. "Let's look for all the anomalies we can see." And the next rocket, I bet they did something for almost all of them.
MATT: I think this speaks to culture as well, you know, NASA versus SpaceX. And I will admittedly say that I am a fan of what Elon's doing. Like, I will not hide that because he is innovation king. But you operate under government regulation, bureaucracy, constraints, and then you go privately held with someone who's a visionary, wants to push boundaries. You see the success rates, right? And those success rates are exponential with what SpaceX can do versus what NASA can do. We haven't... I mean, we're, as far as I know, we're still on x86 architecture on the space shuttle, and I can guarantee you SpaceX is not.
MIKE: Well, and that culture there, you know, we can ascribe it all to one guy, but it's not, right? I mean, there's Gwynne Shotwell [inaudible 26:45]
WILL: I'll give Elon credit.
MATT: He's the visionary behind it.
MIKE: I'll give him credit. I'm not taking away all the credit. But --
MATT: He's the visionary behind it.
MIKE: You can't just have one person. One person is not culture.
MATT: No, but it starts at the top.
DAVE: Well, and we live in a world of identity politics right now, and I think a peace offering we can say on both sides of the aisle is that the people that don't like the identity have a problem with that person, right? With Elon. I don't hear anybody on either side being upset about electric cars, or about having a space program, or maybe getting off the planet and saving humanity, or dealing head-on with the AI potential extinction of the race. We like what's going on. We like what's coming out of there. So...
WILL: Well, you know what I mean? Like, I actually, like, I really like this analogy because I think as you look at other things, you can see this culture. You can also see the limitations of it in that he wanted to build out a rocket company from scratch, right? And he wanted to do it in the most capital-efficient way that he could. And he, I think, correctly ascertained that, like, okay, the fastest way to get to orbit is to test it in prod, right? So, blow up a lot of rockets, right?
Like, I'm not going to spend years and years and years in the wind turbine, you know, like all this stuff. We're going to shoot some rockets off, and we're going to blast them into space. We're going to see how things go. We're going to learn and iterate very rapidly, right? And they're all going to be unmanned rockets, which... initially at least, right, NASA couldn't do that. That wasn't on the menu for NASA. Or well, I mean, I guess that's not true --
DAVE: Well, because, like, Sputnik and the [inaudible 28:33] stuff, sure they did, yeah.
WILL: Yeah. Well, no, no. When NASA got started, they had a lot of acquired experience with one-way rockets from...
DAVE: That's fair.
WILL: Very [inaudible 28:34]
MATT: Yes, yes, yes. I know where you're going with that one.
DAVE: Yes, yes.
WILL: They were doing one-way trips almost from the very beginning. But regardless, right, the point I want to make, though, is there comes a point where this move fast and break stuff thing and the complexity around the emergent system starts to consume you, starts to swallow you whole. And what we have seen, like, I'll go ahead and call it.
I don't want this to be the Elon show, but, like, they've been advertising robotaxis for a very, very long time. And I think that system, the complexity has gotten out of hand on it. And I don't think those robotaxis are coming because I think the move fast and break things, iterate quickly, kind of messy architecture culture has...I think the tech debt around autonomous driving has completely stalled out their progress. I think they're stuck, frankly.
MATT: Well, I think...and you kind of led me to a perfect segue here. And I'm going to go extremely, extremely old school and maybe a little bit off topic, but it takes visionaries to change the way we do things. I'll go back centuries: Leonardo da Vinci pushing the boundaries, trying things that everyone else thought he was absolutely insane. Next, Nikola Tesla and how he was obsessive, and it destroyed his life. He died broke and alone. But he changed absolutely everything for the world, right? And we need that. You can't get stuck in technology, bureaucracy. You need innovation. You need to push boundaries. You need to test outside of those boundaries to really make progress.
And I think, to me, and, y'all know me, that's the most important thing there is to me when it comes to the world of technology and the things I do and what I'm trying to push. And sometimes I'm going to be wrong, but you have to be wrong to become right.
MIKE: Well, let me take that. So, you talk about the robotaxi and the visionary. Yeah, I think you have to be a visionary, and sometimes you have to admit that you're wrong. I think that, yeah, the robotaxis not been successful, and part of that is it's been thus far technologically impossible [chuckles]. There are challenges to making that happen that nobody has solved yet. And --
WILL: What are you talking about? They're done. You can ride in one.
MIKE: You can, with Waymo, because they didn't say, "Hey, we're going to end-to-end learn this." They said, "It's not within modern tech, so we are going to have a really sophisticated 3D map of the environment we're going to work in. We're going to use LiDAR on top, and so we're going to use some algorithms to locate where that vehicle is every time given that 3D map. And all that vehicle's going to do is follow the map."
And that's what they do, and then they just use a little bit of the machine learning. I mean, they still have to have the vision to look for a collision. So, they're doing some collision avoidance, but they're solving a different problem. They decided this tech isn't here, so let's solve the problem with technology that actually does work, and so they're successful. So, they've approached the problem a different way.
Now, you need to try stuff that fails sometimes, right? So, I think it was great to say, "Hey, let's try this with end-to-end learning. Let's see what we can do." At some point, you might need to realize it's going to bankrupt your company trying to do it because it's not going to work. Sometimes it works; sometimes it doesn't. Yeah, you need to experiment, and sometimes it's not going to work.
MATT: That started something revolutionary, though. Yes, they have constraints, right? They can only do it in x amount of cities where roads are in certain conditions, because of LiDAR, and, you know, collision detection is vector maps and machine learning gets a little scary, to me, because probability versus determinism. But you have to start somewhere.
MIKE: You do have to start somewhere.
MATT: And what they're doing is going to revolutionize the industry, and it's going to change the way we navigate the roads.
WILL: Tesla?
DAVE: I think we're solving it from the other end, much, much farther than we've ever been.
I overheard Uncle Bob Martin talking. He's got a Cybertruck. Now, I don't like the Cybertruck. That's a personal aesthetic thing for me. Honest, I joke with people that somebody designed a nice, beautiful SUV, and they modeled it in 3D, and they accidentally sent the bounding box to the fab of the render [laughter]. And that's what they got back.
But Uncle Bob owns a Cybertruck, and he put on Twitter a little while ago that he's put, like, 100,000 miles on it in three years, and 80% of it has been auto drive, and, like, he won't live without it. For somebody to be that much of a road warrior and to straight up say, "80% of this is solved," we've never been that far, and every year we get closer and closer.
And Waymo, they have to solve that other 20%, and, like, 5% of it is, like, road construction problems that LiDAR can't deal with, so they cut that off. They probably just won't deliver you to those areas, right? So, they stay within that. And we're getting it closer and closer, and that's what kind of excites me.
Circling back to AI a little bit, I said, like, five or six years ago when the self-driving cars were coming, I joked to somebody that, like, we look at AI, and we say, "It'll never be there. It'll never da, da, da, da, da." But our kids are going to talk to each other and go, "Can you believe Grandpa got in that 2,500-pound machine of death and controlled it by hand at 100 feet per second? Are you nuts?" right?
We're seeing this with vibe coding. We're past the tipping point. There are companies now that are literally saying, "Why would you let a human touch the crypto code?" Or, "Why would you let them touch this piece of the security stuff?" And by next year, like, 80% of software...I don't know if it's 80%.
WILL: [laughs]
DAVE: But anytime I make these bets, I always take the under, and I always win if I take the under because it's going to hit, and it's going to hit faster than I think it's going to. And I'm calling it, like, next year that over half of the code that we do in prod is going to be...it's not going to be vibe code. We're not going to use that word because it's a four-letter word, but reliable automation in prod that's handled by an automated system with, you know...And I don't want to get into that. It's a different podcast.
WILL: Not a chance. Not a chance.
MATT: However, I will get on that bet with you.
WILL: No way. Not, not --
MATT: Just based on some of the things I'm aware of.
WILL: I would say, like, I don't know, I mean, maybe it's just the domain that I work in, but, like, 80% of the code that I write these days is generated by an LLM. But there's not a snowball's chance in hell that that LLM is ever going to replace me. The LLM cannot exist without me.
DAVE: Oh, yeah.
MATT: No.
WILL: I can exist without the LLM.
MIKE: And nobody's arguing otherwise.
MATT: Yeah. I don't --
DAVE: I think we're all in violent agreement here, yeah.
MATT: Yes. Nobody is replacing humans. At the end of the day, there has to be a human accountable for what goes out. Accountability and ownership is 100% important. Like, you cannot avoid that; otherwise, we end up in chaos, right? And then we see drift everywhere, and hallucinations, and Wild Wild West, worse than we've ever seen in the history of humanity. Like, there has to be accountability.
DAVE: You've just named the next problem that we have to solve, yeah. Every time somebody says, "AI can't draw a hand with fewer than six fingers," the AI community says, "All right, bet. We'll see you in two more model revisions."
MIKE: Well, and I think you just --
MATT: And every week, it changes.
MIKE: You just brought it back to where we started. If we have a culture without accountability, then bad things happen. But if you --
DAVE: That's normalization of deviance, yeah.
MIKE: Normalization of deviance. But if you're watching this thing, and you're developing all kinds of metrics to say, "I want to make sure this code has high quality," and you're establishing those standards and building the constraints to make sure that it is high quality, then you can get to that confidence because you watched it.
WILL: Well, I mean, and, like, one thing that I'm very, very excited about AI in that one thing that it is good at, really good at, is, like, just petty ditch-digging work that people cannot stand. And I'll give you an example of, like, sort of, like, a normalization of deviance in ways that I think big and small, right? We need to, like, you know, we're asking, like, is the juice worth the squeeze? Well, in certain capacities, absolutely, it's worth the squeeze.
I'll give you one easy and one hard example. Like, one thing, I'm looking at a build for a product that is getting a lot of usage, and right now I see 3,874 compiler warnings, of which I am certain at least 3,000 of which are completely spurious, completely nonsensical, totally useless. 500 are nice to have, get out ahead of this deprecated API, you know, you can get around to that. And 174 are a big, big problem. And I don't know which ones are which.
And it'll be a real hairball, and, like, you know, in all honesty, right, because we've normalized deviance to think builds, to think [inaudible 39:02] that pass QA, right? And it's just like, "Ah, that warning's just a warning. It's just nothing," you know what I mean? "Let's tape over that check engine light because I got to get this release out on time."
And that's a big problem because I guarantee you there are at least 100 warnings in there that are bombs waiting to go off. And everybody's build is like that. Everybody's test suite is like that. Everybody's...There's nobody who's like, "Okay, I'm going to run my test, run a real, you know, rake test," and, like, that output comes out squeaky clean. You know it doesn't. You know it doesn't. But it could, and maybe it [inaudible 39:43]. And that's a normalization of deviance, like a real serious problem.
DAVE: Broken window syndrome, yeah.
WILL: Yeah. Well, I mean, like, there's no reason that we couldn't clean these out. I wouldn't even know. There's a really good reason. Development time is expensive, and the juice isn't honestly probably worth the squeeze.
DAVE: It's not always. Yeah, it's always a long tail or Pareto's rule, yeah.
WILL: But my helper monkey, he doesn't get tired, you know? As long as the lights are on at the data center, he'll clock into work.
DAVE: As long as I've got tokens.
WILL: I have to check the GB thing, which is going to be burdensome. But I'm saying that's an easy one, right? And that's an easy win of like, "Oh, look, we can fix this. We can work this out."
I think a more significant issue is due to the continual degradation and debasement of my intellectual and emotional health, I don't keep a lot of apps on my phone for consuming media, and I use the internet as it stands, right? Like, if there's, like, a Reddit article that somebody sends me, I just look at it on a mobile website.
And I don't know if you guys have noticed this, but, like, the mobile web, like, the internet in general, is in a dire dumpster fire state. It's not like the internet does new things. It's just terrible. It's terrible, and it's gotten bigger and bigger and bigger, and worse and worse and worse. Page sizes have gotten larger, and APIs have gotten slower and more bloated, and we're loading more stuff for no reason. And, like, all of our performance is degrading and degrading and degrading and degrading and degrading. That has absolutely direct financial business impacts. And every single organization I have ever been a part of or interacted with on any level has suffered from this. Normalization --
MATT: Yeah, and I would --
WILL: "Oh, well, it's only 10 milliseconds slower," you know? "It's only a megabyte bigger."
MATT: I won't get into the psychological and physiological aspects of that, but there's definitely an impact. Synapse are being reprogrammed, and attention spans are 30 seconds when they used to be hours. And, you know, the world has changed.
MIKE: Well, we're kind of scratching on this into a different topic. So, I'd like to bring this together, this idea of normalizing, you know, these deviances. We've talked about changing culture, right? To having a culture where we pay attention to the data, and that changes things when you do so. And it's easy to let it slide. The default is to let it slide, and the entropy happens, right? But if you flip that and say, "Well, we're going to focus on watching stuff," it makes a fundamental difference, and can even, in some cases, lead to avoidance of tragedies. And I think that's a good place to end this.
Until next time on the Acima Development Podcast.
From the publisher's feed