Working Software

In this episode of the Acima Development Podcast, Mike hosts a roundtable with Eddy, Dave, Chloe, Jordan, Ramses, and later Will, to explore the differences between small and large projects. Mike kicks things off with a personal story about industrial dishwashing as a teenager and how that shifted his perspective on “small” problems like doing dishes at home. He ties this to software by noting that scale fundamentally changes how you approach problems, whether that’s navigating your neighborhood versus traveling cross-country, or building a small app versus managing a complex, multi-team system. The theme: what works at one scale may feel trivial or even inadequate at another.

The conversation then moves into real-world engineering trade-offs. Dave contrasts working in application development versus database engineering, where different constraints (compute vs. storage) flip what’s considered efficient. Eddy stresses the importance of incremental, low-risk changes in large systems, while Chloe shares how her internship taught her that upfront design decisions carry far more weight in big projects than in small, flexible ones. Will and others riff on the “temptation of perfection”—the idea that if you just planned enough, you’d get everything right upfront. Instead, they argue for agile approaches: accept mistakes as inevitable, design for change, and value rework as part of progress. Several panelists tie this to the importance of embracing ignorance, asking “stupid” questions, and adopting a growth mindset to avoid paralysis when decisions inevitably prove imperfect.

The group also digs into the human dimension of scale: coordination. Jordan notes that large projects require consensus and approvals, whereas small projects let you move unilaterally. Mike reframes bureaucracy as a natural outcome of needing structured communication and shared understanding, not just red tape. Dave and Will debate decision-making heuristics—how long to defer choices, what counts as “enough information,” and how to balance planning versus execution. They conclude that small projects build the fundamental skills and intuition needed for larger systems, while big projects demand humility, adaptability, and collaboration. Ultimately, the episode emphasizes that success isn’t about perfect planning but about making good-enough decisions, coordinating effectively with others, and being willing to course-correct as conditions change.

Transcript:

MIKE: Hello and welcome to another episode of the Acima Development Podcast. I'm Mike, and I'm hosting again today. With me, I've got Eddy, Jordan, Dave, Chloe, and Ramses. Now, Jordan and Chloe are our interns. You all were on before, right [chuckles]?

CHLOE: Yeah.

MIKE: So, it's not the first time. It's not the first time here. But today is the last day of the internship that we had for this summer. And as a result, we're choosing a topic that I thought would be really relevant, something that we've talked about some with the interns, if they can give us some insight as to what they've learned. And having somebody new on a problem often reveals things that other people don't see. That's why sometimes grad students make good teachers [chuckles]. They're fresh to it, you know, they haven't been so distanced. So, I'm really interested in hearing about this.

Notice I haven't mentioned the topic yet. I'm going to give a little intro to introduce here. And I'm going to talk about a summer in my teen years where I worked at a Mexican restaurant [laughs] washing dishes, not just washing dishes. They also made me a prep cook and I heated up, like, beans [laughs]. But I washed a lot of dishes, and this was, you know, industrial dish washing. You can imagine what the pans looked like after they've been cooking whatever Mexican food they were cooking [laughs]. Just imagine, you're probably right.

EDDY: Did you use a pressure hose? It's probably the best way to clean dishes, especially large dishes.

MIKE: So, we had two things that happened. So, there was the customer dishes that came in, and those we had a big industrial dishwasher that we kind of did a conveyor belt style. You’d load them in, and they’d go in there, and they’d steam, and spray a bunch of water, and that worked pretty well. Except if you have, like, cheeseburgers, you know, cheese would melt onto it and stick to the plates [laughs]. I see Chloe smiling. She may have done this before [laughs].

And on the other side of the area where we did the washing, there was a series of three sinks, and that's where all of the stuff from the cooks, right, that's where that stuff would get cleaned. And that was the gnarly stuff. And a pressure hose...there was nothing that would take gunk off a pan when it first came over. And this wasn't, like, very authentic Mexican food [laughs]. They had, like, I don't know if it was blackened, you know, Cajun, but along those lines kind of fish that they would do. And they’d burn it all on the pan, all the seasoning that they would have around, and then we'd have to clean that off. So, imagine burned on fishy oil with spices and gunk. So, first sink was a soak with lots of soap. Second sink was for rinsing. And third sink was bleach. Soap, rinse, bleach.

And I’d spend hours every night. Well, before I ended up, like, hosing down the floor and stuff. Weird smells come out from behind kitchen equipment when you hose them down at the end of the day [laughs] [vocalization]. It's unrelated, just something I think of sometimes, that smell [laughs].

This is this industrial operation, right? And I would spend hours doing this, you know, the soaking and then you go to scrub the thing and if it wasn't coming off yet, then you’d go back in the soap soak. And, eventually, it was like magic. You’d soak it long enough, even the worst caked on stuff would eventually come off. And you'd rinse it and get it over to the bleach and out, and rotate back around to the cooks. And I got used to this. I'd been doing this for weeks.

And then came my rotation, remember, this is my teens, came my rotation to clean the dishes at my parents' house. And I went to do the dishes, and I just started laughing. It's like, what are these? Toys [laughs]? It's a little scrub brush [laughs] and, like, a sponge. Like, I used to think this was a chore. This is going to take me five minutes. What kind of problem is this [laughs]?

And it totally changed my perspective around doing the dishes. This is an entirely different job. I have different tools. I don't have real tools anymore, like, and I had to, you know, find the scrub brush. Like, okay, the scrub brushes work better at this than the sponge [chuckles], but still, it just all felt like toys. I laughed. Every time I did dishes that summer, I laughed because it was just so much of a different problem than what I was doing at work.

Today, we're going to talk about big projects versus small projects. And we're not washing dishes [chuckles], but maybe somebody's doing software for, like, a microcontroller for a dishwasher or something. So, maybe somebody out there is washing dishes. But a lot of these principles apply. The scale really matters. And this comes up often, actually, in our podcast. And you’re like, oh yeah, scale matters. It came up recently when we were talking about vendors. If you are very differently sized from a contract shop, then you're probably not going to get the right level of service because they don't cater to your needs. And when you're building stuff, scale matters, too.

And I've got a lot of thoughts on this. You know what? I'm hosting, so I'm going to throw one other thought out here: Navigating inside of a neighborhood. In fact, this is something I've seen in my neighborhood. Where I live the most prominent landmark is the village water tower. We've got a big water tower. It's one of the larger ones in the region, and you can see it from almost everywhere in the neighborhood.

We have a daughter of a friend who once got lost because she got into a part of the neighborhood where she couldn't see the water tower. And, you know, there's a little bit of hilliness, and so she got to a low spot. It's like, I don't know where I am [laughs]. And she was lost for a while because she had nowhere to go.

DAVE: GPS is unreachable.

MIKE: Yep [laughs]. And, you know, she may have been, you know, she was probably, like, she may not have had a phone. This was probably 10 years ago, maybe a little bit before everybody had a phone [chuckles]. She didn't have a GPS. But yeah, the water tower is the GPS that that's where you're going --

DAVE: That's what I mean, yeah. That’s what I meant, yeah.

MIKE: The water tower is the GPS. And so, yeah, she didn't know where to go because navigation is different when you've got a small scope versus a large scope. Once you get outside of that, well, you actually have to know which way you're going. You have to know where your cardinal directions are maybe, right? You have to recognize more landmarks than one. And, you know, traveling across the country requires a different set of skills than walking around the block. They both require skills, but one of them is fundamentally different.

And a lot of times it is hierarchical in that some of the skills that you would apply for that navigation of the, you know, of the small scale map may apply to large scale map, but you're going to have to say, “Okay, well, I need to get from Chicago to Des Moines.” And you figure out at a high level where that's going to take. But then when you get down to the actual implementation, you know, you have a detour. And then you switch down to a different level, and then you have a smaller map, a smaller map in your mind that you're dealing with. So, your hierarchy changes. So, you have to break down the bigger problem into smaller problems.

I wanted to throw out a couple more ideas to kind of get the thoughts. Also, Will has just joined us. Will Archer, welcome.

DAVE: Welcome.

MIKE: [inaudible 08:22] So, glad to have him with us. So, I've talked about big problems versus small problems. What are y’all’s thoughts? You know, open discussion.

DAVE: I will throw a weird curveball into this, which is that I just came here from a meeting with our DBA, Bill. And I think I've mentioned this before that, like, when I first went to the data team, it shocked me that on AppDev, we spend all our time...everything has to be small. Don't do big joins because you've got to get to the database, get your thing done, get back out. It's synchronous. It's, like, single threaded. We don't like doing big updates. We like doing small batches, everything.

And then behind the data center wall, compute everything because storage is ridiculously expensive. And when I went to the data team, they were, like, yelling at me, like, “Stop that. Stop that. Stop that,” because storage is free, and compute is vitally expensive because we're spinning up cloud servers [inaudible 09:19] down. And it was mind blowing to me. And we approached everything...I was so excited when they did, like, this 30-table join, and it came back in, like, one and a half seconds. And then, on the flip side, we tried to do select count from this table, and it took one and a half seconds. It's like you could do anything in one and a half seconds, but you couldn't do anything in not that amount of time, and I'm like, that's the difference. The get in, get done, get out is low latency. So, it's very, very orthogonal thinking is how you come to it.

And it was kind of exciting to talk with Bill about, like, “Well, how would you refactor?” And he's coming from a world where they would take the whole system down, like, to rename a column, take the whole system down, rename the table in the database, rename the table in the code, bring the whole system back up. And this would be after, like, a week of testing it in a replica to make sure that the name change was going to work.

And I'm like, yeah, we don't do that. We're coming from, like, finance and healthcare, where it's like, your system cannot ever go down. So, if you want to rename a table, you're looking at five separate deploys, you know, add the column; backfill the copy; add code to shoot to both places and to read from, you know, and to read from the new place. And once you're syncing always, then you start reading from the new location only. And each one of those is a separate deploy. And yeah, he was shaking his head. And he's like, yeah, that's fair. So, instead of big versus small, it's like breadth first versus depth first approach to covering the entire graph.

MIKE: But it requires different tools.

EDDY: Going the slow but sure route, I think, is the one that I've always appreciated the most, meaning don't go destructive; get it done in a day, but rather do it in a five-step process to make sure that everything you're doing is correct so that you don't lose any data, right?

DAVE: Mm-hmm.

MIKE: That's interesting. You're absolutely right, Eddy. You have to do that on a big project. But what if it’s your own personal project that hasn't even gone out yet? Do those same rules apply?

EDDY: If something's not actively being written to production, is that what you're saying?

MIKE: Yeah, that's right.

EDDY: Then go as crazy as you want, I think. You don't have any sort of sensitive data, like, actual factual data from customers or clients, right? So, I'd say you're free to play around.

MIKE: Now, Chloe and Jordan, you have been working on a project most of the summer, and that scale is different than what you do at school. What have you learned about working in a larger project versus a smaller one?

CHLOE: I think something that I've noticed is that the way that you design things and those decisions at the beginning can have a large impact on the project. When something's really small, if you're, like, I'm going to structure it, you know, this certain way, it's halfway through, and you're, like, hmm, maybe that's not the best, it's really easy to change. Versus a large project, those decisions are usually pretty concrete by the time you realize you might want to [chuckles] change it.

And so, I think it puts a lot more emphasis on the front end of thinking through the process of what's the overall game plan of this project? What do we want it to look like? Do we expect it to grow? Do we need to look at tools that can grow with that? Or I think there's just a lot more questions you have to ask yourself before starting versus when it's smaller, you can kind of just hit the ground running and then deal with problems as they arise. But you're not always afforded that luxury with a large project.

WILL: Ooh, so waterfall.

CHLOE: Yes.

WILL: The siren song of the waterfall.

MIKE: [laughs]

WILL: [inaudible 12:52] sensation to just be, like, okay, listen. So, listen, what if we just didn't make any mistakes in the beginning and we knew [laughter] everything that we were going to do? Despite being literal students, right, and this being a learning project, we're going to get it. We're going to nail it. We're going to tie everything [laughter] together. And it’s just going to be like, boom, bullseye right in the very beginning.

I think that is a very natural impulse that will be with you for your entire career. Like, you're always going to be, like, oh, what if we just didn't screw it up, you know, essentially [laughter]. It's a temptation. It's a very fundamental, natural human temptation. And I think, you know, you could get better and better and better on it. Because you go on and you have better instincts. You make better, like, just sort of, like, rules of thumb, you know, kind of stuff.

We talk a lot about, like, carpentry and building houses and, like, physically constructing things. If you’ve ever watched somebody, you can...you can watch people who are really, like, just good and, like, good framing carpenters, and they can just throw up a shed. And they just kind of know where everything's supposed to go. And it all just fits together, and it's right. And it makes me really mad because I'm okay with effort. But you’ll get better at it, but you're never going to be good at it [laughs].

And it isn't, like...the solution isn't, like, don't try because absolutely try. But there's also a feeling of, like, you know, let's just go and learn. If you accept refactoring and rework and reevaluating of your prior preconceived notions as a necessary process, when you hit that point, and you’re dead end, and you need to turn around, turn the car around and drive back, you'll identify it, and you'll do it, and when it's time, as opposed to, like, just, like, okay, I'm just going to muscle through. You know what I mean?

CHLOE: Yeah, I think that makes sense. I think it's inevitable that you're going to make decisions that eventually you might want to change. And so, instead of, you know, having this idea of no, we're going to make this decision now and it's going to be perfect, embrace that things are going to change and kind of learn to work through those things. I agree. I think that's a good point.

MIKE: The agile software paradigm in a nutshell.

DAVE: That actually came up in my chat with Bill that his database thinking is old school. The reason I had the chat with him was because we're migrating phone numbers in our database. And we're literally moving from one table to another. And I had a chat with him of, like, we've been working on this for, like, six months, and, you know, what should we have done?

And I realized when I invited him to the meeting yesterday, I said, “I'm going to ask you what we should do. And I realized that the first thing we need to do is jump in a time machine and go back and not forget to have asked you what we should do before we do it. Because it's so much easier when you get the design right first.”

EDDY: It's so much easier to let someone else do the design that they're really good at to do it for you than [inaudible 15:55] [laughs]. Agreed.

MIKE: But it's inevitable. And we're talking about large projects. It's inevitable that on a large software project, the business domain is going to shift. So, even if you made all of the perfect decisions on day one, which you didn't, the ground is going to shift under you when what you were building --

DAVE: And if it doesn’t, the company you're at is going to go under. Sorry, if the software you're building doesn't shift, the company you're on is going to go under because the world is changing every day.

MIKE: Well put. You build really good horse carts, right [laughs]? You may have it perfect. You've still got a problem.

DAVE: Yep. I make the best abacus in town.

MIKE: [laughs] That necessity of change is going to come up on you. Tech debt can hit you even if you didn't know that you were going into debt in the first place, just because of that change in the world.

So, you know, we've talked a lot about this, Chloe. You talked about, yeah, I want to get things right in front. You do, and then you can't, right [chuckles]? You do all that work, and then you find out that you were given an impossible task. As Will said, you still try, and you do the best you can. You still measure twice, cut once [inaudible 17:22] carpentry [laughs]. It still might be wrong a year from now.

WILL: This is maybe advice that, like, maybe is specific to me, but I think it generalizes, like, at least decently. I've always been smart, and I've always been good at my job. And I've always been known for being smart and for being good at my job. And I think a lot of people in the business self-identify as smart and confident. And it leads me down a dangerous and destructive path when I fail to embrace my own ignorance and stupidity [laughs]. Embrace stupidity. Lean into it. Lean into it.

Like, you get used to being...you can get too comfortable being smart, especially if you're really smart. You know, like, we don't work with...nobody in the office...if you walk down the office, right? There's nobody...nobody is dumb. And there are very, very few people, maybe in sales, I don't know, who are average, you know? There's nobody. And so, you know, yeah, embrace stupidity.

MIKE: I'm going to build on that because I think that is a profound concept. Have you read any of the work of Carol Dweck, a researcher in learning?

WILL: The name is familiar, but I'm ignorant of her work.

MIKE: She’s done a lot of growth...I think I’ve got this right [chuckles]. She's done a lot of research on what she calls a growth mindset. And what she's found is that people who think they're dumb and people who think they're smart end up running into the same dead ends because the kids who think they're dumb, well, they don't try. The kids who think they're smart have their self-image invalidated when they make a mistake. And so, they don't try because they think, oh, I'm actually not smart. I'm not good at this. And they don't like that feeling, and so they stop.

My recollection is that this actually tends to affect girls more than boys because a lot of parents are like, “Oh, you're so pretty. You're so smart.” And they say a lot of “You are this,” rather than “You did this,” and that can harm the girls, right? Like, you think, well, no, there's nothing wrong with being told you're good. But it can actually be counterproductive when you say that you are frozen in this good state rather than saying, “Wow, you are a hard worker.” Now that completely changes the way you see the problem, right? Like, “Wow, you made a lot of mistakes today. You are working hard. Look at those scrapes you've got. You are tough.”

That is a fundamentally different way to approach the problem and is much more healthy. And the kids who think that way, that growth mindset, are the ones who tend to succeed. And that matters a lot when you're in school where you have to grow a lot. But then to Will's point, we're all going to get stuck. And if we don't recognize that, yeah, we're going to have to be dumb sometimes; we're going to have to be uncomfortable and grow, then we've run into exactly that problem.

EDDY: Yeah, the best advice I was given when I first started is, don't be afraid to sound dumb, right, kind of to add to that, right? Everyone has come from that point, right? And the sooner you accept the fact that someone who's been an engineer for 10 years is also dealing with the same thing as you are, things become much easier.

DAVE: There's an easy thing that I...it was probably me who told you that because I'm --

EDDY: Probably.

DAVE: I’m still happy, Eddy. About six months ago, we were standing around talking, and I asked a pretty obvious question. And you looked at me and said, “You are fearless about asking stupid questions.” And then three people in the group said, “I don't know the answer to that either.” And I'm like, not so stupid after all. And Eddy wasn't asserting that. I'm, like, no, that's actually a pretty smart, stupid question.

And so, the hard work versus...sorry, getting it right versus dealing with it, in my opinion, that's the same shape as being told you're perfect and not wanting to risk damaging that, which makes you nonfunctional and less productive and less successful versus being told if you're here, you need to get here. Just take one step up, and then if we come back to you in five years, you're going to be way ahead of the other people because every day you take one step up where the people who were smart to begin with haven't been.

And this is why I wanted to say this is getting the design right the first time is getting the design perfect the first time. When I put the phone number thing in, it's currently running in the most awful way possible. And I'm proud of it, so I don't feel like this is me shaming the company. We currently have data living in two places, and it's ugly, and it's messy. And both Rails models have triggers in them that when I get updated, I'm going to go check the other data location and if it doesn't match me, I'm going to fix it.

Well, first thing I'm going to tell it, “Don't update me back because you have the same code to update me when you save,” right? So, this data is constantly fixing. And basically, anytime you touch one of these things in the database, it settles with the other one. So, literally to backfill the data, you literally can just say, “Load an object, touch it, save it. Load an object, touch it, save it, load an...” And that's literally the backfill because the data can repair itself in motion. And all you have to do to backfill the data is just touch all the data, and that's great because, now going forward, any new data going in the system comes out correctly.

I went on leave for four months. I came back. That code is still there. We are running on that little donut spare tire for this data. And there's a part of me that's going, this design is terrible, and it's inefficient. There were many extra cycles...da da da. And then I'm going, we are one step closer to having the database right, and the website works. We're selling leases. We are giving people the things that they need. We are taking care of business. We have cash flow. And I love that. And, longitudinally, we are headed towards the right data schema, database schema, and I love that.

I would much rather be good at correcting an error than getting it right the first time because getting it right the first time doesn't scale.

EDDY: Well, I'd be surprised if any of you have been part of a large project where you got it right the first time. I mean, I'm happy to be proven wrong.

CHLOE: I certainly have not [laughter].

EDDY: Yeah, no. I'd be happy to be proven wrong. I think that's a very fair assumption.

DAVE: There's a rule that we found in the ‘90s, which is that a waterfall software process works very successfully if you have built it three times already. So, if you're making version 4 of the thing and it's the same thing, you're not reinventing it for a whole new world, right? It's like nobody has built 3 AI embedded products. That's not what I'm talking about.

I worked on a company that was making graphics cards, and we were making the 3K version, which was after the 2K, after the 1K. And we've reused all the same processes because the way you manufacture silicon hadn't changed. And so, waterfall worked really, really good.

Do you know where we ran into trouble? That 10% of extra stuff that was all new. And we ran into all kinds of trouble, some of which broke the silicon. And I stayed up late one night swapping red and green in the software because somebody had made a change upstream that was now using the wrong channels for the colors in the silicon. Couldn't be fixed in the silicon without turning the chip in the fab, which was going to be, like, a $10 million impact. So, they're like, “No, fix it in software. It will be all right.” We got it wrong.

MIKE: To pull this back to where Chloe originally started, well, you have to think about the decisions beforehand. Well, you do. But you need to make a good decision, not a perfect one. And those are different [chuckles]. You do, in a large project, need to make a good decision. But it's infeasible to make a perfect one, so you shouldn't try because you can't. And it is a fool's errand, and you will end up spending all of the resources allocated to your project trying to make it happen in planning up front, and then you'll still be wrong.

EDDY: So, Mike, are you saying that a good decision implies that you can be agile about your design?

MIKE: I am. Because if you head out in a good direction knowing that some parts of it are going to be wrong, then you step in knowing, hey, I'm going to have to change. I'm going to have to take feedback and course correct over and over again. But that means that at any point in your project, from inception to finish, you're continuously pivoting. And you never reach that perfect ivory tower; instead, you make money. You actually bring in value [laughs] to your employer.

DAVE: Have I told you my mantra on...you get told, if you don't have time to do it right, when will you have time to do it over? That is waterfall thinking. Agile thinking is, if you don't secure the cash flow right now, when are you going to have a paycheck to do it right?

WILL: So, the question I've got, right, like, I love the conversation, right? And I feel like, okay, you know, smart, pat on my head. I made a good point. Everybody likes it. So, you know, I'm being applauded in my mind. My therapist would be very proud of me. But, like, I think, okay, let's take it a step further, right? We said, like, don't do this, right? Like, don't plan too much, right?

And balance in all things is, like, the essence of engineering, right? So, we say, don't plan too much, right? And I think everybody [inaudible 27:29] on that, right? But, like, don't not plan, right? And so, like, how do we balance the scales between, like, okay, you know what I mean? Like, I'm not just, like, shooting off half-cocked, but I'm also, like, not analyzing this thing and creating a 100-page PowerPoint before I even break ground, right? How do we balance that?

DAVE: I have a quote from Sandi Metz. Agile means deferring a decision to the point of maximum information and no further. That's the key, right? It's deferred as late as possible because information always goes up over time. But she does say --

WILL: I don't like that at all [laughs]. I hate that. Maximum information?

DAVE: Everyone does --

WILL: Maximum information. Like, that's, like, I don't...I am deeply unsatisfied by that, but I bet you have more to say.

DAVE: So, information increases over time, but at a certain point, you've lost the opportunity to make the decision because things have just been decided for you over time. And so, that's the ideal perfect time because, like, the schema is turning, turning, turning, turning. No, we have to shift, but now you have as much information on what the correct schema to go with is.

You are absolutely right. She's not saying don't make a decision. You do have to make decisions. And a lot of decisions if they're not impactful, just make them, right? It's pushing not all decisions are the same.

MIKE: Procrastinate as long as you can get away with.

DAVE: Procrastinate when it's appropriate, which is probably more than you think.

WILL: I find this deeply unsatisfying, and I'll tell you why. I think the fundamental thesis is right, but I don't like it as a heuristic because I think what I believe will eventually...if you're a perfect person and you're diligently following, like, that thing, what you'll run into, I believe, is the uncertainty around the information that you can get where it's just like, I don't know. How's it going to work? I don't know. What's going to happen here? I don't know. I don’t know.

DAVE: Not all information is equivalent, yeah.

WILL: You know what I mean? Because you don't have enough data. The uncertainty, I guess, the signal-to-noise ratio gets to the point where, like, you're making decisions, you know what I mean? And there's just no way of...there's no way of possibly knowing, not because, like, the information is not out there, but because you don't have it, right? You run afoul of...you just have to have a lot of wisdom and professional judgment.

And, like, you just need to have been around the block a few times. And you need to have, like, very much that beginner's mindset to be able to have, like, both the wisdom to say, like, “I don't know. How would I possibly know that?” And the judgment and the courage and the humility to say, like, “I don't know.”

And, like, to stand up to say, like, “I have no idea what the answer to that question is, Mr. Vice President. I would like several million dollars of your budget so that we could just, like, go off, you know what I mean, and see if this project lands.” And there might be some dumbass in the seat next to you without the wisdom or humility who's just like, “No, I know, and we're going to do it my way.” I think it's blind human nature and the sort of, like, emotional, psychological, social forces that these decisions are going to have to be made in. Does that make sense?

DAVe: I think so.

WILL: You know what I mean? Like, yeah, you're right.

DAVE: I think so.

WILL: We [inaudible 31:16] work.

DAVE: Yeah, I think not all decisions are the same shape. Like, I worked at a company where the CEO wanted us to look really good to the investors, so he splashed for a SQL server. And it was a nightmare. It wasn't easy to match. There was a lot of impedance. And because we went with SQL server, that forced a lot of decisions that were free decisions until he made that one. And five years later, we moved everything to Postgres. And we moved things to Postgres here at Acima. I'm not talking about Acima, just to be clear.

But that's the kind of thing. Like, if you make a decision I'm going to do it this way, and it's going to lock me into 73 other side effects that I haven't even considered, that maybe was a decision that could have been deferred and studied a little bit better. I'm not saying push everything out. But I would say, by default, I tend to push decisions back unless there is, like, no, this has to be made, or I can see that the decision has very little cascading downstream effects, if that makes sense.

You could probably use some sort of heuristics or some sort of rule of thumb around, like, percentages. Well, this project is, well, this project's got a $500 million budget. So, you have something outrageous, you know, some big government contract. You probably want to spend more than five minutes making that decision. If something’s got a $2000 budget, you probably don’t want to spend more than a few minutes making that decision.

DAVE: Absolutely. I wrote a script to keep track of what branch I'm on in Git. And the very first version of it, I used YAML store, which is literally just a YAML file as my database, and that worked great for about six months. And I eventually had to upgrade to SQLite. I didn't want to go all the way to put a database server on my thing. So, I was like, yeah, let’s just install SQLite. I can do that. And that's enough SQL to do the things that I'm keeping track of.

But what I needed out of that decision was for the decision to be made quickly and to not slow the little project down. And that could have, like, if I was running a production business and I had made that decision, we might run into all kinds of scaling problems. Because pretty soon it's, like, well, we don't want to change databases, so we're going to write all this stuff to deal with the fact that we got this wrong. And welcome to the rest of your career. It's going to be solving your solutions is so much more painful than solving your problems.

MIKE: Going back to what I was saying about the different scales, you can say it in percent. What is it, 5%, 10% of your time should be spent up front making some decisions. And I don't know exactly what that number is. It shouldn't be 50% in general [crosstalk 33:56].

DAVE: I would be tempted to think the number changes, but yeah.

WILL: I mean, I would say, like, rather than, like, sort of, like, a percentage, right? Like, I would say I want a story for, like, a thing. Like, this is a thing, and this is going to happen. And somebody is going to open thing up, and they're going to go on a journey, right? They're going to go on a journey. And, like, I'm going to, like, I want to make a web app. Like, okay, well, what's the first thing I'm going to do? I'm going to log in, right? And then I'm going to see, like, a to-do list, right? Something dumb. I'm going to make a to-do list. I'm going to make, like, a photo, whatever you're going to do.

Like, make a story for, like, this is the person, and they're going to do the thing. And then what are they going to see, and what are they going to do, right? Like, that's...like, it's a school project, and I'm going to write, like, a database. And I'm going to enter data, and I'm going to have a row. And it's going to be a thing. Just have a story of, like, just what this is in plain, simple language.

And then, the story gets more or less detailed when you're in a big organization where it's, like, okay, well, I'm going to go in here on the website. And it’s going to be this team, and it's going to be this page. And this is what we're going to do. And then I'm going to talk to this service. I'm going to talk to this thing. And this isn't...it sounds like, oh, this is a waterfall, but it's a simple algorithm, like, the planning that you're going to need to do in a big organization versus a little organization.

Write out the steps. Don't have a lick of code. Like, I'm talking about plain English and napkin drawings. Like, you can do all this stuff on a whiteboard. Write it out. No code. Nothing. You’re just like, I'm going to talk to the Prometheus service to get a user log in because you got to do that. And you know where you have to go, or you should know before you're building it, right?

DAVE: We might be in violent agreement, Will. When you say push the design out and do it on a napkin, that's what I mean. You have just avoided choosing a tech stack and a server host. And that's the decision I'm talking about pushing out.

WILL: Yeah. I mean, but just a story, like a story, you know what I mean, like, yeah, nothing. No keyboards. Write it out with your fingers, those little [inaudible 36:16] popsicles, and, like, write it out. Go ahead.

MIKE: Are you suggesting, Will, that if you can tell the story, then you're at the point where you can make a suitable decision?

WILL: I believe, like, you'll write out the story, and then, like, somebody in your team will be, like, well, what about that, right? And you'll flesh out the story. Well, what about that? And you'll flesh out the story. And eventually, and this is, like, woo-woo and vague, and, like, okay, it is a craft that you learn, you know what I mean? You’ll get to a fixed point. Like, you’ll get to a fixed point where, like, you’re going to know what you're doing, you know what I mean?

And then, I think, some of it, unfortunately, I hate to like [inaudible 36:58] vibes out there, vibes. But you're going to start feeling some vibes, you know. And you will also blow it because it's just a story. It's a napkin drawing. And you'll be, like, oh, well, what do we think of that? Like, yeah, it's a story, you know.

MIKE: Emotions are sometimes looked down on, but they're there for a reason. You have those vibes because it's telling you something. So, I don't think we should be ignoring them. We should be using them to their maximum potential. And if you reach the point, like, hey, this feels good; this story resonates, that's probably telling you something.

DAVE: What you're saying, Will, matches very strongly with my view of, like, top down versus bottom up. When you build bottom up, you start with, like, the write me an adding machine. Now write the multiplier. And at every step, you can prove that what you have built works. When you start top down, you might get to the bottom and find out you can't actually do it. You made an assumption about the latency of some system or other. But you do know from the very beginning that you're building the right thing.

And we've all had that thing where you build top down and you find out that, oh no, the database literally doesn't have enough latency to support this entire approach. We've also built bottom up and then found out that we climbed all the way up the ladder and found out the ladder was on the wrong wall.

MIKE: We’ve really run with Chloe's idea [laughs] about planning ahead of time and how that really is dramatically affected by scale. I'm curious, Jordan, what thoughts you have about the difference between large and small projects?

JORDAN: Yeah. So, I would say that the biggest thing I face, well, not face, but experience, and maybe this is just, like, the difference between working, like, by myself versus working in a business, was the need to get approval on many ideas and, like, changes to move in a certain direction, I think. So, like, similar to, like, the architecture and design, like, if I'm working on my own small project, I'm allowed to do, like, any change I want, like, willy-nilly compared to, like, here. Like, if I want to do something, I need to, like, propose it, like, architect out, and then, like, talk to people and get approvals there.

And so, I guess that's, like, I can have a radical idea, and then I present it to people, and then it's more, like, normalized into something that might work better for everyone, I guess. So, I guess that is the biggest thing I've experienced.

MIKE: You know, as I was preparing to host today [chuckles], I wrote down some notes. And one thing that occurred to me is that on a large project, you deal with coordination, which is a challenging human endeavor. And, you know, solo project, you're not coordinating with anybody but yourself. But when you get to a certain scale, you can't do it yourself. And now you have to add in another layer of complexity, which is how do I make this work with other people [chuckles]? Maybe I need version control so I can share this with somebody else, so we don't step on each other's toes. Maybe we need an API contract that we agree on so that [chuckles] when I send a message, they can actually receive it.

And working together is hard because you have to get shared understanding, and that takes a lot of work. It's the only way we can build big things. And the more big and complex, the more work that requires because the things have to be precise, right? And you have to have an interface that actually aligns.

I think that sometimes we think, well, there's all this bureaucracy. And, I mean, bureaucracy can be heavy handed. It can also be a representation of the natural work involved in coordinating across groups. You're going to have to have hierarchies of information that travel upward and then are coordinated between each other, which sounds a whole lot like bureaucracy. And it exists for a reason. Now, it can be misused like any tool, but it's going to have to exist because the alternative is anarchy, and then you can't coordinate anything at all. And that's not our goal.

And Will made the point about balance being so fundamental. I agree on that in many ways. I think finding a balance is tremendously important. But my key idea here is that you're going to have to do some coordination. And the bigger the project, the more coordination you're going to have to do.

EDDY: Well, you have more hands in the pot, right? So, you've got to satisfy more people.

MIKE: And that's not free. So, yeah, big project, you're going to have to get signed off. You're going to have to get people to agree with you. You might have to find a consensus or at least get somebody to make a decision [laughs].

DAVE: Jean-Paul Sartre, "Hell is other people," or in our case, hell is other people's code.

MIKE: On the flip side, I was reading somewhere sometime in the last month. I would have to go find the code. I don't have the source on it. But it was a researcher on human happiness. And if I remember the quote correctly, it said, "Happiness is love: Full Stop." Which is to say that hell is other people and so is heaven.

DAVE: Yeah. Yep. I heard a good one on this today, or last week, which is happiness is your circumstances minus expectations. I thought it was kind of interesting. Being present where you are instead of looking over the shoulder of the present moment to see the next problem that's coming.

MIKE: So, Jordan, you talked about the coordination that is involved in working on a larger project. What does a big project look like structurally versus a small one?

EDDY: I'm not exactly sure I understand your question. I mean, if you were designing a new service or whatever, I'm sure you have stipulations on things that you have to meet. In order for that service to work, you have certain metrics. You have certain clients that you work with. You have certain design that needs to fit. I'm not entirely sure I understand how broader than that you're asking.

MIKE: Well, let me give you an example. Let me give you my example. So, at Acima, we've done a lot with Ruby on Rails. It's been a popular web framework for a couple of decades now. And it gives you its particular take on model-view-controller architecture. Here's where your database interface layer goes. Here's this models directory. Here's where your interface goes to the outside world, you know, your controllers. Here's your views. Here's your templates that you're going to use to display things to your end user.

That works really good for a small project. You build a blog. You can throw everything in that structure, and it just works. If you have a large multi-year project, it doesn't answer hardly any of the questions because most of your business logic doesn't quite live there. And it might be needed to call from background jobs, and I could go on and on.

And so, you're going to build a massive set of business logic, some sort of structure to represent your business logic that is behind the scenes and not in that model-view-controller. Now, you might think of it as a logical extension of one of those. But if you try to cram that all into one of those out-of-the-box layers [chuckles]...I'm seeing looks of horror [chuckles] from people on the call. It ain't going to work. The scale deeply matters to the architecture, to what your code looks like.

MIKE: I heard this talked about by Uncle Bob. What's his last name?

DAVE: Bob Martin.

MIKE: Martin. He said that if you look at blueprints, you can generally tell what kind of building it is.

DAVE: Oh, I like that.

MIKE: But sometimes when we start building with a framework, we say, “Oh yeah, it should look like the framework.” No [chuckles], he says, “That's wrong. In a large project, it should look like what your business domain is.” So, you can look and say, “Oh, okay. I see what you all do. You know, here's the kind of the center.” It's centered around...you probably have a directory representing your primary business domain.

If you're doing something in finance, you're probably going to have something financial, right, at the center of your business logic. And if it's not structured like that, if instead it says, “Oh, here's your multi-controller,” then you're going to have an awkward time because it's built around your tool rather than built around your business.

DAVE: I was on a team once where my assessment after six months on the project was I stepped back, and I said, “I don't know about anything we're doing in the business to make money, but we have written an amazing program to support a database.” We were so bottom up. We were just, like, get the database, get the database. Everything had a table; everything had a trigger everything...da da da.

And a year and a half later, very little stuff to support the actual business. It was a Greenfield project, like, an investor's baby, where it's like, go do this, and do this for me. And so, we had a lot of funding. And after a year and a half, we started having some difficult conversations about what was coming out of this money. And I'm like, you got a nice program about your database.

MIKE: So, you talk about the structure needs to be different. How can we use some of the things we learn from small projects? Yeah, go out and build a small prototype in school. So, Jordan, Chloe, in school, you build a lot of small projects. You can almost think of those as prototypes, right? They are very purpose-built, specialized things to prove a point that you'll never actually use in the real world [chuckles]. But you do it to learn about something, and we do that in the business world as well, right? We want to learn how something works. Would this idea work? If we try this out, is it possible? So, we build these prototypes.

That prototype is probably going to have all kinds of bad decisions. You're going to have to rethink to build the real thing. But how can we apply what we learn from the small projects to the big ones? Is there any carryover? Do you have to learn an entirely new set of skills? Going back to what I was talking about in the beginning about navigating in your neighborhood, have you learned anything there that you can actually apply to the larger problem?

EDDY: I've learned to play nice with other people. I think it's really key. And be very receptive to feedback that you otherwise wouldn't.

MIKE: So, you're saying a large project, the larger the project is, the more the human element matters?

EDDY: 100%. You can't just make changes willy-nilly, assuming that it's a benign change. You've got to have adoption across the board. So, it takes a lot more coordination between everyone.

DAVE: That poked my brain. We were having a chat in the engineering...actually, Mike, you started it. We were talking about how does AI help you? And somebody pasted a tweet from DHH that said, "When I use AI as my pair programming partner, it's really skilled, really good. When I let it drive, I stop thinking. I retain nothing, and it's useless."

And, in my mind, that's very close to what you just said about do we build a prototype and then leverage it? Or did the AI build it for us, and we've got nothing? We used to be raised on "build one to throw away," right? Because you're going to learn so many lessons from building the first one. And the important thing is to throw it away when you're done because you've made so many mistakes. Don't fix them. Just build the new one without the mistakes, right? And, like, in AI, if you let it just do it for you, that's not a prototype. You can't leverage that much anyway.

MIKE: That's interesting. You didn't learn from it because you didn't build it. Cheating on your homework.

DAVE: Yeah. And there's times when that's valuable, though. I needed a thing that would log in with Google, right? Just OAuth. Simple. I’ve built three of these in five years. I've been on teams that have built 50 of them over the years. And I’m like, can't remember how to do it. AI, go build this login for me. I don't need you to learn me how to make an OAuth login. It's a small problem.

EDDY: For the listeners, I don't know if we have anyone out of state, but here we're in Utah, right? Salt Lake City area, whatever. If I have to drive to, I don't know, like 30 minutes away to another city, typically, I will drive there. I know exactly how to get there. I know all the directions. I know what the subroutes are in the event that the road is closed or whatever. At that point, it's just becomes routine, right? Like, I’m on autopilot, and I have to just do it for the sake of just doing it because it's expected of me to do it.

It gets to a point where I'm just like, dude, if I had an autopilot on my car, like a Tesla or whatever, I already know how to do it. I don't have to focus to do this because I already know. So, I'm just going to tell it to do it for me, and eventually, we'll get to the same spot, and I know that I got there.

So, like, for mundane tasks, you know, where you know 100% you can catch issues or bugs early or whatever, I would say that's fine. But when you’re having to go to a different place that you've never been to before, you're a lot more attentive, and you’re running the risk, you know, of autopilot kind of telling you where to go without paying attention. So...

DAVE: You just synthesized the thing that we've been talking about this entire time. Prioritizing the future value or assessing the future value of the knowledge and the skill. Driving somewhere, you can't manage a 70 mile an hour night knife fight in rush hour traffic when you are still learning clutch, brake, shift, right? You've got to get the basic skills in.

I've got a note on my monitor that says, “Insight is not cure. Get it under your fingers.” Now, this is a guitar teacher that told this to me. He's like, you can't be thinking about chord, voicing, mood changes if you don't know where your fingers are, and if you don't know where the notes that you need to target are.

And peeling that back, when I told the AI, “Go build me an OAuth login,” I put no future value in me learning that, but I have in the past. 20 years ago, I sat and sharpened the whetstone in software of, like, I'm just going to build this thing. I'm just going to do just this pattern. I'm going to do just this thing. You have to get those skills under your fingers then you can start mentally composing the larger things built out of that fluently, fast enough that you can actually start to create (I'm muddling my metaphors horribly.) but so that you can create music in your software in how you develop it.

And what you just said about driving, to me, it’s the same thing. You start composing those things. And it's all based on, the decision I'm making right now, what is the future impact? What is the cost of it? What is the value of learning this skill? Prioritize later, I think, is what I've heard that said. Not put prioritization off until later, but right now, prioritize the things that will create value later. And when enough later has gone by, you're going to be up here instead of right where you started.

MIKE: So, we talked about building fundamentals, because they will still be useful in the larger structure, but they'll be composed. It'll be made up of many of them. I’m hearing that from various voices here. Suggests that learning your skills on small projects and doing that schoolwork without cheating actually gives you something that will provide value as you work on the larger things because it will give you those components you'll know how to put together. And you may not know how to put them together yet, but if you don't have the pieces, you can't even start.

Well, I think that brings us to a pretty good place. We talked a lot [laughs], a lot, a lot about making decisions and about not getting mired in too much decision-making, but trying to find that right spot where you make a good decision and continue to make decisions, continue to pivot and get feedback rather than getting stuck.

We’ve talked about the human aspect, about the absolute critical human aspect on large projects because that's where maybe most of the work is. Well, not maybe, that's where most of the work is, is in coordinating across the other people so that you can build something bigger together. And we've also talked about building up from small to large. That's pretty good for an hour, if you ask me.

Thank you, and until next time on the Acima Development Podcast.

This episode of the Acima Development Podcast focuses on the complexities of evaluating and selecting vendors, whether for software, contract labor, or specialized services. Host Mike opens with a cautionary tale about a vendor that initially impressed with a knowledgeable representative but turned out to have major security flaws, poor integration practices, and overall incompetence once the contract began. The group discusses how this mirrors a broader challenge—companies often get only a brief, curated glimpse of a vendor before committing, with little opportunity to see the full team’s capabilities. This leads to a conversation about the importance of scale in vendor choice: a large enterprise might thrive with a feature-rich, expensive solution, while a startup could be better served by a smaller, more agile provider. Needs change over time, and the vendor that fits now may not be the one you need later.

The conversation then shifts to the vendor-client relationship from multiple perspectives—Mike, Dave, Will, Eddy, and Kyle all share stories from both sides of the table. Will highlights the unstable economics of contract shops and the difficulty in sustaining top talent, while Eddy points out the importance of assessing a vendor’s responsiveness to feature requests. Kyle and Mike discuss how larger organizations often limit vendor options through partnerships, which can lead to compromises in quality or fit. The group also addresses the consolidation push in big companies, where leadership may favor a single vendor for cost control, even if that means losing specialized capabilities. They emphasize balancing department needs, cost implications, and redundancy to avoid single points of failure.

The episode wraps with a practical checklist of red and green flags. Red flags include security issues, legal troubles, impossible promises, conflicts of interest, overreliance on one “all-star” representative, and immature products. Green flags include a vendor that’s “boring” (reliable and consistent), mature, used successfully in your industry, and willing to connect you with your actual account rep. The hosts stress building vendor-agnostic integrations to allow easier pivots in the future, negotiating service levels, and reevaluating relationships regularly. Final advice: know your needs, verify vendor claims, plan for change, and avoid long-term dependencies on critical functions that should remain in-house.

Transcript:

MIKE: Hello, and welcome to the Acima Development Podcast. I'm Mike, and I'm hosting again today. With me today, I have Kyle. I've got Dave.

DAVE: Hello.

MIKE: We've got Will Archer, and we've got Eddy. And we are going to be talking about something that comes up over and over again in the industry. And I'll just jump right into it [chuckles]. We're going to be talking about how to evaluate a vendor.

We talked about build versus buy, I don’t know, a few weeks ago, maybe a couple of months ago. Today, we're going to go specifically into strategies when you do decide to buy. And this also applies even if you're going to go with, like, an open-source project. How do you evaluate the vendor? You know, how do you choose when there's a variety of options?

And story time. Years ago...I think I can be a little open about this because the company isn't even in business anymore [laughs]. I worked for a company that did content management for publishers, mostly newspapers-newspapers and magazines, small newspapers. There's a lot of them out there actually. And we did a lot of their websites. And we were evaluating a partner, like, a business website builder. That's a common thing out there. There's lots of ways you can get your content out on the internet [chuckles]. And there are some niche products that allow you to do it in a specific way. We were looking for a partner to do it in this niche way.

And I'll get a little bit into the business model because it actually is interesting. With the newspaper industry, sometimes people think it was killed by other media, and that's a little bit true, but not really. Newspaper industry was killed by Craigslist because they made their money off of classified ads. If you're old enough, you remember that when you used to want anything, you'd go get a newspaper.

When you needed something like an apartment to go rent, for example, you'd go get the newspaper because everybody in the community knew it was there. All the business in the community knew it was there, and so they published there. And we were trying to come up with a way to kind of preserve that model where businesses could have a permanent presence within the newspaper website. And we were working with a partner to help with that.

There was a friend of somebody in the executive suite who had a company to do something that was loosely associated with what we wanted to do, who said, “Hey, we'll just use my buddy's company.” And they sent a tech rep over to get some evaluation. They did most of the stuff without evaluating with the engineering team [chuckles]. They sent one person over to the engineering team.

And I talked to this guy. He was their...well, I'll get back to that. But he was great. He talked very clearly. He was able to talk about their software. I talked to him about what you do with security. He answered it very well. He was able to talk about their stack. He knew what he was talking about. I didn't see any real red flags. And really, I was just supposed to be putting a rubber stamp at the end because the business deal was basically already made.

Well, when we went to integrate with these folks, we never saw that person again [chuckles]. Turns out he was their best person. And he was trying to keep the company running because nobody else really knew what they were doing. Very quickly, we saw glaring security issues. We had to send a password to them, which they could provide to us in plain text. So, they kept all of their customer passwords in plain text. If you don't know, that is, like, the quintessential security faux pas. You never save a password in plain text [chuckles]. It's just the wrong thing to do.

DAVE: That's not in OWASP anymore because we all know not to do it. That's the only reason...it would be top of OWASP if we were all still doing it.

MIKE: Exactly. The partner was a disaster. The security issues were just the beginning of it. They were hard to integrate with. They were a difficult partner. And they're not the only company I've worked with that was like that. We brought in a vendor, and then it was a disaster. And none of us want to have that.

I just read, earlier this week, there's a new website that allows you to rate your date. It's popular in urban areas where there's guys who will go around and be bad dates, mistreat their dates a lot. It's open only to women, apparently. It allows them to rank the guys and say, “No, this guy's a creep.” And they're happy about it, right? You can identify the creep.

There's lots of problems with that business model, by the way. There's some challenges. There's several companies that started it and then failed because it can be abused. But it'd be great if we could have something like that, right? You get some crowd-sourced, semi-objective information to say, “Yeah, this is a bad vendor.” But we don't. You never get that. What you get is more like what I got, like, here's the best person from this company. Try to evaluate whether they're a good vendor or not. You've got 10 minutes. Go. And it's a challenging problem. It's a challenging problem.

So, what do we do? How do we evaluate a vendor and determine they're the ones you want? How do you stack them against each other and choose which one to go with? And I've got some thoughts in here. I've got lengthy notes that I can start talking, but I don't want to do all the talking. I shared one horror story. You all probably have some as well. I'll kind of open the floor here. Do you all have any horror stories? And maybe you don't have horror stories. Maybe you've got positive stories. What are your horror stories, and what did you learn from it?

DAVE: I have a quick scoping question. When we say buying from a vendor, this can be anything from buying a $100,000 site license for some software, to purchasing, to, like we were talking in the pre-call, to contract houses where we literally...where this is the opposite of buying, that we're literally renting workers. But it's the same idea, right? We’re going to go [crosstalk 06:34] for money instead of for a high-risk [inaudible 06:37]

MIKE: Exactly. It's the same principle because our expertise only goes so far. There's only so much that you can build. And [chuckles] you're going to have to buy something, almost certainly, that's a rare company and probably doesn't even exist that can manage everything themselves. So, at some point, you're going to have to work with a vendor.

DAVE: So, I wasn't going to tell this story, but you just jiggled it in the back of my brain. And Eddy’s heard this story this week. I built a project 20 years ago that I called Data Monkey. It was all written in Perl, and you gave it a DDL, basically a schema, a DSL of a database schema, and it would go build your database for you. And somebody told me, “Have you seen Rails?” And I'm like, “No. What's that?” And they’re like, “Oh, you need to see this.”

And, Data Monkey, I had spent over a year and a half building this. And I was running a content management system for a bunch of online webcomics with it. It made us very happy. But it was clunky, and I was the only maintainer, and developing and adding features was hard. And I couldn't do migrations. I could not understand how to do deltas on a migration. So, you had to throw away the entire schema and then run Data Monkey again and build it all over again. It was a 1.0, right? It was a very early thing. And it only ran on MySQL because that was the only database that I used as a single person.

Somebody showed me the DHH's How to Build a Blog in 15 Minutes Using Ruby on Rails, the viral video that kicked Rails off. And I killed the Data Monkey repository, like, 30 minutes after watching that video. I'm just like, there's an entire team building out all the other database adapters. So, you get Postgres; you get SQLite; you get SQL Server. You get all that stuff. And they figured out, like, delta, like the harder problems that were still...like, I scratched the worst of my itches, but it just got harder and harder to reach, and deltas were in there.

Being able to pick it up and buy it versus building it yourself there is a huge, huge ego component. And this story has stuck with me because only after the fact did I realize, I really should have had some ego in this. This was a personal passion project that I was making money off of. I had every reason to want to defend Data Monkey, and I didn't because what I wanted was all of the features. And I wanted a full team supporting it and building it up. It helped that the price was free. I mean, total cost of ownership my time involved, like, the next 20 years of my career has been sucked up by Rails, so I can't exactly say it was free.

But I would say that, if you are in pain, and you are scoped down, and you're trying to fan out into something, yeah, don't build, buy.

MIKE: Well, did anybody have any, you know, I threw out my story of the awful vendor [chuckles]. Any of you worked with particularly awful vendors that you'd like to share that it's not, you know, maybe go back a few years [laughs]?

WILL: I myself have been a vendor, right? I myself have been a vendor, but I was, like, a solo freelancer running my own shop vendor. And consider the reliability of the narrator. But I had been successful at it, and people had been really happy about it. And people had been like, you know, generally pleased and happy to refer me, and hire me again, you know, when they had the budget, and they had work that needs to be done.

And I've hired vendors, and I've had a very difficult time with it. All the people who have hired vendors and then had a difficult time with it I have been, like, Mr. fix-it, you know, have pinch-hitting for vendors that didn't work out.

I was trying to zoom out to, like, and here I'm talking about, like, hiring development shops. And one of the issues that I think I always think about, like, sort of like the macro, right, like, what are the incentives that drive this thing? Like, get it in a messed-up state.

And what I'd say, economics of it are weird, or they may be unstable, right? Because, like, you know, I'll be straight up, right? Like, if you are a contractor, right, you're parachuting into some other big enterprise, some big project, some big codebase, some crazy thing. And you're going to drop in like SEAL Team Six, and you're going to get stuff done. That's hard, okay? It's just not everybody on your team that's going to want to do that, going to be able to do that. That is tough.

And, you know, to be totally honest with you, most people don't want to do it. Most people are just like, yo, I'm going to get in this codebase. I'm going to get familiar with everything. Like, I'm going to know who the people are, where the people are, what levers to pull, how to get things deployed, how to get things debugged, where the skeletons are buried, so that I can be more effective and be more productive as a developer. And that's a lot less strenuous a way to work.

And so, if you want to get some people like that, right, you want those people in, why are they going to do it, right? And so, it could be like, you could pay them a bunch of money, right? Hey, cash talks, flexibility of lifestyle. You know, like, I myself am a freelance gun for hire right now because, like, I'm at the phase of life where, you know, working remotely is pretty attractive to me. And even in our RTO world, there's still a lane for you if you can deliver the goods, and you're willing to work contract, you know? Like, there's a little bit of a loophole there, right?

So, where these big sort of, like, software developers as a service come in is there's a lot of incentive, I think, for the business to keep as much of that money as they can and then work the people as hard as they can. And it's very difficult to find a place that can sustainably, you know, hire and retain the kind of developers that they would need to be able to deliver on their promises and achieve them. Because, like, the business wants to make money. So, they make money when the cut is bad, and they don't make money when the cut is good.

And so, you know, you have a lot of people coming in doing a very hard job for, like, not a lot of money, and, like, that's not sustainable. Like, people who are good at their job will have other opportunities. So, the business just sort of, like, it just kind of degrades. And I find, like, there's an inevitable life cycle to these places is then, as they get bigger and older, they just all kind of fall apart in the same way.

MIKE: That's an insightful take about the challenges of a contract shop.

EDDY: I was going to say, I mean, I got to imagine, I mean, I don't have much experience with integrating with vendors or whatever. But I got to imagine that part of the decision factor when you're deciding to integrate or buy a product is how easy is it to request a change or a feature change, right?

Because maybe the product that you're buying doesn't fully encompass what your business actually needs, right? So, if you're trying to evaluate the pros and cons, like, you know, one can only assume, you know, is that something that you...is it a request that you, as a vendor, can...or as someone who has purchased it or licensed it, like, can they actively make that change themselves, or can they request a change? You know, what's the turnover on that, right?

MIKE: Well, you actually...you're touching on something, Eddy, that actually ties into what Will was saying indirectly, what I was thinking about before we got together. And you talked about them. Will they make the change? Do they have the features you need? There is a scale issue, which means the same vendor may not be the right choice for everybody, and, in fact, almost certainly aren't.

If you are a large, big, mature company, your needs are different than a startup. You might need all the features. You might want the enterprise plan and have the deep pockets to pay for it. If you're a startup and you're going with the enterprise plan, you're probably doing it wrong because [laughs] you don't have that money, and they don't care about you. You know [laughs], that big company, they're just trying to get money from other big companies. That's their business model.

And you might want to go with a small company that doesn't have very many features because, you know what, you don't need them. If they can give you what you need and it's cheap, that probably meets your needs. And your job is to stay afloat. Your job is not to spend all your money on a product that might work for you 20 years from now. And those needs are very different.

And, you know, Will talked about the contract shop, you know, growing and kind of degrading. Well, that can happen at the full enterprise level. But it also happens even within a single organization. You might get a team. They've got a new team, and they're great. Maybe even within your own company, you've got a new team. They're great. They grow to the point where they're not very manageable anymore. Maybe as you grow the pay is becoming more normalized, and some of the best people start leaving. You might have that exact same kind of problem happening within your company, and part of it has to do with that scale. And there's some tricky timing there.

The vendor you need today may not be the vendor you need 10 years from now, and you have to make those kinds of evaluations, both when you're hiring a team and when you're buying some software.

Think about a team, big companies, the real big companies, Microsoft, Google, they hire contract shops. They'll hire engineers by the thousand, and they have a longstanding relationship with them where they can manage these long-term things. They've figured out the requirements where they can make that work. But if you're mid-size, you're not going to get those precise requirements because you don't have the resources to make that happen, and if you're tiny, even less so. You're going to get what you get.

And finding a single really good engineer is probably going to serve you a lot better than going and trying to contract with some huge shop that is nameless to you. And they're just going to rotate through a series of people you don't know and probably don't care. But if you're really big, you can set up a long-term relationship that’s not so much different from your internal employees. There's scale effects that really matter here. And you've got to do some deep soul search here about where am I, and where am I going to be a couple of years from now? What is it that I actually need today?

WILL: And one of the things, I mean, you say there's not that much difference in terms of, like, you know, the difference between a long-term contractor and a long-term employee. Well, I'd say one of the big things...so, first off, why would you have a long-term contractor that didn't have a personal relationship with the company or a specialized skill set?

A long-term general contractor is like, oh, he's been on contract with us for five years. That's weird, and probably not...difficult to justify. Because one of the important things is if you have an employee, you have an understanding and a say in their working conditions. You know what I mean?

One of the problems with a contractor is somebody who...say they’re here with you for five years. So, they go from a junior, maybe not a junior, but, like, a mid-level engineer, all the way up to a senior, maybe even a staff. You can make staff in five years, if you’re a hitter, right?

MIKE: Sure.

WILL: So, you keep on bumping their salary because they're doing more for you. But are they getting it? Are they getting it? Is that money making it to them? Maybe. Maybe not. Maybe. You know what I mean? But possibly. But it might not be that way. You know, what's their working environment, right? What are the hours worked? You know, there's all kinds of things that are outside of your control. Like, a vendor might just say, “It's not your business. That's my business,” right?

I don't think, well, this is...we're going a little bit afield of the topic, but I don't think companies should have long-term core competencies outsourced. Like, you're always going to need QA, right? QA is your business. QA is your job. You have to have QA all the time. It’s like, oh, I'm going to get a QA shop to do my QA. Like, but why? Like, why? You can outsource stuff that isn't part of your core competency. But if you're running a software shop, QA, if it isn't a core competency, you need to make it one. You need to make it one today.

It's like, oh, well, we don't have a mobile team, right? I don't have a mobile team, so I'm going to outsource my mobile team. But you have a core part of your business is running this mobile app. You need to have mobile people running a mobile app because this is a direct line of business.

Anyway, I mean, there's always a line. But it’s like, I'm going to have these people work for me for five years. Like, just hire somebody. It's a right-to-work state, baby. You could fire people. Trust me. In the year of our Lord, 2025, you can fire a developer.

MIKE: Look, I'm not in disagreement because there's a lot of stuff I'm with you on, on what you had there. There are some circumstances where having a close relationship with a long-term contractor might make sense. For example, we've got a partnership with a fairly small contract shop in Eastern Europe. And we've had some engineers from there where it would be challenging for them to have a full-time employee because we don't really have a presence in Europe. But they're able to work with us on a contract basis, and they are just part of the team.

The fact that it's a contract is a necessity of the international labor exchange, but they are effectively part of the team. They are treated like part of the team. They have the respect of the team mutually. We respect them. They respect us. They meet with us. They act as an extension of our team that happens to be outside the office. I feel like that is a success story.

On the other hand, if you just have some generic person that you don't really know who they are, you give them a task, and then forget about it and come back a few days later, “Hey, did you get it done?” Well, that's a very different sort of relationship, one that you probably should never have gotten to in the first place. But if you just maybe have the gun-for-hire person who really can get it done, but that becomes a much more challenging relationship to maintain, especially at scale.

KYLE: So, I'm sitting here thinking about this and vendors. Right here we're talking about developers and stuff. I'm thinking vendors and software vendors as well. You're talking about the size of a company. You brought that up a couple of times. That, I wouldn't say matures, that transitions, I feel like, as you go up the line.

If you're a smaller company, you're picking and choosing. You're going with anybody. You're really kind of nitpicking what you want within your budget. But I feel like that almost narrows in because when you're a small, mid-sized company, you're like, oh, I want this contractor. And you kind of get personal with them. You know them.

Where I've seen in larger businesses, it's, hey, vendor that we're using, we need somebody with this expertise. We don't vet them. We don't do anything. They hand us somebody, and then we try and work with them. It's kind of that same thing, but with software vendors that I've seen, too. But instead of the smaller, mid-sized companies, it's, you know, I can go with anybody. It's, okay, you can go with these guys because we have a partnership agreement with them. They scratch our backs. We scratch theirs. And you kind of get that partnership going on. Then your vendors are almost limited, and then you may not even be able to get the best resource for your organization because you have a limited vendor pool, I guess is what I'm saying. And so, like, I’ve ran into that in enterprise locations as well.

MIKE: That's interesting.

WILL: Yeah. I'd love to hear more from you, Kyle, about sort of like that...We’ve talked about this sort of like software consultant contractor thing. I would like to hear more about, like...the thing that I think is most interesting and critical is, like, okay, I'm going to get a vendor, and I'm going to get a big site license. I'm going to write a big check for a big license for a core piece of my business. It makes sense for me to buy versus build, right?

But I don't know these people, and I have two, three, four vendors that I could go with. I don't know them, right? And I'm going to be stuck with them for a year or two at least, even if they dog out. How do you vet people? How do you vet quality of service? How do you vet quality of support? Like, what are the questions that you ask to figure out, okay, this is what I'm getting into, right? Because, you know, they'll promise you the moon. The salespeople will promise you the moon. And, in the meeting, like, they're going to bring out the all-stars. But when rubber hits the road, how do you figure that out? You know, how do you vet?

KYLE: That's currently a problem that we're actually dealing with. Like I say, in kind of a [inaudible 24:05], or I'm just saying, in a company's atmosphere, we're more worried about money most of the time, right? And so, we're asking one vendor, “How much will this cost?” We're asking another vendor, “How much will this cost?” They're giving us a price. Well, that one's a little bit higher. These ones are giving us a price. They're not telling us who. We're not really interacting with who might be doing the work. We're not vetting those individuals out. They're just like, “Here, work with this team,” right? And so, there's that atmosphere.

And then when it comes to the software world, you know, SaaS products, it's a similar situation. And then it kind of gets even larger because you have engineers that will use the SaaS products, for example, that will have favoritism as well. One team will appreciate a product for the application monitoring. Another team will appreciate a product for their tracing capabilities, right?

Well, this SaaS product is really good at that one, and the other SaaS product is really good at this other one. And then, you know, they're not good at the other one. And it's like, we're running into that as well, and, like, how do we mesh that? That's actually one reason why I was excited about this conversation. I’m like, I need to know, too, because those are active problems that we run into day in and day out.

Now, I will say, when I've worked in smaller companies, it was much easier to meet face-to-face with the individual that was, like, going to be performing the work if they were a contractor, you know, and discussing because then it was easier to go back to your manager or whomever, the hiring person, whoever's writing the paycheck, and say, “Hey, I do or do not recommend this individual to perform this work.” But that's not been my experience lately. If that answers your questions, Will [laughs].

MIKE: Well, you know, you've raised a lot of challenges. And two vendors, they're both decent. They offer somewhat overlapping services, but they're not fully overlapping, right? And how do you go with which one? Which team do you choose [laughs]? And that's a tricky one because you're going to have to evaluate from a business perspective which department's more important to get their needs met. And that's hard.

KYLE: Then there's a lot of time where it's like, consolidate to one vendor, right? There's always a push from upper management, “Consolidate to one vendor,” when it's like, well, they both facilitate our needs in different ways. And it's just like, go lesser evil, I guess. I don't know.

MIKE: Well, one thing you can do is, sometimes you're going to lose something, right? You can't win every battle. There are some cases where you can legitimately make the case, but you've got to speak with money. You can say, “Well, if we consolidate on a single vendor, this is what we're going to lose, and this is how much it's going to cost us.” And if you can compare costs and say, “This will actually cost us more year over year,” that's a pretty easy argument to make to people who are trying to save money. And if you can't make that argument, then it's uncomfortable.

But you might need to look a little bit inward because they have accountants for a reason. They've got a finance team for a reason, or a spend management because things do tend to get out of control. And you're combining departments, you know, buying, doing acquisitions. Sometimes you're going to get redundancies. And going with a good enough solution, you're going to have to give something up. And that's always painful, but it might still be the right answer.

WILL: Is there a counterargument to having somebody else bidding against them? I mean, you don't want one vendor for everything, do you? You know what I mean? You know, going at...payment processing, it's nice to have a few payment processors, where they just say, “This is your [inaudible 27:50] I'm like, I don't think...for all the traffic? Maybe not, you know.

MIKE: So, single points of failure are very much worth the cost to avoid at scale. Again, startup may be different from enterprise. But as you grow, the more those single points of failure cost you, you know, if something goes down.

We certainly had vendors go down before. And recently, we’d mitigate a single point of failure. If a vendor went down, like, hey, well, we've got a backup [chuckles]. And it was fantastic because we had done the work. And we were in a position of much more strength.

WILL: Well, I mean, not for nothing, but, like, you know, business organizations can fail, too, and they fail in different ways. But it’s like oh, well, you know, founder decided he wanted to cash out, get that private equity money, buy a boat. God bless you, you know, I'd love that for you, but I might not love it for me so much.

MIKE: And then changes in ownership can have a big impact on what that company is doing for you. A number of times, we've gone with a vendor because we preferred their product, and then they got acquired by their competitor [laughs]. And there you go. You didn’t really [crosstalk 29:10]

WILL: That's a for real thing. It's a big thing to think about in the software space just because, okay, so let's say if you're going out and your vendor is a venture capital funded company, right? You know a couple of things, you know, with a high degree of confidence, never 100% confidence, but a high degree of confidence. They're on a burn rate, right? They're funding. They're funded, and they are spending more money than they take in for growth because that's what venture is like. Otherwise, you're just a private company that has some investors, right?

And the venture model, 90% of those companies fail, right? You know that, right? Those are just the rules. Those are the cards you're betting on. So, fold or acquire. Or, you know, maybe they get a lot bigger, you know, and they get a lot bigger. And they go from, like, a series A kind of a thing to a publicly traded kind of a thing.

That's a brand new company. That's an entirely different organization, and it's going to happen fast. Two years, they're either a completely different company because they won or a very different company because they lost. But, like, they will not be the same company two years down the road. It won't happen structurally.

KYLE: That goes into, you know, acquiring companies, which is, I think, what you’re saying there. Where we’re at, we have Acima. We have RaC, and we have Brigit under our Upbound umbrella. And all of these have different ecosystems that were already in place. And then, how do you consolidate those to the specific vendors, right?

Like, so, when you're acquiring these companies, big or small, how do you make that transition and decide, you know, well, you've got a large group of people over here that they prefer this vendor? However, the core organization uses another vendor. How do you consolidate? And that's a hurdle, too.

MIKE: And somebody's going to not get what they want. Somebody's not going to get what they want. And, sometimes, those decisions do come down to money. Sometimes they come down to size. Well, more people want this vendor than want this vendor. Or the costs to transition are going to be huge.

A very specific one that I think I can share...You talked about RaC and Acima. RaC used Microsoft, and Acima used Google Docs, and we had to choose, right? And, in the end, the tech company who was able to, you know, navigate tech are the ones who had to change because they could [laughs]. They didn't get what they wanted because it was a core competency. They were good at dealing with tech changes. And so, they didn't get what they want because they were good at it [laughs].

And it's a little painful, but it's probably still the right decision because the cost to go the other direction would have been really high and really painful for the company. And that's a difficult choice and not one that made us, on the tech side, very happy because we had to change. But it's still probably the right decision.

KYLE: I had an older engineer that I worked with when I was at NCR, and he made a comment to me one day. I grew up in a mentality...I was very fanboyish. I liked what I liked, and I didn't want to go outside of it. And we were a C# shop at the time, and we were transitioning to being a Java shop. And so, I was kind of teasing him, and I was just like, “Hey, you know, are you going to hate this? Are you going to stick around?” And he's just like, “I'm absolutely going to stick around. I'm a software engineer.”

And I was just like, “I don't quite understand what you're saying there.” He goes, “I'm not a developer. I am not a software developer. I am not tied to a language. I am an engineer. I will use whatever tool the company provides me with, and I will get my job done.” And I feel like that's kind of stuck with me, in the sense of, like, that should be everywhere. It's just, you may not get what you want. You may not get what you've always wanted to work with. But they're all tools. Figure out how to use them for your job.

MIKE: Nailed it, right [laughter]? One thing I wanted to bring up here that we haven't talked very much about are red flags. Because we talked about sometimes you're not going to get what you want. And we've talked a lot about going with contract vendors, you know, with the contract shop you go with, and how scale matters, how there's some challenges with that relationship altogether that you need to be really watchful of. But just in general, and this probably applies more to buying software that you're going to use, I think that there are a lot of red flags. I compiled a list coming [chuckles] into this call because there's a bunch of things that I've seen.

And I mentioned the first one in the intro, security issues. If you do a security, in fact, we recently...unnamed vendor. We evaluated a new vendor. They looked like they had the product we needed. The security team looked at it, and they got an almost failing score. Like, oh, actually, never mind. We're not going to go with them. I feel like any security issues are an indicator of something deeper. If a company doesn't care enough to take care of something as fundamental as security, then what else are they not taking care of? Every time. So, I started listing companies that I've seen security issues on, and it was just a debacle [chuckles].

Way back to right at the beginning of my career, company security issues, total disaster, maybe even sunk our company I was working with at the time. I mentioned the one earlier. I've seen, more recently in my career, partners where they were real fast and loose. They were really willing to share sensitive credentials over insecure channels. As soon as you see that, like, wow, that's not a partner I want to do business with because it's a symptom of an organization that doesn't care.

And it's led me to think, well, there's a strategy. If you can somehow, like, ask them, like, “Could you send us some credentials?” You know, ask them in a sideways kind of manner. Give them an opportunity to do something wrong and see what they do with it. You could probably learn a lot.

So, there's my first one. Security issues are a huge red flag. And if you can somehow find a way to allow them to reveal their nature there...and you can't always. But if you've already got a contract with them and you learn that they have those, it's time to start looking for somebody else because, in my experience, it never works out well. Like, they're not going to fix it. Thoughts?

WILL: I love it, man, just the idea. Like, maybe more broadly is, like, are they minding their Ps and Qs, right? So, not, like, the salespeople, right? But, like, is the contract, like, is the contract right? Are there, like, just goofy things? Like, you know, like, they messed up the contract. They messed up...there are typos in their stuff, like, not in what they say, but, like, in that.

You're looking for, like, I think the quality, the personality quality, is called conscientiousness, right? Conscientiousness. Are the Is dotted? Are the Ts crossed? Because you can see...if you're doing enterprise stuff, right, like, big enterprise things, there's work product, the kind of work product that isn't a good sales guy doing a good job, right? Like, the guy just, like, boring, gritty, pick-and-shovel, get-it-done stuff that is, unfortunately, enterprise software development. You can see that. That will leak out if you're looking for it.

MIKE: And, you know, taking that a little further, next red flag I was thinking about, is there legal action against the company? Now, if they get big enough, pretty much any company has probably had some sort of legal action, which may have more or less credibility. But some stuff has very obvious credibility [laughs], right? And, you know, if there are multiple attorneys general signing on multiple times, it's a red flag, at least, that maybe there is something not going right. And it goes back to the care. If somebody is careless about their customers in that way, they may not be treating you very well either.

WILL: I mean, I'll say, as a small shop, like, I'll say two things, right, like, here I am generally showing my ass a little bit. I'm not a voyeur, and I'm small, and I don't have, like, you know, all that stuff. But I have definitely...when you are a little fish in the pond, there are a disturbingly large set of people that will bully you through the legal system. I have been involved in more than one or two legal actions, all of which, every single last one of which was somebody trying to rob me and then saying, “Do something about it.” So, you know, beware, but I would say that, like, a bigger company would have learned their lesson the first time, right?

MIKE: [laughs] Well, that's why I try and preface that. Well, pretty much every organization is going to have spurious legal action, or at least in a gray area, where, well, yeah, maybe, you know. There's somebody who doesn't want to pay you, and then there's Enron [laughs].

WILL: I get patent-trolled. Like, do you get patent-trolled? I got patent-trolled, you know, and it's just, like, bro, what?

MIKE: [laughs]

WILL: Taking a credit card, like, that's the patent you're going to come and get me on? Like, if you could really afford that patent, you wouldn't need my money [laughter].

MIKE: Yeah. So, at the end, there's a tricky one. It's not that there will be no legal action, but you should at least look. You should at least look. And it's not like companies I've worked with have not been involved in legal questions before. But I've left a company before because they were doing some legally questionable stuff, and I'm glad I did. You know, there's a line, and a great deal of it from reliable sources often means something.

WILL: Absolutely. Yeah. Yeah. And I'll say this, especially legally, you know what I mean, highly regulated industries, you know, minding your Ps and Qs. Acima has always been scrupulous in following the law to the letter. Other big companies that I have worked with they've all been on their game. But Acima is definitely the leader of the pack in that regard. The current gig is a close second, but, like, yeah. This is the financial stuff, right? You've got to have it together.

And, like, living in Utah, I will say that Utah...there is some business in Utah. We do some stuff in Utah [laughs]. I'm not familiar enough to pursue that one. I’ll let that go. That sounds interesting.

DAVE: I am, and I'm also not going to pursue that one, at least not on the call [laughter]. We’ll talk afterwards.

WILL: You can just do a quick Google. And this has nothing to do with me. But how many Utah attorneys general have gotten indicted in the last 20 years? A lot more than you think. I think we're batting over 500. I sincerely, hand to God, I think we're batting over 500. This isn't Mississippi, man [laughter]. What are we doing?

MIKE: Illinois has had a streak with governors [laughs], so I can relate a bit, and other political entities. Moving on.

WILL: You also have a reputation. Yeah, moving on [laughter].

MIKE: Impossible promises. If they're promising you the moon, walk away. Just walk away. It gets to some of our biases, to wanting what they have to offer.

Relevant story here. So, this isn't a vendor, but I'd say it was an employee I hired once who was doing some sales, or business sales, like, relationships. And he made some big promises, and they all turned out to be hollow. And we listened because I'd actually done work with the guy before, and he'd brought me business before. He seemed credible enough. But I got that bias, like, oh, this sounds like the ideal work for me and my shop. And so, I overlooked some of the red flags there. It was too good. It was too good. And got to watch out for it.

So, on the flip side, I've got a list of green flags as well. A vendor should be boring. If they're boring, that is the biggest green flag ever. If they're boring, that means they do their job and it's not all marketing. There's nothing better than a vendor who is just, like, watching paint dry, right [chuckles]? You know, there is nothing interesting about them at all. That is your first choice. Because --

EDDY: So, I'm halfway there. Is that what you're saying?

MIKE: [laughs] So, boring, great sign. That means that they know what they're doing, and they're probably doing it really well.

So, going back to the...well, I'll give an example of impossible promises. And I'm going to make a positive shout-out to Google here. They were pitching us on some tools, some AI tools, and they told us what we'd get from it. And it was honest [laughs], and it was not a huge, dramatic change, but it was something that could make a meaningful business difference.

And it's hard to argue, you know, that was such a good sign. It was such a positive sign to see somebody who would give you honest assessment that was not going to make everybody jump up and cheer, but was something that would be enough to make it worth it. You know, it's always possible that the numbers weren't perfect, and they probably weren't. They weren't for many vendors, you know, from any...it's hard to get a perfect number. There's maybe no such thing, but it was believable.

A couple of other red flags I thought of: possible conflict of interest with internal leaders. I mentioned that with the vendor at the beginning of the call. You know, if somebody might be getting a cut, or somebody has a long-term relationship there, you know, you want to tend to trust that, like, oh, so they know them, so it’s good. Well, maybe, but give it some scrutiny.

If there's a conflict of interest there, you should be really careful. Now, especially if you're going with a smaller vendor where you don't know much about them [chuckles], but, yay, this is my friend; he knows exactly what he's doing, give it a little scrutiny; often a good way to get a contractor. Maybe not such the best way when it's not...again, we're talking tech. The non-tech executive who brings in somebody and says, “Oh, yeah, we're going to go with them because they're my friend,” that's a red flag.

Another red flag: they have one person who talks to you, and they're great. It's easy to have one person who can talk well, and they may even be really good. They might be their only real good person [chuckles]. If you can get a team of people who can all speak confidently, that says something about their culture. Wow, I’ve got a group of people who know what they’re doing. In a really large organization, well, maybe again that’s easier to fake, sending their best, and they’ve got a lot of people who aren’t the best. But at least for a smaller vendor, [crosstalk 45:02].

WILL: Is it possible to say, like, “No, I want to talk to the person who's my account guy?” Is that a reasonable ask? Because, I mean, on one hand, like, I...how do I put it? Like, if I'm busy doing support, right? Like, I don't have time to do sales. Sales isn't my job. I'm busy supporting the customers I have, and I understand that. I understand that. But, like, if they don't have time to talk to me before I write the check, what are they doing after?

MIKE: [laughs]

WILL: Like, I understand that, like, you know, you don't want to burn up their entire time. But, like, I want to talk to the guy, not, like, your best guy. Don't give me the all-star. I just want to...I want to talk to the person that I'm going to be dealing with now.

And I also, like, maybe as an aside, you know, you guys have...you guys have bigger, like, all the software that I've bought, like, I buy, you know, off the shelf, you know, these are just SaaS, like, you know, because I'm not...I'm not big and bad and all that. Like, you can negotiate sort of quality of service in these big enterprise contracts, can't you? Where it's just, like, if I send you an email, I want an email back, you know, in this time frame [inaudible 46:10].

MIKE: Absolutely. You can set up all those quality of service items in the contract. And I think they're fairly typical. You know, I wish...you said, “Hey, can you go talk to that person?” I wish I'd done that several times. And maybe something that I'm going to ask next time, “Can I talk to my actual account representative? Not your best person you sent to me, but, you know, who's the person who's actually going to be working with us? Can I talk to them?” I think that is a fantastic question. And if they'll make it happen, it says something about the vendor as well.

I threw out a big list there. And I haven't gotten a lot more than, like, nods and mm-hmm.

DAVE: You're covering it well. Yep.

MIKE: [laughs]. Yeah, I thought it through. Anything you'd add to the list of red flags?

DAVE: I would add a thing on there. You mentioned, like, SLAs and, like, quality of service. The thing where it has hurt me recently is people saying, “Oh, I want this. I want this, I want this.” And then, they come in, and then here is the third party we're going to use. And then, you hand this out to all your employees. And then you never hear back on...you know, the only SLA report you get is from the vendor, not from your people. And so, yeah, you can run into some fun things there.

Joel Spolsky, the guy who originally...he ran FogBugz. I think Stack Overflow, I think, was his baby way back in the day. He talked about selling software in the 2000s, and he called it steak and stripper-based software sales, which is where you take somebody from the C-suite out to dinner. You buy them steak. You show them a good time, and you sell them the software. And then they go home, and they tell everyone at work, “This is what you get to use because it's great.” No, they had a great steak, and they had a fun after-dinner entertainment. They never even looked at the recs, right?

And you used to be able to make really good money selling software that way, and then come to find out that once it actually hits the ground, the service you're getting, yeah. Make sure you've got transparency into, like, how the QoS is working, how the SLA is being honored.

MIKE: That's an interesting one. And it says something about being willing to push back if you can. If you've got some capital to spend there, social capital use it to do real evaluation because there's a lot of things that slip through when you don't scrutinize. Go ahead.

KYLE: Another one that I've got is maturity. At least to me, it's a red flag. If it's a B1, if it's a beta, I'm sorry. I'll probably pass until you're a little bit more mature. I don't want to take that gamble.

MIKE: That goes back to the scale. If you're a startup and they have exactly what you need, nothing more, that might be the right choice if they're in beta. If you have gotten any scale and you actually need that thing to stay running, you should probably walk away.

WILL: I mean, a big piece of it, too, is what kind of software are you buying, right? If it's ops-type software, I don't know, man. Maybe not. Maybe not. You know, there's a lot of people where it's just, like, just write Oracle their check. It's fine, you know? [laughter] Maybe not that kind of a check. But, like, you know what I mean? Like, somebody with some gray in the beard, you know, like, there are things you can mess around with, and there's things you can't. Some software, it's just like, yeah, I want it. I want it battle-tested.

KYLE: Even there, you know, and I'm saying maturity within their product line, too, because there are features specifically, you know, I'll throw out AWS. I don't think that's bad to throw out. We have done evaluations from time to time, and it is generally better to wait for V2 [laughter]. We have run into that recently and have had to backtrack our implementations, and we're waiting for V2 of a product.

MIKE: That's interesting. You could think about AWS as a bunch of little companies [chuckles].

KYLE: Yeah, you really can.

MIKE: Or a bunch of little vendors, yeah, because they operate so independently.

KYLE: Yeah, if you want to see that, just go to the different services page for each of their products. It's designed by a different team, every single one of them [laughter].

WILL: I can't keep up, man. AWS is coming out with a new, like, something every week. I remember when AWS came out. I was an early adopter, right, because, like, it was brand new, and, like, we were just getting going. We had just moved off of the data center model. And, like, our team, our stack was, like, just brand fresh out of the oven, and we didn't have a lot of dough.

And so, we just got right on AWS, like, right off the jump, I mean, nearly at launch, and whoa, it's stunning. They’ve come out with so many things where I'm just like, I sort of start to question, like, who needs all this? This is crazy.

MIKE: The interesting thing is a lot of their products came from internal, at least originally, came from stuff they were doing internally.

WILL: Yeah, absolutely.

MIKE: If there's a niche [laughs], there's, you know, there will be product to fill it.

WILL: Yeah, absolutely. Well, anyway, a little bit of a tangent, you know. I liked AWS over the other maybe things because they did, like, less, you know, and now they're very much doing more.

KYLE: They are doing more, but I will give them the benefit of the doubt in this scenario. AWS is the more. If you want less, you do, what is it, AWS Lightsail? That's kind of your small individual one, you know.

MIKE: So, we’ve talked about some red flags, some positive things, right? They should be boring. They should be mature, have some longevity, used by other companies in your industry. They have multiple competent people who come and talk to you, you know, they'll let you talk to your rep, and they leave a good impression. There are some positive things. We’ve talked about a lot of red flags.

WILL: I have a red flag for you.

MIKE: Okay.

WILL: So, I won’t be too specific, although, you know, if you know the space at all, you'll probably know who I'm talking about. So, there was, early in my career, I worked on the corporate side of a multi-level marketing company, which don't get me started. I was young, and poor, and hungry, and I needed to eat.

DAVE: Needed the money.

WILL: Hey, I needed the money, you know, whatever. So, there was a vendor for their sort of, like, downline payment processing payout for their tree. Tree was upside down, but it was a tree. Not a different geometrical shape, but they were nightmares to work with. Absolutely horrible. They were expensive. They were slow. The SLA was more of a middle finger. It was pretty intense. It was pretty bad.

And the red flag...but it was also, right, it was also textbook case of, like, buy it; don't build it, right? Because it was this very specific, very niche financial product. You wanted it to be right. Like, I'm here to be the pharaoh on top of the pyramid, not, you know what I mean, this sort of, like, esoteric, arcane financial billing thing. Like, that's not my job. I'm here to do whatever it was that company did. You know, it didn't have anything to do with software.

But the red flag is, like, they were the sole player in this space. They did not have competitors, like, meaningful competitors, right? Because it was the 800-pound gorilla in this very niche space, and, like, there wasn't a negotiation, right, where we're, like, we're getting our enterprise contract. They were, like, this is it. That's what it is, right? Nobody was coming behind them. They ran things, and they set terms.

And so, like, you know, like, the old, like, Ma Bell monopoly. When you deal with people like that, you're going to get a...they're established. They're very big. You're going to get what you get, and you don't get upset. And I don't know if there was a number two, but I feel like the number two would have cared. They would have pretended to care at least a little, I think.

MIKE: [chuckles] Well, there's times when you may not actually have choice. But the small choice might be the right one, the disruption.

WILL: Maybe if we were a bigger account things would have been different. You know, I think fly-by-night, multi-level marketing, like, startups, are not [laughs] afforded the respect that you feel they might be deserved to. I know what I thought about the place, and I worked there. They were my only client. And I was just like [laughter], make the check out to cash.

MIKE: [laughs] One thing I wanted to touch on before we close is you've got the relationship. You've got the vendor. What do you do in your first year, I'll say? It may not be your first year. You've got a contract for a year, and you’ve got a contract for two years. How do you establish that in a way that makes sense going forward?

And I've got one thing that I wanted to touch on. Try to build your code. Try to build your integration to be vendor-agnostic. Build an integration service that you go through before it goes to the vendor. And this isn't always possible if they're running your infrastructure. And, Kyle, you know [laughs], sometimes the infrastructure can't be agnostic.

Well, that's a lot more...infrastructure is a lot easier to move now, right, to be multi-cloud than it was in times past. If you can make it vendor-independent, then you save yourself a lot of pain going through a year from now and changing the name of that vendor everywhere in your code where you're stripping it out and completely changing to call somebody else's API.

If you build a generic integration service that just abstracts that idea, you've got to change it in one place, or more or less. You know, you can minimize the damage. And maybe they're a great vendor, and they were a venture-funded startup that goes out of business. There's all kinds of reasons where maybe they were even a great partner at the time, but it's not going to work out in the long term. And having the ability to pivot is good. I think that building that in is worth it almost every time. What do you all think?

KYLE: Yeah, whenever we're evaluating new infrastructure components, that is something we take into consideration. I'm thinking specifically AWS. But, you know, we try not to run anything that only AWS provides, for instance, for that reason. You know, we're using Postgres. We're using OpenSearch, you know, stuff that the other cloud providers will also offer. That way, if we do need to pivot, those type of things will historically bite you, and those are large migrations.

Even monitoring tools, you try and make it a drop-in. So, kind of like you're saying there, Mike, you write a layer or something to where you can drop those in, and they're not hard integrated into your environments. That way, you can pivot.

MIKE: We do this audio only, but we record with video [chuckles]. I see nods all around the room [laughs]. All around. Well received. Any final words?

DAVE: I would say be careful to pay attention to what's important. If you listen to a recruiter's ad, they will tell you that we solve the problem of how hard it is to find the right people. If you're hiring people through a contracting agency, that shouldn't be their primary sell. I mean, it should be. They should absolutely be finding the people for you. But if the only thing they're doing for you is finding the people, are they managing the people? What work are you outsourcing? And especially if the only reason you're doing contract-to-hire through an agency is because you're bad at hiring or bad at recruiting, you're probably paying through the nose for not having somebody good at recruiting. Focus on what's important.

EDDY: I think at one point or another, all of us who have used a service or software have said to ourselves, man, I can build this product so much better [laughs]. And if you can, maybe that says more about the software you're using [laughs].

KYLE: It's also a chance to reflect, though: are you using the software correctly? Is your practice correct? Is something I would warn against there.

MIKE: Well, and it goes back to being willing to evaluate. If you don't do some evaluation and say, “Is this still the right vendor?” Then you may find yourself in a long-term bad relationship, and it's worth it to do some introspection. There's no shame in reevaluating that choice because times change, and maybe it was a bad choice to begin with, or maybe it's an even better choice, right? There's cost to moving. And if it's okay, that's good enough, right? But be willing to reevaluate. It's not quite presence of mind here, right? Have the business willingness to revisit your choices.

Great discussion. One of our --

DAVE: This was good.

MIKE: There was a lot of stuff to cover here, and we covered a lot of it. Hope that you get something from this and can make some better and better decisions when you're picking your next vendor. Until next time on the Acima Development Podcast.

In this episode of the Acima Development Podcast, host Mike opens with a gardening metaphor to frame the episode’s central theme: cultivating healthy company culture. Drawing from his personal experience transforming barren soil into fertile ground, Mike contrasts short-term fixes like chemical fertilizers with long-term strategies like feeding the soil itself—likening these to how companies often rely on perks or high salaries to attract talent rather than investing in sustainable, growth-focused environments. This sets the stage for a broader conversation about what truly nourishes both plants and people—ecosystems, relationships, and meaningful investment.

The discussion evolves into a rich conversation among remote team members, including Justin, Javier, Jorge, and Will, who share their experiences working in international and distributed environments. They highlight the importance of communication, inclusive documentation, and feeling culturally and professionally supported. Javier and Jorge reflect on challenges faced by Latin American contractors, including language barriers and differing communication styles. They emphasize that well-documented processes and clear expectations can bridge time zones and cultural gaps, especially when employees are empowered to grow and contribute meaningfully.

The group critiques superficial attempts at building culture, such as gift cards and perks, and stresses the importance of leadership that fosters personal growth, purpose, and strong communication. Leaders who spend time mentoring, understanding their teams, and clearing barriers can create “fertile soil” where people thrive. Whether it’s through offering growth opportunities, creating psychological safety, or enabling people to do impactful work, the team agrees: you can’t shortcut your way to a strong company culture. You have to invest in it intentionally, consistently, and with heart.

Transcript:

MIKE: Hello, and welcome to another episode of the Acima Development Podcast. I am Mike, and I am hosting again today. With me today I have Justin and Javier. And this should provide some good perspectives. Justin previously worked for Acima [laughter] [inaudible 00:39] and is now elsewhere; I'll leave unnamed [laughs]. Javier is a contractor that works outside the office, outside the country, which will provide some good perspective because we'll get good perspective to the topic today. I'm going to announce the topic in a minute [laughter]. Although you could probably see it from the title of the podcast recording but, you know, at least I'll pretend to leave some suspense there [laughs].

JUSTIN: I'm looking for a good story, Mike [laughter].

MIKE: I'm going to talk about my garden.

JUSTIN: Oh.

MIKE: I've enjoyed gardening since I was a kid. I like growing things. I like being outside. I do have a garden. To be honest, it's maybe a little neglected as of late, but [chuckles] I do have a garden. And I say neglected, I've let some things grow and spread. So, I've got a big patch of raspberries, for example, because I love raspberries. What's wrong with letting the raspberries spread [chuckles]? They're delicious. I've got quite a bit of garlic, and I've got some other berries. I’ve got some goji berries, some garlic chives. So things that have kind of spread some, but I like them, so it works great. And I have a lot of kale because I love kale. I eat kale every day.

I've read a lot about gardening. I, at one point, considered changing careers. I was going to work for a gardening company on their tech side for a while. I didn't end up doing that, but strongly considered it. And my original major was botany, so definitely very interested in this topic. I've got, like, a hundred cactus plants in my office that I love. Most of them I grew from seed [laughs]. I like plants. Pandemic, you know [laughter].

So, I'm going to talk about a thing that people regularly get wrong, and even to the point of agriculture sometimes gets wrong, and they're turning from this. And there's been a turn over the last few decades to fundamentally shift the way people think about not just gardening but farming. Where I live, I have quite a bit of farms nearby. In the state of Illinois, there's a lot of corn and soybeans. If you've ever been there, you know what I'm talking about [laughs]. There's a lot of corn and a lot of soybeans.

They used to just plow everything under every year or just treat the waste from the previous year's corn and soybeans as waste, and just kind of ignore it, let it blow away. And when they’d till it under in the fall, the soil would blow away all winter. And there's places you can go and they'll show you how deep the topsoil used to be and how deep it is now [laughs]. On the rest stops on the side of the road, they'll show you how much topsoil has been lost because it's a big deal, like, half or more of the topsoil has been lost in areas. And it's a really serious problem because now you can't grow crops as well because you've lost that deep reservoir of fertility that allows you to grow.

And so, they've got techniques they do in farms. One of them is that they’re moving away from tilling. They allow the stubble to stay on the ground so the soil doesn't blow away. But it also means that it decomposes into the soil. And there's a reason they have corn and soybeans also. Because soybeans are legumes, they've got nodules on their roots that feed microbes, that take nitrogen out of the air and make it into a form that the plants can use. So not only are soybeans high in protein and nutritious, they also leave the soil better than before you planted them. So, they improve the soil.

So, it's part of a rotation where they grow the soybeans to improve the soil. Then they grow the corn, which is really hungry and uses up all that nitrogen. And then, they'll grow soybeans again to improve the soil again. So, there is this cycle of rotation, which is something that was done years in the past. People have done crop rotation for probably a millennia, realizing that certain plants will take certain nutrients from the soil. This is getting to engineering, by the way. This is getting to engineering [laughter].

JUSTIN: I'm getting a minor in botany right now.

[laughter]

MIKE: In my garden, so my garden, when I moved into where I'm at now, I took the worst soil in the yard, along the back of the property. And they’d cut away the topsoil [chuckles] when they built the house, and it left just subsoil. And it's just...it's clay, and sand, and gravel, and really nothing else [laughs]. And I thought, this is a project. I want to see if I can make this grow stuff. And I guess the previous owner of my place was a grass seed salesman. He couldn't get grass to grow [laughter]. It was just bare dirt on the area where my garden is now, and there was ravines all over through it. Cutting down, you know, it was bad.

And so, to the key point, what a lot of people do is they'll go buy some chemical fertilizers, and they'll throw that on the soil, and the plants can actually grow in that. You know, you can get some things to grow. In the extreme case, you can do, like, hydroponics, and you can actually kind of make it work with just chemicals [laughs]. It's a fussy business. And the minute that you stop augmenting with those chemicals, the whole thing falls apart, right? And everything dies [laughs]. You need to continuously add those chemicals in order to make it work.

And they actually degrade the soil over time because they usually come with some salts that will build up in the soil. So, you'll get these minerals that build up and actually make it harder and harder to grow things. It's a problem. It's a real problem. And it reflects a misconception as to what you're doing there in the garden. Because I have a garden...it's a small scale. I can really focus on that. And the change in mindset that makes all the difference is you don't feed the plants. You feed the soil. You don't feed the plants. You feed the soil. If you feed the soil...so, think about the difference that would make. So, I'm going to focus on improving the soil.

Going back to those farmers who have lost all the topsoil, like, oh, wait, we need to stop this, right? I need to preserve the soil I have and improve it. In some places, they actually are improving the soil. They're growing it again. If you feed the soil, then the healthy plants will follow because the plants don't live in a desert, in a vacuum. We're not on a space station. Even if we were on a space station, plants have not evolved to live in a sterile environment, and don't live well in a sterile environment.

You may remember the book and movie The Martian that came out a few years ago [chuckles]. A guy had to poop in his garden in order to get microbes to grow because the plants wouldn't grow without it. It’s the same idea. You need to have this ecosystem, or the plants just don't do right. And if you build healthy soil by feeding all of the stuff that's living in the soil...because it's not just the plants. There is a huge network of fungi, bacteria, protozoans, insects [laughter], rodents. There are all kinds of things that live in that soil. And your plants are just one of them.

If you have incredibly healthy soil, by adding things to feed the soil, feed that whole ecosystem, then the healthy plants will follow because you're focusing on the right thing. You're building the healthy ecosystem, the healthy environment, and then the plants will grow well as a natural consequence because they'll be in a place they want to live. All of the things that would be good for them are there. And so, you don't have to go and find all of those precise chemicals. Oh, I need this much nitrogen. I need this much phosphorus. I need trace amounts of copper, whatever the case may be. You don't have to worry about that because everything else is taken care of. That will just naturally be taken care of because everything else is so healthy.

And you might have to worry about weeds [laughter] because they'll grow abundantly in your garden as well, but that's a problem you like to have. You want to have a garden where you can actually have things grow. You can actually solve weeds the same way. We actually want to diverge. You feed the soil on top of the ground where you don't have the plants you want, and then the weeds cannot grow there. If you put a lot of mulch there, then it will gradually break down, become good soil, also smothering weeds.

Which was a long introduction [laughter], but I think it's hugely relevant here because we're going to talk about company culture and how you build it. And talking about this garden shows some of the ways you can do it wrong [laughs], as well as the ways that you can do it right. There's a lot of ways that you can kind of fake it. You can add those chemical additives, and you can pay really high salaries, for example. And yeah, you'll get a certain sort of mercenary person who wants to go for the high salary. But it doesn't really build a healthy culture a long time.

Not that it's bad to pay people a decent amount, don't get me wrong [laughs]. But you can't solve some problems with money. If people hate being there, really, there's not enough money you can pay. There's an example of this that's been in the news somewhat of late. I guess Facebook has publicly somewhat denied it, but there's been news reports that they are offering seven-figure salaries to AI researchers if they'll switch over to Facebook. You start adding up those zeros, and that's annually, right? That is huge amounts of money. And I think they've got, like, one [laughs] new researcher that's --

JUSTIN: I think there were a couple that came over, but yeah, it just goes to show that, I don't know, they may be focusing on the wrong thing.

JAVIER: I just wanted to share an experience because, previously to Acima, actually, I was working in a company earning probably 30% or 40% more than now. I really hated my job.

JUSTIN: Oooh.

JAVIER: Yeah. So, definitely, you know, money is good, and I do have money. It's good for me and my family. But the culture sometimes, you know, the way you feel doing your job can be definitely different, even with good money. So, I’m completely agreeing with you.

JUSTIN: Yeah. And if you somehow get that combination of good money and good culture, I mean, you get people who are willing to go the extra mile for the company, and that is fertile ground for explosive growth. Going back to that garden thing, you know, you look at what you reap from a really, really fertile garden. It may take a season, but you're going to get two, three, four times what other people are getting because you spent the time to prepare that ground. And not only that, but it's like, you'll be spending less time weeding. You'll be spending, you know, all your focus will be on growth or productivity. So, that is, like, the ideal.

And so, I've been in a couple of places that have been good, and I've been in a couple of places that have, basically, it seemed like they were salting the earth [laughter], and people were leaving left and right. So, it's like, wow, what a great analogy you have here, Mike. But you look at it and you're like, it's not something that just happens. If you keep on going back to the analogy, the gardener or the farmer, the person in charge there of the soil, they are spending time preparing that soil. And they are just, like, you know, doing everything that they can to prepare it. Because it's like, they aren't the ones that are necessarily producing the thing. They're preparing the soil such that the thing could be produced.

MIKE: You hit on a key point there. They're not producing it [chuckles]. They are building that environment so that the production will happen on its own. And I think that distinction is so important [chuckles] and really easy to miss with short-term thinking.

JUSTIN: Yeah. And this is funny because I'm always torn about the huge amounts of salary that CEOs get paid. And you look at that and you're like, how could that guy, usually guy, how could that person be worth that much money? And you realize that sometimes they aren't, obviously. Sometimes those are massive failures. But sometimes they are amazing at farming.

MIKE: Yes [chuckles].

JUSTIN: They're growing people. They're growing a company. And they have the right ideas to prepare that soil such that that company can be very successful. And I can think of a couple of places that I've worked that have gone through a couple of different CEOs. I think it's been long enough now, but I specifically worked at SoFi. And they started out with a decent, I mean, they recognized that, but they had kind of bad leadership at the top.

And then, the person that came in as current CEO, whose name is Noto, who I really respect still, he came in, and he has created a culture that even though I don't work there anymore, I recognize that, hey, that was an amazing culture that fixed a lot of the problems that the previous one had, the previous CEO had, and he did it from the top. And he came out and figured out all the intermediate leadership. And to the point where it's now, I still have friends that work there, and they are very happy with the way things are going. And so, it's just like, I look at that and I'm like, that is an example of a great leader who recognizes how to grow a company and the people.

MIKE: Well, let's talk a little bit about what might be some of these chemical additives [laughter] they might use to cope for a little bit, but don't really have long-term success, and then what actually matters.

A lot of years ago, I used to have a job where I'd take the same bus to work every day, and I got to be kind of friends with the bus driver, who was also a university professor. That's the problem with adjunct professors [laughter], is that they have to go drive a bus to actually make the money. That's a topic for a different day, but there's some challenges in academia.

So, he was a university professor who also drove a bus, and he talked a lot about the incentives that companies will give to try to get their employees to do something, like, they'll give out, like, here's a $50 gift card, to give an example of that. And a lot of companies will turn to something like that, to, like, oh, we need to improve our culture, so we'll give people lots of these little gifts. He was not a fan, I'll say [laughter], because he said, “You know, people can look...” So, he was thinking of somebody who wasn't getting paid enough at his job, so he had to go get a second job. He says, “You know, people can look and see, you know, I'm getting this $50 gift card. It's maybe nice today, but it's not feeding my family [laughter]. It's pretty superficial.”

Not that it couldn't be a little nice to do now and again, but if your entire plan for helping your team be happy is to give them a gift card once a year, you're probably not going to be very successful, especially, I'll say, if you're working with a distributed team where you've got contractors all over the world, you know, where a lot of times contractors aren't getting that, right, you're not going to build a company culture off of handing out those little gifts. It's an utterly inadequate strategy.

JUSTIN: I do have to say, I knew of a company that took that to the extreme. They handed out those all the time, and that was the culture [laughter].

MIKE: Oh.

JUSTIN: And so, every time he saw somebody, he was like, “Here, here you go.”

You remember Overstock? That went through a lot of different phases, right, growth and shrink. Well, actually, you probably aren't as aware with, you know, where you're living right now. But here in Utah, overstock.com had a very interesting CEO who had phases where he would grow the company a lot, and then it seemed like there was a great culture. And then, something would break culturally, usually the CEO, and then they’d go through a bunch of shrinkage.

And it got to the point where it was kind of a joke within software engineers in Utah because you all did your time at Overstock [laughter], you know, you'd go in...I interviewed there, and luckily, I declined their offer. But it was interesting because they went through phases where the culture was actually really good for software engineers. And they were well-compensated, and they were well-regarded within the company. And they were producing a lot of successful websites, and the company was growing, and things like that. And then, they went through periods of time where the budget got shrunk, and all the little perks that they had gotten used to got cut, and the pay got cut, and then they went through several rounds of layoffs.

And so, it's interesting because you have stuff that's external to the culture that gets cut, I mean, if the company goes bad and their budgets get cut, all of a sudden, those little perks that were part of the culture they go away. And you're left with, oh, why am I here if I'm not getting that $50 gift card, or if I'm not getting that great quarterly vacation to Las Vegas, or something like that? And people were looking at it like, you wonder if they ever got to the point where they were an actually good engineering organization, or was it that they just had a lot of neat perks [laughs]?

And so, the perks inspired a certain amount of loyalty for a certain amount of time, but it was, like, those specific chemicals that got added that...and when those chemicals got taken away, people didn't want to stay. And it went just beyond just, like, the cuts to the, you know, the layoffs. They had people actively leaving, you know, looking for other jobs because the company culture got toxic. And it was interesting. It was an interesting story. And yeah, I don't think I have any friends that work there anymore, and the company's gone through several reinventions. They're now known as Bed, Bath, & Beyond, actually.

MIKE: Heard that.

JUSTIN: But it's funny because the culture got so bad, and they got such a bad reputation that they actually had to rename the company so that people wouldn't associate it with them anymore. But yeah, a lot of my experience, obviously, has been with local culture, and, you know, we've been over some of the bad things that happen locally. How about you guys...can you guys talk a little bit about the bad things that may happen, you know, badly for remote? And then we'll get to the good stuff, right? So, that's the intent.

JAVIER: Well, you know, as remote developers, sometimes communication is a big deal. Well, for example, here in our team, something that is very important for us, I think, is that we are able to have a community of Latin people. Because, you know, Latin people is a bit more fun people. I don't know how to say it. Maybe Jorge has a better idea. But yeah, that is part of our challenge, is to have communication with, let's say, the other side. And when I say the other side, I mean no Latino people. Sometimes it's for culture, or whatever, it's difficult to have this communication, and that is difficult to handle. Yeah, that's my thoughts right now. I don't know, Jorge, if you have some opinions about this, or...

MIKE: By the way, Jorge has joined us. He joined us a little bit late [laughter].

JUSTIN: Jorge, where are you located at?

JORGE: I'm in Lima, Peru.

JUSTIN: Lima, Peru? Oh wow. I've never been. That sounds exotic and awesome.

JAVIER: You have to go, best food in the world, I think.

MIKE: Oh.

JUSTIN: Oh wow. Hey, I'm going to have to go now [laughter].

JORGE: Yeah, but if you come here, you're going to return to your country a little bit fat because here is, like [laughter], a lot of food, every day [laughter], every hour.

JUSTIN: Gotcha.

MIKE: Do you have any thoughts, Jorge, on what is critical for company culture? So, Javier said something interesting there. He said that not being, like, on an island, but having a group of people that you can identify with. And that can mean a lot of things, but maybe most importantly, it means that you're not alone, that you've got somebody else that you can pair with, that you can ask questions to, that you can feel like is a friend, seems like is pretty important.

JORGE: Yeah. In my experience, as Javier told us, communication is really important, and even not only communication but defined processes are too important. Why? Because, for example, sometimes we don't know about the best practices inside a company to fix a problem or an issue, for example, and that's a really good point. That's one of the reasons I love a lot of the process here in Acima because we have, like, a really big Confluence page [laughter]. We can find whatever we need inside of it, right? But that's a good practice that a lot of other companies don't do, right?

So, that's key in the company and in a good technical culture because if you need to do something, at least you can check your recommendations and try to fix something in a good way, in an expected way for maybe the teams, right? So, I think it's a really good, important part of our jobs to get that documentation as updated as we can, right? As we can. So, I think both of them, the communication and the processes, are the key in a remote culture and work.

MIKE: It’s interesting. You mentioned the documentation. Have something written down. Go ahead.

JAVIER: I have something to say about this because, again, as a Mexican, as a Latino...and I think this is many of us. I think many of us we think the same, because sometimes...of course, this is not offensive, not at all, but, you know, the American communication could be different if you compare it with Latino communication. And sometimes documents and documentation itself is prepared thinking in that culture, and it's perfectly okay.

Just for me at least, it sometimes could be difficult to understand the sense. Maybe I can understand the words in English, you know, but the sense, for example, you guys use a lot of acronyms [laughter], and Latino people make me crazy sometimes with acronyms [laughter]. And sometimes the descriptions are very small, and maybe Latino people need a bit more, you know, something a bit more [SP] más carne.

JUSTIN: Más carne. [laughter] Meaty --

JAVIER: Yeah [laughs], and sometimes that makes you feel a bit like I'm part of another subculture inside of the culture of the company, in this case, the companies from United States. At the same time, this subculture could mean you're in a different group, and that sometimes feels like you can, you know, be completely apart. Maybe I'm not expressing totally because my English is not so good, but yeah, hopefully, appreciate my thoughts.

JUSTIN: So, I have some thoughts on this. I've worked with contractors from Colombia, Medellín, and Argentina, and a couple of other places, and as well as with contractors from Eastern Europe and from India, and, hopefully, I'm getting a couple of contractors from India in the next month or so.

But you brought up some really good points there of, like, you know, making sure that you're included on the culture and that the documentation is really good. Because you can go read documentation, and if it's good, you don't have to, like, necessarily ask questions, and it conveys what the culture is. If the documentation is not good, oftentimes, you're kind of, like, left in the dark, and you're trying to communicate with people, and nobody will respond, and so on.

But if the documentation is good, it helps you understand, and it enables you to understand and fix the problems yourself. And also, if there's, like, a culture of fixing documentation, or adding to it, that is really key as well to, like, helping people not be frustrated. And this is something that I've noticed, is if you can enable people to be successful on their own through documentation, through clear processes—I like how you talked about that processes part—if you can, like, show how the processes work, and what the theory is behind them, and why they work the way that they do, as well as, like, have good documentation, you're enabling those other people to be successful. And you don't have to be there to, like, answer every question.

And it's especially important as you get, like, further time zones away, and you're not, like, overlapping. Because, you know, all of a sudden, if somebody offshore has a team, and they, like, have an eight-hour difference to you, they'll write their message. You'll get it first thing in the morning, but they'll already be asleep. And you'll reply, and all of a sudden, this task that, you know, in reality should have taken them a couple of hours at most, really ends up taking, like, a week or two or more.

And nobody likes to be unproductive. If you are being unproductive, generally, it's, like, you are unhappy, at least myself. And, all of a sudden, you start looking around, like, oh, this company is not utilizing my potential. I'm not growing here, and I'm just banging my head against the wall. And that'll enable you to go look for something better, and even that something better could be a pay cut. So, it's, you know, all those things. And I've rambled a little bit here, but yeah, those are some of the items I know.

MIKE: You touched on a couple of things that I wanted to bring up. You said that everybody wants a chance to grow, and everybody wants a chance to do something meaningful. And both of these go down to deep human need, right? We have [chuckles] our human nature in that we're not robots. We have things that we need, and one of those is to grow. It's just deeply wired in us. And, again, we want to do something meaningful.

If we're not growing and if we have no opportunity to do anything that feels like it really matters...you work on a project that seems trivial, and then it gets dropped. And then, you move on to something else [laughs], and that project gets dropped. And you never actually get anything up that actually accomplishes anything. You're not going to like that job. You're not going to like anything about it. You're going to feel like I just wasted however many years of my life doing something that didn't matter at all, and you leave.

Whereas if you feel like you participated in building something that mattered and you get to see that actually going and affecting your customers, making your customers' lives better, then you're invested. You care. And there's probably other things. But coming into our session today, the things that I thought were most important for culture are those opportunities for personal enrichment and fulfillment in our job.

I think there's also an aspect that Javier touched on, which is having people that you like working with, having those conversations, being able to interact with people and have positive interaction, feeling safe. We've talked a lot on this podcast before about psychological safety, feeling like you can be yourself, like you can be open and be respected; there's another key attribute as well. Those things trump just about anything. If I can go to a job where I am able to grow in my skills consistently, where the things that I do matter, why would I want to leave? It's giving me fulfillment in my life.

JUSTIN: And that pays you something so you can live, but yes.

MIKE: Yes. It is important to have enough money [laughter]. If you decide you're going to undercut all the competition on pay and pay less than everybody, it's probably not going to go well. You have to meet basic standards. But there's research that shows that once people reach the middle class, more money doesn't make that much difference, and this is outside of work, necessarily. But, you know, just in general, being super rich, there's a little bit of a bump as you get more money, but there's a lot of very unhappy rich people [chuckles].

But if you have enough, if you look around and you're more or less on par with other people, approximately equal, that's usually okay. It doesn't have to be the best pay in the industry, as long as it's not the worst. As long as you're making about what's normal for the industry, it's fine.

JORGE: I love to work with international teams. I started about six years ago trying to work with American companies and in international teams. And I love it because all of this stuff that currently we are talking about it's really awesome to know people from other countries. I only have the opportunity to go to Mexico City. I was living, like, four or five months in 2019, nothing else. But I need to visit a lot of other countries.

But I work with other teammates from a lot of other countries like here, for example, Colombian people, American people a lot, people from Europe, even from India. It's a very rich experience, at least for me, because I try to learn a lot from other developers and how they handle or how they solve a problem because they have other way to think and way to do the things, right? So, that's awesome. And, obviously, I'm taking advantage of improving my English skills every day [laughs]. So, it's really good, at least for my career, and I hope to still be getting close with all of these kinds of international teams. It's really good.

JUSTIN: So, I see Will joined. Hey, man. So, we got, like, three remote people. Will, are you also remote 100%?

WILL: Yep. It's been...

JUSTIN: Wow.

WILL: It's challenging, man. It's really, really tough. It's been frustrating. Yeah, I'm not exactly sure where the thread of the conversation has gone. I have personally been impacted a lot with the RTO mandates, and it's been challenging.

JUSTIN: So, I guess my question for you guys is, it's interesting because the company I work for went through a big, huge return to office and five days a week. If they don't have an RTO plan that is done well, you have high attrition. And it is interesting that that done well part...because I've worked in this industry for 20 years, right?

Before COVID, I was almost 100% in the office, and there were parts of the office that were annoying and parts that were fine. But the office, you know, when you're in the office, if you have good leadership and good recognition and you're working on good things and everything, and you have recognition and opportunities for growth, and your commute is not bad and things like that, there's a variety of things that make the office doable.

Like, a good example is today, I, you know, I camped out up in the mountains above here with my wife and one of my kids, and then I came down into the office at 8 o'clock. I had a shower, worked for four hours, and then I took a lunch break and did a workout for my lunch break, and then came back. And I'm, you know, working for another couple of hours. And it's not bad, but at the same time, I'm still working from home one or two days a week, and, for me, that's actually a good balance.

But every single person is different, and every single person has to make that decision for themselves, you know, what they're willing to do to return to office. But it has helped so much if the leadership makes that return to office palatable and, you know, enjoyable even. And they have the right incentives to go back to the office, and not just incentives, but, like, they have the right culture. So, otherwise, I forget what the attrition numbers are sometimes, but they are pretty high.

WILL: Yeah, like the big, long bus ride, like, heavy-duty commutes. A lot of people in big cities are doing serious, serious commutes, and that's a big...it's a big deal. I mean, like, two-hour commutes out on the West Coast is not unheard of, not unheard of, not crazy.

MIKE: Well, I think it's interesting. You talked about it as a perk, and we talked a lot about perks early on and how perks can keep you there for a while [chuckles]. But in the end, the things that matter the most are an opportunity to grow, an opportunity to do something that matters, and enough money. You have to have enough. You don't have to have the best in the industry, but you have to have enough.

So, as a leader, so I’m talking to any leaders who are listening [laughs], if you give the people you're working with an opportunity to grow their skills consistently, make time, schedule time. Say, “Take some time to grow your skills.” Give them projects specifically to give them a chance to grow their skills. Find out what they want to work on. Try to give them those opportunities.

Actively work with the people that you're working [chuckles] with and lead by giving them opportunities, clearing out the way, making their lives easier, and trying to give them an opportunity to grow. Also, give them opportunities to do something that matters. Don't stop somebody in a corner to work on a really long backlog of issues that nobody has reported in five years because it didn't really matter [laughter] that much. There may be some value. Most people probably aren't doing that.

But there's ways that we do that are less blatant than that, where you don't give somebody the support that they need. And you were talking about this before, I think it was you, Justin, talking about when they've got time differences, time zone differences. So, you don't give...the communication is so bad that nobody can actually get anything done because they don't know what it is they're supposed to be working on.

So, really address head-on some of the communication challenges that Will was talking about. You have to. And we've got people all over the world. We've got people here in multiple countries here in this call. If you don't do the work to try to make it easy to work with people who are doing the work...and most of the big companies, I think, are working with people not local.

Now, locally, you might have people in office, but you're going to work with people remote. And if you don't make that experience positive, then you're not going to have a healthy culture because people are going to be frustrated. People are not going to get things done. As a company, it'll be inefficient, and on a personal basis, people will be frustrated. And you won't keep people who want to do good work because they'll get frustrated, and they'll leave.

But on the flip side, there's levers you can pull, right? You can feed the soil. And for those who arrived a little late, we started the call by saying that, in a garden, you don't feed the plants. You can just throw a bunch of chemical fertilizer, and the plants will grow for a while. But as soon as you take away the fertilizer, they die. They go into dramatic decline. Instead, you feed the soil, and then you build a fertile ground where everything just kind of grows without you thinking about it.

Put the work where it really needs to go. Do the work involved to give people opportunities. Know who they are. Know what kind of opportunities would be good for them, and give it to them. And, secondly, build the kind of communication structure and processes that lead to people getting the information that they need so that they know who to talk to. They can talk to them. They have the information they need. You've got things documented as you build them. You've got your processes documented so people can be effective. And that will go so much farther than a bunch of gift cards [laughter].

JUSTIN: I'm trying to think of some specific things that leaders have done for myself or my team that have resulted in growth. And some of the things include, you know, when I've had technical leaders, they've actually been in the code and doing code, you know, writing code and doing code reviews and things like that, spending the time to get to know technically how things go, how things work. And so, that's always been great.

Another one is, you know, it's hard to do if you have multiple, multiple layers, but, you know, the engineering director or the engineering VP them going down to individual engineer level and, like, chatting with them for 15 minutes about themselves and their careers and things like that, where they spend the time getting to know the rank and file. That is something that I've seen be really helpful in terms of, like, encouraging culture growth at a company, and recognizing, you know, asking them, “Oh, you know, what are you working on?” You know, “What's the next step in your career?” You know, “What do you want to do with yourself five years from now?” All those sorts of things.

And it doesn't have to take long. But, you know, spending those 15, 20 minutes to get to know the people who are actually doing the grunt work has resulted in a lot more...a better view of leadership than it not.

MIKE: We've covered quite a bit of ground. We've talked about the importance of actually building, you know, what matters, and allowing people to grow on their own, and kind of used that as the framework we've focused on, about the downsides to thinking that you can just throw perks at the problem and call that culture because it doesn't work in the long term. And we've talked about things specifically that you can do. You know, we've talked a little bit about growth.

You know, I will say, schedule time. One thing I've seen be incredibly effective is just saying, you know, 30 minutes a day or whatever the number is, do something to study and grow your skills. And then, you have to lead by example. Not everybody can do this, but a lot of times your tech lead, your media leads can do that and show they're doing that. And maybe get other people doing it, you know, pair. Pair with people so that they can learn from you.

And there's probably a great number of other things you can do. But it has to be on top of your mind. You have to actually know the people you're working with. Recognize what their needs are and do something to meet those needs. And then, finally, set up the conditions where they can work effectively, where they're not always stuck, where they're not blocked by something. And so, they can, you know, work effectively and get stuff done like we all want to do.

WILL: I'd like to close in saying that extreme programming solves all these problems [laughter], and we shall rise again [laughter].

JUSTIN: I've never regretted time invested in making sure people understand, you know, how to be a better engineer, like, whether that's very specific on a task or a, you know, how things work. It's like an investment in time that will pay off later on. But that's the thing, you've got to spend the time.

MIKE: You've got to spend the time. You can't cheat [laughs]. Great. Well, thank you. Interesting discussion today. Hopefully, you've gotten something from it. And until next time on the Acima Development Podcast.

In this episode of the Acima Development Podcast, Mike kicks things off with a humorous comparison between the DMV and employee onboarding, using the DMV’s predictability as a metaphor for what onboarding should feel like: smooth and well-organized. The team, including interns Chloe and Jordan, along with veteran team members like Will, Matt, and Tim, dives into the realities of onboarding experiences. Chloe and Jordan reflect on the value of strong documentation, access to multiple mentors, and feeling emotionally supported. They emphasize how overwhelming onboarding can be when information overload hits or support is unclear, and how even small gestures, like developer lunches or Slack channels, can make a big difference in building comfort and connection.

The discussion expands to include insights from more senior voices like Will and Matt, who underline the psychological and emotional complexity of onboarding, particularly for seasoned professionals used to excelling. Will points out that new hires, especially mid-career professionals, often face an identity crisis when they’re suddenly inexperienced again, and that trust between leadership and new employees must be earned, not assumed. He stresses the importance of proactive communication, asking questions, and building relationships over time. Meanwhile, Matt emphasizes that leaders must take responsibility for initiating that trust and creating a culture of safety and availability, even if time and organizational bandwidth are constraints.

Finally, the group turns to strategies for remote and global onboarding, with Tim detailing Microsoft’s best-in-class processes that pair automation with strong support systems. The consensus is that technical logistics like IAM provisioning and documentation matter, but they’re not enough on their own. What truly shapes a successful onboarding experience is human connection: pairing new hires with mentors, creating safe spaces for questions, recognizing cultural differences, and setting realistic expectations. The episode closes with a collective realization that while automation can streamline processes, it’s trust, empathy, and communication that ultimately empower new team members to thrive.

Transcript:

MIKE: Hello, and welcome to another episode of the Acima Development Podcast. I'm Mike, and I'm hosting again today. We've got a good crew here today. We've got Tim, Kyle, and we've got a couple of interns who are joining with us today¬¬—Chloe and Jordan, great having you. I’m excited to hear your input. It's very topical for today. We have Justin and Dave, and, finally, Will Archer, the crew here. I think I got everybody. Did I get to you, Kyle? I think I mentioned you. If not...[laughs]

Let's start with an ordeal I'm going to have to go through in the next couple of weeks. So, it's been five years or so since I last renewed my driver's license, so I have to go to the DMV next week [laughs].

JUSTIN: I want to see how you're going to tie this together with new hires. This is going to be really entertaining.

[laughter]

MIKE: People do not look forward to going to the DMV. There's an old song that comes to my mind that says, "I've been to hell. I spell it DMV [laughs]." I think of that every time. I went and looked up the lyrics. There's some other lyrics in that song I'm not going to share [laughs]. I'm going to point out the reference. But it makes a point that I think most of us tend to agree with.

However, when I go to the DMV, I know exactly what to expect because they have done this a million times, right? Maybe literally a million times, maybe more than a million, you know, many millions in a large state. They just do this over and over again. So, they have a routine. I know I walk in there. I'm going to get the number, and then they're going to send me down to a seat and wait for who knows how long [laughs]. They might have one of those little counters to let me know when the number is coming.

At the one DMV I would go to, there's, like, three different desks, so there's actually three different lines [laughs]. You have to know which one you're getting to. But they have signs. They've got tape on the floor that sends you to the right direction. And then when they call you up, they've done this so many times that the people there they don't even, like, see your face anymore [laughs]. They just walk you through the routine.

And it's so standardized because they need to make sure that it always works every single time. And it generally does [chuckles], as long as you didn't forget to bring whatever document you needed to bring, and then they send you to the back of the line or send you home [chuckles], and you have to come back. If you get everything right, it just works because they've totally standardized that process.

Now, it's totally impersonal, and [chuckles] you wait forever sometimes, and nobody likes that. I have heard that they've got, like, some...There are offices in Utah, and I've heard that they've got, like, some express DMV out there that works really well. I've heard some people say some good things about that. I'm seeing some nods out there. Because they’ve cut the process...streamlined so you can go in there and just take care of that part that's fast, and they run you through. If you set up an appointment, you go right through.

So, it actually can be pretty good when you have everything lined up and planned ahead of time. I think that we've all been involved in starting something before that did not go so well [chuckles], when things were not planned, and it can be an absolute disaster. We're going to talk today about onboarding new employees.

And I bring up the DMV because if you have everything lined up, you might not enjoy it, but it'll be relatively fast. You're probably not going to be there for a week [chuckles], and they will take care of your needs, and you will leave with your business taken care of. And sometimes when we onboard new employees, it does not go like that at all. I've seen some horror stories where people did not get their computer for a month [laughs], and...I’ll work someday, you know [laughs], and nothing goes through. It can be a total disaster. And somehow, we've been doing this for however many decades we've been doing it in software, and it still takes...it's still hard. It's still hard.

That's what we're going to talk about today. We talked about onboarding, actually, about a year ago. You can go back, and I think it was July of 2024. We talked some about this. And that time, we focused a lot on the mentoring aspects of it. Today I thought we'd talk a little bit more...to have a little more opportunity for horror stories [laughs] we’ve seen. And we've got some people who've recently onboarded. We can talk about the good and the bad. And we’re going to talk about a little bit more of the hands-on, nuts and bolts. How do we make this work?

So, I would like to start by asking our recent hires here [chuckles], so Chloe and Jordan, what went well, and what did not go well about your onboarding process?

CHLOE: I can start. I feel like one thing that went really well is there was a lot of documentation, so a lot of really good resources to get help when it came to onboarding, as well as I was paired with somebody who has recently onboarded just a few months prior. And so, they had a lot of really fresh experience as well as advice when it came to onboarding because they had just done it, so they had already kind of figured out a lot of the things that could go wrong or any problems that I might have encountered.

Something that was a little bit more challenging was, I think, when you are onboarding, it just feels like you are in a pool of water, and there's these big waves. And it's really hard to feel like you can catch a breath and feel like you're understanding everything, and that's common to a lot of places. But it feels like there's so much to learn, and you always feel behind, which is a tough feeling to feel when you're coming into a new company.

MIKE: Great. Thank you. Jordan, any thoughts from you?

JORDAN: Yeah, so this is, like, something that is kind of obvious, but I think good documentation is a must. And this isn't any fault of the company, but maybe of my own from last year. Since we started the project, there's very minimal documentation for it [laughs], and it's simple enough where you can get it figured out. But I think that was a little bit of a struggle trying to remember what I did and asking some people, like, “Did I do this right?” So, there's that.

But something I thought was helpful was having some early low-hanging fruit tickets that got ready for us, basically. He had chosen them and said, “These are good tickets for you guys to kind of remember what you're doing or learn how to do things again,” so that was very nice.

MIKE: Nice. Well, thank you. Now, we've talked to you all who just recently started at Acima. But we also have some others here on the call. Will has recently started a new position, so I think he's got some fresh onboarding stories in mind as well, and I think some others as well. So, go ahead, Will.

WILL: Well, I've onboarded a lot, and I think, like, oh gosh, like, I'm getting through it. I mean, part of it, like, the biggest thing, I think, to keep in mind is you're just going to have to embrace the suck. You know, like, it's going to be bad, and it's always going to be bad. There's no comfortable way, I think, for, like, ordinarily high-achieving people who are used to not looking like an idiot to be real dumb and real helpless and completely ignorant for a period of weeks, if not months. There's just no...you know what I mean? Like, there's not a lot of, like, C minus students [laughs] getting onboarded, but, like, you're just going to have to take...you're going to have to take a bunch of Ls.

And I’ve sort of, like, as I look...so, I mean, like, having done it, I always find the process really, really rewarding when I'm through it. But I have to remind myself, like, “This isn't supposed to feel good [laughs]. This isn't supposed to feel good,” at every point there. And so, if I'm reminding myself, like, as I'm through it...because, like, it's me and a buddy. We both came in here. We got recruited at the same time, and sort of, like, we're both sort of our support group, right? Because, like, I'm extremely old, and I get hired on because people expect me to come in, you know, plug and play. Like, I'm expected to produce.

I'm not an intern, and there is, I mean, like, I think everybody is understanding, but, like, there is a certain level of expectation that I have to, like, get things done. And so, like, me and my buddy are just sort of...we're constantly reminding each other: you need to be vocal. You need to be out there, like, asking questions. You need to be bringing people in. You need to be sort of getting on pairs. You need to be proactively searching out documentation and following it and not being surprised when it's wrong because it's frequently wrong.

And you need to be aware of asking questions in the right way, where, like, if you're asking a question, you need to be doing the work. You need to be visibly working harder than whoever you're asking a question of. You need to be sort of, like, cognizant of chain of command, whereby you go up to your team lead, and then you go up to your manager, and maybe you go up to, like, a staff engineer, a principal or whoever, and then, like, if you've got to go...if you have to go to a director, you can go to a director, and I will.

But I'm not going to go there without the checklist filled out where it's, like, I would do this and this and this and this and this, and this led me to you. This is what I'm trying to do. This is the problem, and this is what I need you to sign off on, right? And then, as you do that, eventually, the sun will come out, you know, come out through the clouds, and you can start getting things done, and you're not going to piss too many people off in the process. I don't know. Sorry. It was a vague prompt, and I'm trying to distill, like, you know, what to do, right [chuckles]?

MIKE: Will, you and Chloe both pointed out something that I think is really interesting. You both talked about from the perspective of somebody who's being onboarded and said it's hard, and Chloe swimming against those waves [chuckles]. And you talked about, you know, you're just going to feel dumb, and that is hard. I’ve worked with --

WILL: You're going to be dumb [laughs]. It's not a feeling. You suck [laughs].

MIKE: Well, I've worked with...I've seen a number of people who switch careers later on, and some of them really, really struggle because they were successful before and now they're not, right [chuckles]?

WILL: Yeah.

MIKE: They're the noob, you know, they don't know what they're doing. And they are making mistakes, and they can't figure things out. And they feel like everybody knows this better than them. And that is not a good feeling. But you have to just embrace that for way longer than is comfortable if you want to be successful. And you have to say, okay, yeah, I'm back in high school. Maybe I'm back in junior high [laughs], and I just have to live that role again.

WILL: Well, man, I think...I don't know. I mean, this is maybe, like...this is not an intern problem. Like, you guys have been dumb recently enough that, like [laughter], you can still remember the feeling and cope with it. But, like, I'm 45 years old, and I've been good at my job for a very long time. And, like, imagine, like, going back to school, going back to high school and, like, not doing great [laughter]. But I feel like that's a trap. I feel like it's a trap that later on in your career you can get trapped in, later on in your life, you know.

Like, you could get into your mid-40s and be like, I always wanted to learn, I don't know, basket weaving or, like...you know what I mean? But, like, I'm so good at everything else in my life. If I go and do this thing, you know, I wanted to learn ballroom dancing and, like, I'm bad at it. And there's no skipping over that first...that first rung of the ladder. You're just going to be bad at it. So, you're going to either figure out a way to, like, embrace stupidity later on in life, or you're going to stagnate, and that will just be...you'll just be stuck forever. And I think that's --

MATT: We've all gone through it, right? And we still do, regardless of our level. You talked about going up through leads, managers, staff, principals, directors. I'm currently a director with a company.

WILL: I'm sorry.

MATT: And I still go through it. We don't always know everything, and we're always going to need some help. Part of it is just accepting that and understanding that everybody goes through it. Other people go through it, and more likely than not, they're going to be willing to help you through it. So, go in with that mindset, and I think you'll do all right.

I don't require someone to go through all of the chains of command to come talk to me, at all. If it's something really trivial that they could have leaned over to the guy next to them or the girl next to them and got an answer really quick, then maybe I'll say, “Hey, did you ask these guys at all?” But door's always open. You need help --

WILL: Yeah, but they don't know that, Matt. They don't know that. They don't know you. They don't know who you are.

MATT: Yeah, it's my responsibility to make that clear, right, and, hopefully, I have with Jordan and Chloe, you know, as I’ve talked to them over the time they’ve been here but --

WILL: I’m going to lean back on that one, Matt, because it cannot be done. Like, what you’re talking about, you could say that, and you could say it clearly and effectively, and you could say it with clarity and confidence. You can say that in that sentence, right? But, like, you cannot establish that level of trust in a 15-minute getting-to-know-you meeting. Like, that has to be grown over time.

You can't just be like, “Listen, my door is always open.” And I'm more open, and I'd say, like...you know what I mean, like, most critically, I think virtually everybody, you know, at a director level or VP level, like, if you reach out to them and you need help and they can help you, they'll help you, but you got to get on their calendar. And the reason you go through the chain of command is, like, A, you know what I mean, like, yeah, don't waste your director's time; don't waste your VP's time. But also, B, you know, that pyramid gets pretty narrow towards the top, like, you know, you can't be on your calendar just at a whim. But your team lead, yeah, man, if I need five minutes from my team lead, yeah, they better give it to me, you know [laughs], like --

MATT: Yeah, and that's fair. That's fair. Calendar can certainly be tough, you know, it's generally booked all the time. But the way you earn that trust is, try me. You know, you have something you need to talk to me about; you want career advice; you want software advice, try me. And if I don't give you that time, then that trust isn't earned, right, but if I do, then we can establish that rapport. And, yes, it does take a little time to build it, but that's how you do it.

WILL: I don't know, man. I think you have people for that reason because that trust is something that is built over time. You don't have...you can only...what is it, the Dunbar's number? You know what I mean? Like, how many relationships can you maintain? Well, you got to have...that's why you've got people to build that trust among, you know, people that you can't meet with every day or every month, really, you know? It just can't be done.

MATT: And that's correct. You know, it's much easier as you're down the pyramid because the bandwidth is there, right? But ultimately, trust comes from the top. If I can’t trust my leaders, then I don't want to be wherever it is I am.

MIKE: There's something there to be said about the leaders making themselves genuinely, like, doing something to make themselves a little bit vulnerable, like, oh, hey, I can trust you. You know, the thing that, you know, the dog rolls over on its back, shows its belly. And, like, oh, I could hurt you, but I'm not going to. It's giving the opportunity to be hurt so that you know that, hey, I'm here, and I'm not here to hurt you. And the person in that role, in that leadership role has to do something like that, reveal a little bit of vulnerability, and that's, you know, that's just part of relationships.

They still have limited time. That calendar is still going to be limited, no matter how much they do that, and that's a challenge. And you're a new person. You're that new person. You say, “Okay, here's somebody I can go talk to three weeks from now on Tuesday at 8:00 a.m. [laughs],” and that's a problem. So, you're going to have to be assertive with the people closer to you.

MATT: Somebody --

MIKE: Go ahead.

MATT: Yeah, somebody once said to me something that really struck a chord, and that is, we all have time; it's, do I have time for you?

DAVE: Don't say, “I don't have enough time.” Say, “I have too much to do.” Because you can't give yourself more time, but you can reduce the things you've committed to.

MATT: Yes, you can rearrange your schedule because we have time. We have 24 hours in a day, every day. So, it's priority, right? So, am I going to make you a priority? Is my superior going to make me a priority? And that's something that I think we all need to try to do. Sometimes it's not possible because there's other commitments and other people are relying on us, but we can make time.

WILL: I mean, the reason I'm pushing back on this is because you can't. It's a zero-sum game. You can burn yourself out. You can spread yourself too thin, you know? But I think the reason I'm like, no, you can't command trust; you can't command a relationship; you can grow the relationship, but your bandwidth is limited, and you're only going to be able to maintain so many relationships. If you don't respect that process, then you'll find yourself in a trap.

We’ve diverged really badly from onboarding, but I do think there's a point that needs to be made, where you'll go to somebody and you'll say, "Listen, my door's always open. I need feedback. If something's going bad with this project, I need you to let me know. I need reality. I need this relationship to be strong." But then you say that, and you mean it. It's not that you don't mean it, but because you haven't grown that trust, that relationship isn't actually there. And you rely on it, but it isn't actually there.

And then, you can get these really weird communication mismatches. And you can get these really weird things because you said, "Do this thing," and they said, "Sure, boss." But it isn't there, but you think it's there, and you rely on it, but it isn't. It's ephemeral. You have to respect the need for cultivating those things and the limits around what you can and can't do.

DAVE: I think you guys are both right. I'm hearing something interesting from both of you guys. What I'm hearing from Will, and I think you're right, that there's a boundary. You can teach me something, but you can't understand it for me. You can offer psychological safeties, but you can't make me trust you.

But I think Matt is right that you have to get up and go offer it. If you're the training or the onboarder, it behooves you to actually make an explicit, “I have to go do this. I have to go talk to you and let you know that my door...I have to tell you what the culture is, unless I'm assuming that you're weapons grade good at reading rooms,” which most of us aren't. So, I think you guys have both got the right...you can only go up to your own boundary. And so, what Matt is saying, you've got to charge your boundary if you're a leader. You've got to get right up to their boundary. And I think Will's right that you can't go over it. You can't make them take it.

MATT: I totally agree with that.

DAVE: I have an illustration for that, but I figured I would yield the floor.

MATT: There's responsibility and accountability on both sides, right, for --?

DAVE: 100%.

WILL: Kind of, sort of, but not equally divided. Like, if you're in charge, then you're in charge, you know?

DAVE: Scope of power.

WILL: If that reporting relationship is not good and you're a people manager and managing relationships is your primary function or role, then, like, yeah, that's on you.

DAVE: You’re bad at your job.

WILL: It's not 100% you, but it's 80-20, and besides, you know, if they can't do it, like, you hired them, so what? But I'll say this, right? In the context of onboarding, right, because, like, this is something that I don't want to talk out of school. But there are some people who are having a hard time integrating organizationally. And we're bumping up against this sort of, like, lack of trust, which must be cultivated in the chain of command because they need, let's say, resources. I'm trying to be really vague, right, because it's not my story to tell. But they need resources out of the organization. They need, you know, engineering allocations, and they're not getting that, right?

Like, they're like, “Hey, I need access to this. I need explanations of this. I need a...” you know what I mean? “Endpoint opened over here. I need, like, a security review here because, like, I got a hot deadline,” and they're not getting that stuff. And they need to go up that chain of command, and they're running into that lack of trust. Somebody that I know who's also onboarding, and they've been talking to me, right, because, like, we're homies, and we started together. And we're sort of, like, blind men in the dark trying to figure out, like, how this org chart actually operates.

And, you know, I'm just like, listen, I'm just going to email the director because I am a door kicker by nature. And I know that if I go in with good intentions, I will be forgiven even if I do the wrong thing [chuckles], you know? Like, yeah, I've been running my mouth for a long time, and, like, occasionally, I screw up, but I'm always forgiven because I'm not...you know what I mean? Like, I'm not out here, you know, trying to, like, I don't know, take from the organization. I'm just like, you told me to get this thing done, and I'm running into this wall. I’m stuck.

But, like, this is exactly...like, this relationship of trust has yet to grow, and sort of this guy's like, I don't know, “What do I do? What do I do? Nobody's getting back to me. Nobody's returning my emails. I'm stuck.” And not everybody is a door kicker. It's for the best that most people don't think and act like I do. [chuckles] Society kind of depends on it.

[laughter]

DAVE: We can't all be fuel rods. Society would go nuclear. We need control rods between us.

WILL: All gas, no brakes, is not the way to run a functional engineering organization.

DAVE: Yep. All kite, no string. Yep.

MIKE: But you talked about that person getting stuck. You know, I gave examples before of some people who... switching careers later. And I’ve seen people be successful, too. So, like, I’ve seen it go both ways. And the people who are successful are the people who find themselves stuck, and then they go and they ask somebody. And if they don’t get an answer there, they go and they ask somebody else. It’s the people who are willing to do that that are successful because it’s going to happen. You are going to get stuck. It’s inevitable.

DAVE: There’s a really powerful bit of self-help going around the internet these days. I mean, it’s been going around forever, but it’s popped up on my feed, which is that you are responsible for you, right? Nobody’s coming to save you. You are here to save yourself. And yes, it’s wonderful when you’ve got co-workers and managers who will come help you, but this is just your job. You are in charge of your career, and nobody else is in charge of your career.

There are times when getting onboarded or getting the knowledge you need is going to be tough. It’s going to suck. It’s going to be hard. And what you have to do is sit down and go, well, when I started this career, did I think this career would be easier? Did I think it would be hard? No, I thought it would be hard. Okay, well, this is what hard feels like. And that can kind of help you light a fire underneath your breeches without burning your pants off. That’s a weird metaphor [laughter]. But you get what I mean. You’ve got to get up.

And so, it’s like, if you are in that 20% of the 80-20 and you’re not getting that 80, it behooves you to master your 20, to own it, try and get it to 21. And that’s the kind of thing. It might not save your job, but it will absolutely save your career. It’s a tiny, little percentage of investment that pays off over time is take the initiative and go get it. Because if you’ve got a manager like Matt who’s like, "Door’s always open," and Matt has an employee that’s like Will that will come kick the door in and say, "Hey, you closed your door, and I need it open because we need to talk," that’s going to be a good relationship. And it only takes one and a half of you to make that relationship work.

JUSTIN: So, having said, like, just to pivot a little bit, you know, you are responsible for yourself, but as leaders, you know, we’re trying to onboard these folks so that they’re most successful. And I’m bringing this up because, starting next week, I’m going to be onboarding some folks that are very remote. So, it’s, you know [laughter], I want to do what I can to make sure that they’re successful. And we’ve had a couple, you know, I’ve onboarded other folks onto my team, and, you know, so I have a process and everything. But, you know, the very remote is always difficult. What have you guys run into, like, when you’re onboarding folks that, like, you’re not sitting next to, you know, or you’re not even in the same time zone or even in the same hemisphere?

TIM: So, I will steal from what I learned from Microsoft. So, Microsoft, they scale instantly overnight, over a week. They’ll hire offshore engineers for a sprint, and then they’re done right after that sprint for two weeks.

WILL: Woow.

TIM: Or they will hire someone for an epic, which is three months, and then they’re done after that point. So, they have to get their shit figured out. By day two, everything is done and running. So, how do they do that? So, first is, the first step recipe for them is IAM automation. It’s all Entra ID, all automatically driven through Entra ID. And you plug in your own tool, whether it’s Okta, Ping, whatever you’re doing there. That’s the first step: IAM provisioning and automation.

Second is engineering environment setup. Microsoft, since 2014, has had this really cool tool called 1ES, or One Engineering System. And it does exactly what Will was saying in the kind of pre-podcast is, how do I standardize an engineer’s developer platform? That’s what Microsoft does. The One Engineering System automatically gives engineers what they need, and that image, so to say, is built and maintained and automated through their tech leads.

Second is the corporate and security, the stuff that we always overlook that we do within, like, the first four hours, but it’s, you know, it’s really important, kind of compliance policies and conditional access, and signing all the junk.

And then, they have what they call an onboarding buddy. So, they have, like, a 30-60-90-day plan. Someone is always there to answer questions, to help escalate issues. They’re setting goals for those 30-60-90s, and they have what they call buddy KPIs. So, what are you going to be trying to achieve within that period of time? Because engineers thrive on getting stuff from the to-do to the done column. And if it feels like you’re getting stuff done, you still will want to work there.

After that, it’s just like what Chloe and Jordan were saying: documentation and internal wikis. Holy cow, dude. We need it so bad. When I started, my manager said, "Okay, here’s everything that we’ve got." And he just showed me this torrential flood of mismanaged SharePoint folders and Confluence pages that were just all over the place. I’m like, how in the world am I supposed to divine what you want me to learn out of all of this? No idea. So, a well-crafted wiki system is so important.

And then, finally, the analytics and feedback loop, you know, the managers, the HR team, the recruiters, everyone is going to ask you after that 30-60-90 window, “Are we hitting the mark? What are you missing?” Stuff like that. I have never seen an organization do onboarding better, faster, and more efficiently than Microsoft. Those guys are so good at it. But again, that’s just what I’ve been exposed to. I’m sure someone else is probably better at it than them.

MATT: There is one key piece of this missing with Microsoft and how they do it, and I happen to know who their partner is. They also have dedicated teams.

TIM: Yes.

MATT: So, if they need to spin up a team, that team has already worked with them and familiar with their domain and ecosystem, which is why they can move so quickly, and that’s key. However, all of those other things are good, and, you know, everyone should be doing it. But the instant onboarding, that’s a key piece, is those teams are dedicated to Microsoft and only working for Microsoft when they get spun up.

TIM: Because the money’s there, right? They can afford that.

[laughter]

MIKE: You just pointed...That’s a fascinating thing you pointed out there, that the team was prepared to join. You know, interesting, some years ago, when I say some years ago, like, 30 years ago, maybe more than 30 years ago [chuckles], there was a movie that came out. What was it called? Stand and Deliver, maybe, about a teacher who took his inner-city school math class and helped them all pass the calculus AP test. And it was this inspirational movie. Everybody watched it.

And I’ve read some of the back story about that, and they left out...and it’s true; there was this inspirational teacher based on a real person who got his class up there. But they left out something really important. This math teacher had been working with all the teachers in the junior high and in the previous levels in the high school for years, prepping them to teach those students. So, by the time they got to his class, they were already ready, and if you leave out that part, it doesn’t work.

There’s some baseline preparation that you have to have. And it is an inspirational story, but they left out the best part is that he did the work. If you’re going to make this work, you have to do the prep work and lay the foundation. Microsoft, it sounds like they can make this work because they have partners that already know, you know, they’ve got people who already kind of know what’s going on that --

WILL: It sounds a lot. I mean, I love the system, right? Like, they did all the stuff that I just dreamed of, right? I think that’s so cool [laughter]. But at the same time, it sounds a lot closer to, like, an internal transfer than onboarding a new hire, right? It sounds like these are, like, "We’re going to put these assets that have worked with us a bunch and know our systems and stuff like that," and like...Like, oh man, I was twisting arms for a week, literally, to get access to all the Git repos. I was chasing people down.

I mean, in terms of, like, I mean, we can call this directly, right? You’re talking about onboarding India hires, right, or people from Asia, where you’re on the other side of the world, right? I mean, the biggest thing that I have seen people really trip up with, like, these cross-time-zone hires, right? Because it doesn’t really matter. It doesn’t matter so much, like, cross-country, but, like, cross-time-zone, you put people on an island where they don’t have real-time support.

It’s like, I can’t work like that, or, more to the point, I can’t work effectively and efficiently like that, and it’s profoundly demoralizing. Like, if your first two weeks are just sort of, like, sitting there like, [vocalization], I work 15 minutes, stop, wait eight hours for people to wake up, right, like, that’s brutal. I don’t know. I mean, like, I’m pretty good at my job, but I’m not that good. And it’s profoundly demoralizing. And it’s sort of, like, I don’t know. I think it establishes a relationship where, like, you’re kind of, like, you know, you know what I mean? And then, if you treat people like that, then they produce like that.

So, I mean, the thing is, if I could do anything, I mean, and this is not, you know, necessarily good news, but, like, people are going to lose some sleep, you know? Them and you, you know, because, like, you just got to have a block of time where, like, you’re just getting them up and running, you know?

MIKE:  Exactly.

JUSTIN: I really like that. It's like, you can prep all you want, but you got to invest in them for them to be successful. And all the time that you invest in them it's definitely going to cost you something. And you may have to adjust your hours for a while, but it pays off long term. And, you know, whether that's on the front end where you are prepping all the documents and prepping all the training videos and things like that, or it's on your own time where you are, you know, walking them through the application and answering all their questions, and debugging an issue, and walking with them, pair programming through their first PR, you know, all of that is well worth the time to me because, you know, that pair programming time is, you know, it takes up your time, but it enables them to be much more successful.

WILL: Well, I mean, like, a broader thing, but, I mean, like, in all honesty, you know, like, you never really get out of it, right? I mean, like, you could never, you know, I mean, like, if you're going to have somebody on the other side of the world, like, okay, we all know why we do that. But, like, you know, that relationship still has to be cultivated.

Honestly, I've been in a lot of places where they didn't really...I don't know, they're a human being, and they're a human being just like you. And they have the same feelings that you have, and they have the same everything. They're just as smart as you, and if you treat them like an equal, they'll produce for you. And I've seen so many places that just completely...they completely bungle that relationship, and then, you know, everybody knows the horror stories, and, like, I've seen it over and over and over and over and over again.

Because, like, when you, yeah, you're saving a bunch of money, and that's fine, but, like, you're also taking on managing this relationship across this very challenging set of circumstances. And, like, if you're just sort of, like, just trying to save a buck, and you don't want to treat people fundamentally with the exact same amount of respect that you would expect somebody to treat you with, then, like, you know, you're going to get what you get.

MIKE: Well, and you don't save money either.

WILL: No, no, no. No, you do not.

MIKE: Do you know how expensive it is to hire a huge team of people that you then completely ignore and can't accomplish anything for months at a time [laughs]?

WILL: Oh my God.

MIKE: You develop a culture where nobody cares whatsoever because they're not paid attention to. It doesn't matter what they do.

[crosstalk 39:09]

DAVE: They care. They care. They're just caring about the incentives.

WILL: I don’t think [inaudible 39:12]. It ain't your father's India offshore. Like, the developer market in India is competitive these days. If you treat people like a dog, they're going to bail, and if they're good, Microsoft has a shop in India, and so does Google, and so does Facebook. And there are bigger fish than you swimming in this pond, and, like, good people are not any easier to find there than here. Your good people, like, they'll split.

Like, I was surprised at how competitive some place that I was working with before. They were spinning up in an India office, and, like, we had some guys that were like, yeah, hell yeah, like, yeah, this is...okay. All right, I'm [inaudible 40:00]. It sharpened my game up a little bit. This guy is giving me a run for my money. Like, oh, he got a job with Netflix. Bye.

MIKE: You know, I love what you said about treating people as the human beings they are. You can't have an effective relationship without a relationship. It’s not going to work. And I've seen it be very successful working with teams from all over the world. And it's harder the farther apart the time zones are. It absolutely gets harder. Every hour you add to there is another level of difficulty. But if you put in that overlap, are willing to make it work, take that time, it can become a well-oiled machine. And you can have great success with very talented people from wherever.

KYLE: I think, too, one thing that I've had to learn is cultural differences with these distributed teams, right? Yeah, it's easier. Time frame does make a difference. But also, the culture just really affects, like, how you're talking to them and how you might be understanding why they might be stuck or something, right? Because I've worked with teams in, you know, India or in Poland, and they're closer in time zones. But it's definitely different hurdles.

WILL: Oh, man, I love an Asian-European team. If you ever wanted it straight, man, they'll give it to you.

[laughter]

MATT: It's true. We work with some good ones.

WILL: Yeah, wear your helmet, but, like, you'll definitely get that feedback.

[laughter]

MIKE: The cultural differences are a real thing. In some cultures, I have noticed this...we're talking some about India. It's probably not universal, but there is often a hesitance, I found, from some people to speak up, to ask questions because they feel like, oh, I shouldn't be asking people. I should just know this. And it takes some extra work, and good people will recognize that, read the room, like, okay, they're wanting to talk to me, extra work to reach out and say, “Hey...” make it clear that you want them to ask. Make yourself available. Show that vulnerability. Let them know that you're willing to have that relationship. I think that's really important.

TIM: What I learned...and this is coming from someone from Bangladesh who told me, you know, every time you work with a team offshore in these areas, always explain what you want people to do. And it's customary or culturally acceptable for the person who is listening to you to constantly say, “Yes, yes, yes,” to be an active listener. But over at Stateside, when someone says yes like that, it's generally an acknowledgment that I understand what you want me to do, and I will do that. But over there, it's just kind of that unconscious that's how we proceed the conversation forward, not necessarily that --

So, his feedback was, at the end of describing what the task is, ask that individual to repeat it back to you in their own words so that you know the translation, the communication process has occurred successfully. Otherwise, it's almost a surefire recipe for miscommunication. To be honest, that doesn't even have to be, like, an Americanism to another culture. That's probably just good practice for any kind of communication.

MATT: Yeah, that's actually a really good observation, and I haven't thought much about that, but I think it's probably correct. You know, we've been doing a lot of talking, but I'd be really interested to hear some perspective from Chloe and Jordan. A lot of us are leaders that have been talking, and as you guys are coming into this [laughter] career and going through the process, I'd love to hear what your thoughts are. What are some things maybe we're missing that we could help you with?

CHLOE: I can share. One of the most helpful things that's happened since I've been here was, like, very early on, there was a developer lunch. And so, we all kind of were able to go out, have lunch together, and get to know one another. And I was able to get to know one of the team leads (Later that day, we were going to shadow), and just had a very normal conversation. He likes to play video games. My husband plays a ton of video games. We talked about that.

And so, when I went to go shadow him earlier, I felt so much more comfortable asking questions because it wasn't like, this is a team lead that I don't know. It was, oh, this is a person. Because I think, as you guys are talking about how people that we're hiring are people, we should treat them like that, but so are our managers, the leaders in the company. They're also people. And so, I think having that experience made me feel so much more comfortable asking questions, and it didn't feel like the questions were unwanted.

And I think as well with that is, as someone new, I have so many questions, and it's not fair to put that all on one person. Even if they're so well-meaning and they want to answer the questions, they probably don't have the time. So, creating an opportunity where there is a wide range of people to ask questions to helps each of those people not feel overwhelmed. That also helps me not feel like I'm the annoying one asking this one person questions all the time, and also, I get to create more relationships. And so, those are some things that have been helpful, but I think just continuing to have a pool of people for me to go to or for new hires to go to is extremely helpful.

MATT: Do you think, and this is something that we haven't historically done, but what you just said made me think it might be a good idea, do you think something like a roundtable with current employees and full-time staffers would be helpful?

CHLOE: It depends on the nature of the roundtable, but I think, yeah, it could be very helpful of just kind of giving some of that feedback, but also just creating an environment where questions are open, and you can get to know a lot of people at one time.

MIKE: Well, so one thing we did with the interns this year is we identified a group of people, and we're deliberately having them rotate through a pool of people rather than just dropping them on one person like, hey, “Here's your job [chuckles]. Help these people,” because then they have split loyalties, and they don't have time. But having a pool of people they can work with, we very deliberately set that up so they could have a group of people to ask questions to.

I know I'm going to be talking to this lead tomorrow, and this lead the next day, and so on, and then that lead can assign somebody else on their team. So, there's an intent to give structure, to give them lots of opportunities to meet with that group of people and have many people to ask and not just one. I don't know if that's been useful or not. So, I'm curious for feedback on that.

Before I ask, though, one other thing that Chloe and Jordan have done is they've compiled lists of questions, and that was awesome. I've already mentioned that to them. Multiple times...I've been working closely with them this year. I've received a list of questions in a document. Can you answer this, this, this, this, this, and this? We've already explored the options, and now I've got these questions. And that is so helpful because it's a little bit asynchronous. I can go through it when I get a chance. These are well thought out, and it's documented. I can put the answers there. Being willing to get those lists of questions together was actually a really helpful thing.

MATT: Also, something we could share with future internships, so...

MIKE: Absolutely.

MATT: Really valuable.

MIKE: So, back to the question. I'm curious if having the pool of people was helpful. Maybe it wasn't. Maybe it was a failed idea. And I'm also interested in your thoughts, Jordan, because Chloe has shared her thoughts, but you haven't this time. You shared at the beginning, but coming back --

JORDAN: I kind of cheat a little bit because this isn't my first time here, so I know some people.

MIKE: That’s true.

JORDAN: But I think just having a pool of people that you can go to for questions is phenomenally helpful. For example, just onboarding in general, I had multiple people I could ask for different services and all that, but I think being able to know or meet someone is great. And so, I think on the subject of having a list of people that we can go to every day, I think it's very valuable. Having one mentor is nice, and they'll have lots of context on what you're doing and be able to help you very quickly, but that is to the detriment of them not having any time for themselves. So, I think it is valuable to have multiple or even just having someone to help you, so...

TIM: To Chloe and Jordan, I'm curious, would it be more valuable to have a buddy that you can ask for help when you need it? Like, Jordan, you just described, it's valuable, but it could be to their detriment. Or would it be more valuable to be plugged into a Slack channel or a Teams channel where you have onboarding specialists, people who know the questions you're going to ask, and it's a safe space for you to ask stupid questions like, “Hey, I can't get to this. How do I get to this?” Or “I need to submit a ServiceNow ticket. Who do I assign this kind of ticket to?” Which would be more valuable as a newly employed person?

JORDAN: Ideally, both [laughs]. And I say this because having one person that you can ask quick questions to, so you don't have to wait for a random small thing for, I don't know, a couple of hours, is very nice. But, at the same time, being able to document the questions, like the intern chat, was so helpful this year. Like, I came across some personal blockers that, coincidentally, someone else had experienced last year, and I could just go back and read the thread. And if I'm missing some context, I could just ask very quickly to get up to speed. But I think having both a safe space and someone, or people, to ask very quick, simple questions to is great.

TIM: Like the Stack Overflow of onboarding. There's a place you can look for commonly asked questions. Yeah, that's good. I'll be honest, I've been here with Acima for, what, nine months or something like that, and I still will run into things like, where do I send this stupid ticket to? And I debate. I sit there for, like, three minutes. Like, do I put this in general engineering and ask and look like that one guy that can't seem to find that one Confluence page? Or do I just bug someone who's already being bugged [laughs]?

WILL: I love how we keep on looping back to that psychological safety, where it's like, well, I feel bad because I have one person that's onboarding me, even though you have one person that has been specifically selected and tasked to onboard you. And it's like, oh, but I'm asking them so many questions, right? And it feels uncomfortable.

It's totally...like, my point there is not, like...my point there is just, like, this psychological safety and these relationships and bonds and trust and integrating into a community, like, that psychological safety, like, it's so critical. And that is just something that has to be grown. And, as leaders, as people who are already embedded in the community, if you leave anybody with anything, it's to keep that at the top of your mind, and, like, growing that, and, like, being cognizant and aware of its lack, of its absence, right? Because, like, people just...everybody, right?

I mean, you have, like, people who are really senior, who've been doing this for decades, and we're still talking about this stuff, right? Like, me, like, I'm going through it the same way as you. And I suppose, like, the biggest difference is not in how I feel about it, but, like, that I just know that if I do the right things, I'll get the right results. So, I'm okay with, like, my trust fall, you know, into the organization.

MATT: Someone who's really, really good at providing that for people happens to be on this call, and that's Mike. People just feel safe working with Mike. And, you know, I used to report to Mike, and immediately, I just felt that way with him. Because he's there; he's reaching out, and he's making you feel that comfort. And I think, you know, as leaders, we need to be cognizant of that and take an example, right?

And as new people coming on...and it's one of the hardest things to overcome. It really is. That's why a lot of people struggle with paired programming, right? It's the ego, and I don't want to feel dumb. And what are they going to think if I mess up? We need to just accept that we're all human beings. We all mess up, and it's okay. And it's okay to ask those questions, even if it may feel dumb, you know, you're asking it for a reason.

CHLOE: I think I'd really just like to echo that sentiment. I think, coming into this internship, I felt, you know, very overwhelmed because it was in a different language and never had this kind of experience. And one of the first things that Mike said was, “You're not supposed to know everything.” And so, I think that took off the weight.

I think when you're interviewing, you have this pressure of, I need to prove myself that I'm worthy for this position. And then, when you get into the position, I think sometimes we still have that mindset of, I'm still proving myself. But you've got the position, and I think it's an opportunity for you to be, like, it's okay if I don't know everything. I got the position. I'm here. Now it's time for me to learn. And so, I think Mike did a really great job of setting that as the expectation was, I'm not going to know, but I'm here to learn. And I think if we can do that for a lot of people, I think that would help.

KYLE: I would wonder, Mike, you're saying that we've put the interns with a bucket of leaders, or people that they can cross-train with. Have auxiliary teams been included in this? Because do they feel comfortable going to IT, DevOps, security?

MIKE: We have put them with data, but we have not put them with some of the other teams yet. Although I'd love, actually, to have them rotate through some of those other teams. I think that'd be really helpful.

KYLE: Because, yeah, I wouldn't want them to be, you know, I'm on DevOps, and Tim over here is on security. I wouldn't want them to feel timid to reach out to either of us for anything, too.

JORDAN: You know, I think something that kind of happened as a side effect was that we were seated near data and IT, and IT, they're nice, and so they would talk to us. And I think as a result of being able to sit close to them, I'm more willing to go up and talk to them, so I just wanted to point that out.

WILL: [inaudible 55:20] is not all bad [laughs]. Mike wouldn't know [laughter].

MIKE: So, we've talked about a lot of things, and we've been here for a little while. We've talked some from the psychological safety aspect from the leaders. We've also talked a lot about your role as a person being onboarded, that you're going to have to assert yourself a bit. You're going to have to find that way, and you're going to feel stupid [laughs]. That's the reality of doing something new. And that also speaks to leaders. Well, recognize that you've got a bunch of people who are feeling that way, and try to give them the support they need in that very uncomfortable situation, and try to help them get through that quickly.

TIM: I think what's interesting the most out of this conversation, for me, and it speaks to kind of being a little myopic to my role. But every day I fight challenges with automating a lot of the IAM mechanisms of onboarding people, or fighting, how do we get the engineering environment set up, up and running?

But I swear, I think 80% of this conversation has been the soft skill side, the aspect of onboarding, not so much, okay, we get a new employee, and I have to populate a million attributes about this person in our directory, and try to build an automation so that if they have this job title, they automatically get these roles, and stuff like that. That's where my head is at all the time because, in my mind, that is a successful onboarding, that on day one, they have all the roles they need. But when you talk to, I think, employees at large, they're kind of more like, I actually care more about a buddy that's there for me. And I think that's absolutely true.

We hired a person for our team, and it was just to his virtue that I had gone through the same wall that he had months before, and I just had a list of everything that needed to get done because I had just done that. And I think that experience of him being able to lean on me and say, “Okay, what did you do to get things up and running?” I had that list, and that was more valuable to that person as an employee than a shiny system that everything figured out. Which doesn't mean I don't want the shiny system, but I think it's interesting that the human aspect is more in focus than I thought it would be.

MATT: Yeah, I think, and probably true for about every career, but those soft skills, communication, trust, are every bit as important as technical skills.

WILL: I mean, I keep on, like, I don't know, the further I go, the more I think it's more that than anything else. On a technical level, I've had really, really hard technical jobs, really hard technical jobs. But this isn't that. This isn't, you know what I mean, this isn't cutting-edge engineering research.

This is business logic, line-of-business stuff, making sure, you know, I mean, it can be tricky to take a block out of the Jenga tower without knocking the whole thing over, and that can be hard. But it's all about relationships and networks and people and, like, how you get people together to do the job. Like, that is the challenge of this role, not, like, you know, some fancy algorithm, you know, whatever LeetCode may have told you, but you're not going to be doing any of that.

MIKE: No. Maybe if you're a Ph.D. and you're getting hired to do some research somewhere, and those relationships are still going to matter [chuckles], even if you're working on that tricky technical problem, which you almost certainly aren't.

Thank you for joining us, Chloe and Jordan. It's been great having you, and thanks to everybody. It's been nice having a big pool here of people, with everybody contributing and offering different perspectives. It, I think, gave us some really good food for thought, how to do this better, do something that's really hard better.

Until next time on the Acima Development Podcast.

In this episode of the Acima Development Podcast, host Mike is joined by Kyle, Matt, and Ryan to discuss how technical interviews are evolving in the age of AI tools like ChatGPT and Copilot. The conversation begins with a story about a candidate who plagiarized code during an interview—years before modern AI assistance—which sets the stage for a deeper discussion on integrity and how easy it now is to cheat during remote interviews. The group reflects on how traditional coding challenges and rote questions are becoming less effective, given the accessibility of fast, accurate answers online.

Instead of focusing on textbook knowledge or code syntax, the team advocates for interview methods that prioritize problem-solving, communication, and curiosity. They emphasize the value of questions like “Tell me about a project you’re proud of,” or “When did you break production?”—questions that reveal depth of experience, passion, and the candidate’s ability to troubleshoot in real-world scenarios. They also suggest leaning into pseudocode or open-ended challenges that require candidates to explain their thinking, not just deliver a solution. These formats are harder to fake and better assess a person’s reasoning and learning capabilities.

The conversation concludes with a shared sentiment that the role of engineers is shifting away from just knowing code toward applying it creatively and collaboratively to solve meaningful problems. Tools like ChatGPT can assist, but they can’t replace real understanding, adaptability, and team skills. For both candidates and interviewers, the core takeaway is clear: be honest, be curious, and focus on showing—not telling—how you think.

Transcript:

MIKE: Hello, and welcome to another episode of the Acima Development Podcast. I'm Mike, and I'm hosting again today. With me, I have Kyle from our platform engineering team, who often joins us, and Matt from...well, you do a bunch of things, Matt [laughs]. He's a leader, I'll say that, and he's a director here at Acima, and is involved in doing a lot of things.

I'd like to defer introducing the topic and tell a story. I'm going to do that for...oh, and we also got with us, oh, there we go. We've got Ryan, who's a delivery manager here at Acima and should have some great input.

I'm going to start by telling a story. Oh boy, how long ago was this? Over, actually, I don't remember how long ago it was. I think [laughs] I don't even remember which company it was at [laughs], whether it was at Acima or not. I think that this was, like, 12 years ago, so I think this was prior to Acima. I was interviewing somebody, and I thought that I'd give him a coding test. It was a little hard to tell whether he knew what he was talking about or not. And I think I asked him to write some Ruby code that would output a Fibonacci sequence, output the first 10 digits of a Fibonacci sequence. And he said, "Okay,” and he just got quiet for a bit. This was a remote interview, by the way. I think he was overseas.

I waited a couple of minutes. He came, and he dropped some code, and there it was, and it worked. It was weird. It was, like, weird code [laughs]. There's stuff that's idiomatically normal for the language that you're writing in. And this was Ruby code, but it barely looked like Ruby code. It looked like somebody had pulled it out of some other piece of code that was doing something else and then tweaked it. It barely fit. It was just really strange. And you don't usually write really strange code when you're on the spot. You're going to write something really simple. And you might have a couple of quirks, but you're not going to have something that's just really weird.

So, I finished our interview, and I went and I Googled some of the code because it was weird enough that it was unique. I Googled for it, and it came up immediately on Google. He just copied and pasted from something he’d found online. And not only did he not get hired, but the contract shop he was coming with took a real ding because of that. There were some bad vibes thereafter [chuckles] between us and the shop because you sent us somebody, and he just cheated on the interview.

That was before all of the AI tools that we've got now [chuckles], before ChatGPT. Think about all the changes that have happened over the past 15 years or so. This was prior to that, and still managed to cheat. Well, he didn't get away with it [laughs], but he still managed to cheat the interview.

And that introduces our topic today. We're going to be talking about how to deal with technology, how to do interviewing in a world where we have technology now, where it makes it very easy for people to pull out answers quickly and hard to verify.

This is more broadly applicable even outside of engineering, you think about in schools trying to do testing. It's a general problem. We're going to talk specifically about the software engineering world, because that's what we do, and how to do interviews. How to do your interviews broadly, but specifically, how do you do interviews when it's really easy to cheat? So, I've told a story about [chuckles] somebody doing that sort of thing. Any of you have some stories or similar experiences that you've had?

MATT: Well, I don't know if I have a story. However, prior to this call, I was just conducting an interview. And something I think is really important, especially these days where a lot of people are working remote and interviews happen virtually, make sure their camera's on. And ask questions that, A, might not be as easy to Google, but require some conversation. And a demonstration of understanding of concepts, I think, is important. I think these days you have to dig a little bit deeper and ask the whys, just not the hows, on a lot of these technical questions.

MIKE: I've had other interviews where I've asked somebody a question, a technical question that was a little broad about architecture, and they answered it really fast. He talked about extracting services, for example, to clean up code and decouple things. And they answered them really well and quickly on all of them, but their answers were brief. This was, like, they gave the right answer really quickly and then never elaborated.

And I'm thinking about one example where this candidate did that repeatedly, all the right answers, one after the other, like, "Okay, they seem like a pretty good candidate." Then, when we actually employed them, they didn't seem to do anything [laughs]. They apparently were really good at either Googling the question or had researched interview questions before, so they knew what to say, but they didn't actually have the deep knowledge. They were able to give quick answers.

So, one thing I've learned is a red flag, and you've touched on it, Matt, you've got to ask questions that allow them to elaborate. Talk about that “Why?” That conversation is hard to duplicate. You can't fake it in the same way. If you can come up with a tool that will do that, well, then you've got a tool that can maybe do the job, but we’re not there yet.

MATT: Or if I have my phone with ChatGPT on listening in conversational chat, it's going to respond as fast as I could, right? And that makes it a little tricky. By the way, I think I was present in that same interview with you.

MIKE: [laughs]. You may have been.

RYAN: So, I guess the question here is, are you guys against using such tools like Google and LLMs, agents out there that an interviewee can use to answer your question?

MATT: Well, I want people to have an understanding of what they're being asked. I use tools all the time to help me with my work, and that's just the way of the world these days, right? If tools are available, let's use them. But I need to understand what I'm doing, or the next problem that comes along, I'm not going to solve properly. And these tools are not a replacement for a human being.

All of us have probably used Codepilot at some point, or Copilot at some point, or Cody, or something similar out there, right? That code needs review, and it doesn't always solve your problem the way that problem should be solved. So, yes, I support using tools, but if I'm hiring someone, I want them to understand the problems they're trying to solve.

RYAN: And I think that is the crucial point here that we want to discuss is, like, because ChatGPT or any of the LLM model what do they do? They go and surf the internet, right? So, when you get your answer back, it doesn't know whether that's the right answer or not. It just knows that this is an example it found somewhere on the internet, right? And the person who put that together had to understand what they put together. So, when Mike said, "Well, this guy found that answer on the internet, and it worked.” So, why does it matter if it's strange? If he understands and he can explain why, so why does it matter at that point if it's strange?

MIKE: Well, you hit on something important. If he had told me, "Well, I'm going to Google something and then talk about it," then that would have changed the whole conversation, and we might have even hired the guy. That's not what happened. He passed off the work as his own.

MATT: Integrity is important.

MIKE: Yeah. So, there's deception there. I expect people to Google for their job [laughs]. I think we've talked about it here. That's a key aspect of engineering. It's hard to do engineering without Google. I remember the first time I used it. My boss told me, “Go use Google.”

RYAN: It's going to change the nature of how we do interviews because there's no way we can...even with camera on, and even if we ask the questions that need interaction, there's no way to prevent them from looking up information on the side screen. We can't see it. I mean, you can probably have a big enough screen with multiple windows open. You can do that right there and get away with it pretty easily. But I think the nature of the interview has to change. Instead of asking, “Hey, go write me a function that generates Fibonacci or do a binary sort,” or something like that...because that kind of stuff is not useful, right?

But you can say, “Hey, I have this problem here. How do you solve it for me? And feel free to use the tool that's available out there. You can use ChatGPT or whatever out there. But you need to be able to explain why you put that solution together for me and explain it to us,” right? I think that the nature of the interview has to be like that, instead of asking what or how to do some basic stuff. We have to move beyond that type of interview.

MATT: I agree with that. You know, I'm not one who's a real big fan of code challenges anyway, but, you know, maybe it's as easy as, “Explain to me in pseudocode how you would solve this problem,” right? And I'm not going to give you some academic computer science problem because most of the time they aren't real applicable in the real world unless you're doing low-level stuff, right? But a real-world problem.

RYAN: Yeah, yeah, yeah.

MIKE: I had the same thought about pseudocode as I was preparing for this session. I thought, you know, pseudocode goes a long way because you can't usually Google it [laughs]. But even --

RYAN: But you can put your thought process in there, and you can see it, you know, forming. And even though it doesn't compile, compiling might give people, the interviewee, that anxiety, like, oh, code doesn't compile [chuckles], you know, and then they just drift away from the main problem they’re trying to solve.

MIKE: Which is what you're after, right? You're trying to see how people think about the problem.

MATT: Yes.

MIKE: The solution itself is almost irrelevant [laughs].

MATT: It's the approach. I wholeheartedly agree. And I think there's a soft skill involved here as well. And that is just the more you interview people, the more you can read people, right? The more human interaction you have, the more you can read them. If I see someone on camera and they're taking a second to answer a question, I can see if the gears are turning in their head, versus if they're trying to type something on a keyboard quickly, or vice versa, right? If someone's thinking something through, I think for the most part, you can pick up on that. But we're getting into a whole new world of the way we have to do things these days.

KYLE: I have to kind of tack on with what Matt was saying there. That's kind of...when I've been interviewing, I haven't necessarily cared even what level a person is because regardless of the level, it's how much can they learn? And I think that's more of what I focus on in interviews is like, does this person seem like someone that I can teach, someone that can pick up on what I'm trying to tell them? And, respectively, a junior I would give more leeway to.

But it's one of those things where, like we've spoken, do we really care that it's solving these generic coding challenges, or handing them a marker and telling them to go write on the whiteboard, even on camera? Just, I want to see what your thought process is. Can you solve this? And I almost feel like we can still use those coding examples or whatever it is that we're wanting to use. But even historically before AI, I would give someone homework in the sense of like, “Hey, go spin up an EC2 instance, and give me a script to do that.” And then, I would judge based upon the quality of that script.

What script did they do it in? Did they use Terraform? Did they use Pulumi? How did they do it? How repeatable is it, or did they just give me a set of instructions to do it? What was the thought process behind this, and how far along are they? I was judging on those criteria because you can, even in today's world, you can ask AI, “Well, how do I spin up an EC2 instance?” Well, depending on, you know, it's going to give you answers, right? But are those the answers that you want to give during an interview?

So, I think we kind of need to take those into consideration, too, because, yeah, these generic answers aren't what we're all going to be wanting. And even taking issues that we're currently running into as a team, I guess, and saying, “Hey, this is what I'm actually facing. If you were on my team, how would you solve this?” and kind of seeing how they would think through that that would be helpful, too.

MATT: One of the things, and Mike's probably sick of hearing me say this because I sound like a broken record in all of our meetings, more important to me than ability to write code is ability to communicate and ability to solve problems. If you have those two things, the code can be taught. I don't see that as even being a big hurdle. But those are the things I'm really looking for, and AI just can't fake that.

KYLE: Well, and in today's world, right? I mean, code can be taught. At some point, will code even be needed? You know, that troubleshooting and ability to learn and communicate is going to be the top skills that we're looking for.

MIKE: I think it already was, and it just increases it, right?

KYLE: Yeah, more prevalent, right?

MIKE: Yeah, because you're removing some of the esoteric knowledge that was, to some degree, a waste of time, you know [chuckles]. If you have to spend a long time learning the language, that may not be a good indication that you're using the right tools for the job. Instead, you want somebody who can use the best human skills, their ingenuity, their creativity, to come up with an effective solution, and then make tools available that can make it easy to put that solution into practice.

KYLE: An engineer I used to work with at a previous company I always thought he had an interesting mentality for when he was doing interviews for his team, and this was way before AI. But I would ask him...because we were a Java shop. But I was in one of his interviews, and I was like, “Why did you not have any Java-related questions? Why aren't we worried about whether or not these people know Java?” And he said, “If I'm looking for a Java developer, it's just that. They are a developer. I'm looking for an engineer. I'm looking for somebody that can use any language, so I don't care how much they know Java.”

And that's always kind of stuck with me. It's just kind of like, okay, there are situations...I understand there are situations where you want a Ruby pro or a Java pro, but there are situations where you're wanting an engineer, someone that can use any tool, and you want to vet out that situation.

MATT: I think I want those more so than a specialist in any language.

KYLE: Yeah, exactly.

MIKE: Yeah. In the end, our job [chuckles] isn't to write code; our job is to solve problems. And the difference matters.

MATT: It does. I don't need to be able to write code to make software do something, right? I need to know how to solve the problem that we need to solve, and I can use code assistance to help me write the code if it's in a new language. I can, you know, yes, I need an understanding of software engineering and design patterns and, you know, all of those important things. But anyone who wants to or is a software engineer should really focus on being language agnostic because you should be using the right tools for the job at the right time.

MIKE: I don't want to give the wrong impression that you shouldn't get good at using your tools.

MATT: Oh, absolutely not.

MIKE: I agree with what you're saying. But one thing I've heard said is you should get really good at least one language, and I agree with that. It's a mark of professionalism [laughs]. And, you know, if you care about the tools you use, you'll learn at least one of them really, really well. And they do differ. It does matter some. Ryan is really good at functional language, who’s here with us, and they tend to approach a problem in a different way than more procedural languages.

Likewise, if I was interviewing somebody who's completely unfamiliar with object-oriented aspects of a language and I was doing Java, I'd [chuckles] know that there was going to be a lot more time required for them to learn the tool. So, I do think that there's something to be said for knowing something, but I think that that kind of expertise will probably decline somewhat in importance over the coming years as the ability to automate a lot of that work increases.

MATT: Someone can be good at a language, you know, and you see this really often in the Ruby community because all the bootcamps for years were Ruby, right? And people know how to do things in Ruby, but they don't understand the language and what's happening behind the scenes and under the hood. So, I think, to your point, Mike, I think knowing a language really well and being an expert at a language, you have to understand what's going on and why you're doing the things you're doing. And I agree with that 100%.

KYLE: This kind of goes back to a previous topic which we had, which was learning languages. I can't remember what the topic was specifically, but it's entirely that, where you get good at one language, and it makes learning new languages that much easier. And that will show in an interview.

RYAN: I have a story about, like, doing interviewing. So, I'm a big fan of Elon Musk [laughs]. People don't like him. It doesn't matter. The way the guy approaches interviews or his work ethic it's just something I'm a big fan. So, I follow him closely and listen to a lot of the things that he posts.

So, one of the things he said about how you weed out the people who just know how to talk really well during an interview versus the people who actually do the work on a specific project is to ask them to tell him about the detail of that project. So, he was like, you know, “Tell me something that you...some code you worked on in the past,” and then drill them on it because the people who actually worked on it they’d get excited about that kind of stuff.

They would go into detail that you didn't even care to know, but that would tell him whether this person actually worked on something, or they just read about it, or just heard about it. Because when you get to a certain degree, you can talk your way through a lot of these things without having the detail of it, right? But when you have to actually explain the detail of a certain implementation to solve that problem, then it shows whether you actually worked on that problem or not.

So, one of the guys that I interviewed with in the past he kept going on and on about how performance tuning is one of his biggest hobbies that he enjoys. It's not even his main job. His main job is a developer, but he would spend time on his own and try to get into codes and try to fine-tune it, right? And I thought that's an interesting subject. Then I started drilling him on it and said, “Okay, so what...” It was in .NET, and it was an ASP.NET web application, but they run on Windows server. It was not even on .NET Core.

But anyway, so when I started drilling him on what he'd done on it, he'd actually start explaining stuff that people not only wouldn't know. He would go into how he would go to the web server, change the thread pool so that you have, by default, 100 thread pool. He'd maximize it up to 1,000 thread pool on the web server because he'd only get one dev machine to test his code. So, he wanted to be able to run concurrent, high-concurrency code, [inaudible 22:19] high throughput. He was like, “I figured out how to get into the configure at the root and change the thread pool to allow 1,000 threads by default.” And he would increase the performance for the process power of the CPU up to 100%. He'd max out that CPU running on that server. And he'd just go and do that kind of stuff.

When you hear that, it's the enthusiasm behind the story. The excitement behind the story is kind of a telltale sign that he actually worked on it, and he knows what he’s doing there. I mean, of course, I hired that guy [laughter]. But nobody goes into a web server and changes the thread pools. And when you work on Windows, it's like the worker thread pool versus the IO thread pool are different. And people who don't just write web server code they don't understand what that means. One is run the receiving of the request come in. One is run all of the IOs processing behind the scene. But they have two different thread pools. And he actually understood how to explain all of that. I was like, “Wow, okay, you impressed me [laughs].”

But, you know, I mean, that's the kind of thing I've been using as my tactic on the interview. I ask that kind of question, drill them on it. And if they don't give me enough information, then I'll just mark that as, you know, he didn't really actually work on it.

MIKE: That's actually become my favorite interview question as well. “Tell me about a project you're proud of and go into detail about what was involved.” It's interesting hearing what people are proud of, and then interesting hearing about the details. If they’re like, “Oh, yeah, I don't remember it very well...”

RYAN: You didn’t work on it [laughter]. I would make that assumption. If you don't remember, you didn't work on it. Because if you actually spend days and nights trying to make it work, you remember why it didn't work or why it worked [laughs].

MIKE: Absolutely.

MATT: Days and nights.

RYAN: They might not remember the code level, but they know exactly what the problem is, how they solved it, how they figured out what the problem is. Most of the time, you just had a problem. You don't know what the root cause is. And just to spend time on it and figure out what the root cause is, it kind of differentiates the type of people who have no problem putting the time to either solve a problem that they're interested in. Whereas if somebody just showed up and, you know, meet their hours and go home, it's --

MIKE: I can think of...Yeah, you just talked about that. I can think of a clever solution I was proud of that I implemented over 20 years ago [chuckles]. And I couldn't tell you at all what the code did. I remember this because a few years later, the company reached out, and I hadn't worked there in years. The company reached out to me. Like, “Do you remember how this worked [chuckles]?” Because it was in Java, and I don't think they had any Java engineers on staff. They didn't know what was going on.

So, they just reached out. Like, “Hey, we found in the commit log who wrote this,” tracked me down somehow and reached out to me and said, “Hey, can you help us with this?” And I talked them through it because I could still, right? Because I remember that, that I could go back and talk about it. And I remembered because when I looked at the code, I didn't recognize the code at all. I hadn't looked at that code in years, and I had no familiarity with it. But I remember exactly how it worked. And if somebody can't say that, I think you're right. They weren't engaged or, more likely, they really weren't making those decisions.

MATT: They weren't interested in making a difference. And those are the kind of people we want, right? People who come in, want to make a difference, want to be innovative, and think about those tough problems. The level of gratification you get out of solving really tough problems for a company, even if you don't get compensated for them, it's still huge. It's a big deal. And it's something to be proud of.

MIKE: Yeah. We could probably all think about solutions we're really proud of, even from a long time ago.

RYAN: That's why we get into tech in the first place, right?

MIKE: [laughs] Yeah.

RYAN: It’s those kinds of little wins that keep us going [laughs].

MIKE: We all probably have some significant other in our lives who's seen us reach that moment where we solve the problem, whether it's debugging [laughs], or you finally get there. And they know [laughs]. They've seen it a few times [laughs].

MATT: Absolutely possessed or obsessive in the process. You haven't been to sleep for two days [laughs].

RYAN: There was a...Oh man, this was a fun problem that I had to deal with. So, I was a .NET programmer for a long time [laughs], so a lot of my fun stories come from the .NET days. I think by the time I got to Haskell, things just got easier because of Haskell [laughs]. It’s crazy because when you get to Haskell, it gets boring because things just work [laughs]. I think that’s --

MATT: Said no one ever, Ryan.

[laughter]

RYAN: I was like --

MIKE: It's true. It just takes longer to get to the working.

RYAN: Yeah. So, when it works, it just runs, right? You don't have to do anything. So, on that day, we released these brand-new features. I used to work for this analytics company where we bring in customer or user survey data and then do all kinds of analysis on it. So, the different kinds of math equations that we have to run on the data one guy decided that instead of using the data that had been passed through the pipeline, he would clone it so that he can make the calculation on it without affecting the original data going.

It sounds like a good idea because there's a lot of calculations that he had to run on that piece of data. So, every single piece of data going in will get replicated two or three times based on the equation that we had to run on it. And then, at the end, he would just return the result back and then just change the initial object, right? Just by copying it three or four times, it tripled or quadrupled the amount of memory we used on the web server [laughs].

And we have to roll back, like, the release night it come in the next [inaudible 28:57] because it merged, and we didn't have that kind of like we roll back plan that we have here where we just deploy the previous thing. It merged with the code when it build. So, now we have to undo what he did and then release a new version on the release night. And just all of a sudden, the server would just markdown. Everything slowed down to a halt when we started running automation tests.

Automation tests, what they do is they run hundreds of thousands of requests through the application [laughs]. The server just exploded in the middle of the night, on release night. Oh, that was a fun problem, got it figured out by the time everyone realized that it was the memory consumption. And I was like, what jinx? And then, by the time we looked at the codebase, it was a copy. It was a copy, and Windows and .NET didn't really do a very good job of memory management. When you make a copy of an instance, it doesn't reuse memory. It just copies a whole brand-new copy of it. So, by the time we got through it, it was three days’ worth of death march to roll back this code and retest it [laughs].

But then, when I go from .NET into Haskell, and I was like, how does that work with the immutability, right? Because when you change something, you practically have to make a copy of it. You can't just change the object it passed in. It doesn't allow you to change the object it passed in. You have to make a copy of it. And I have a nightmare of that problem that I had to deal with. I was like, so, how does that work in here? But it turns out Haskell has something called light thread.

So, basically, it reused memory behind the scenes without even compromising the ability of immutability of the data. So, with the data coming, you can't change it. But if you changed it on a copy, it still somehow referenced the old data. So, it didn't make a full copy of it [laughs].

MIKE: Nice.

MATT: There's that passion we were talking about.

[laughter]

MIKE: Yes.

RYAN: I just take a look [inaudible 30:56], product right? The way it sucked in this entire XML, and it just parsed out a piece of it lazily. And that’s kind of, like, the power of the Haskell right there. It just got me excited. But the thing about Haskell when it work like that, we hardly see any problem anymore, right? We keep rolling out [inaudible 31:14] product and the XML...look at the Grand [inaudible 31:18] XML. It was huge. But it didn't bring down the server at all. Memory consumption on it is so small on it. But, anyway, that was a fun problem to solve [laughs].

MIKE: As Matt was saying, there's the passion. You care about that. You love the tools that you're using. Love the aspects...but there's probably things you don't like about the tool, too. But there's things that you just absolutely love about it, and you just can't help but want to talk about. I heard, I don't know why it took me so long to hear, but I heard some time in the last year the joke, “So, how do you know somebody's going to run a marathon? They'll tell you.”

MATT: Oh, they'll tell you.

RYAN: Yep.

[laughter]

MATT: Burning Man, they'll tell you.

MIKE: [laughs] It's the same deal, right? If you care about it that much, you just can't help yourself. You're just going to be talking about it. And that's the kind of questions that I think are evergreen, right? They're going to keep working. They're going to work 10 years from now because what you're really exploring is somebody's humanity, and that's a powerful thing.

RYAN: It’s a bragging right. When you solve a problem, it feels good, and then the company would benefit from it. It feels good. The younger years of my life [laughs], I didn't care. I’d sleep in the office if I had to [laughs].

KYLE: I was analyzing Ryan's stories as he was going along and thinking how many questions I didn't have to ask him if I was interviewing him, right? Because now I know that Ryan can troubleshoot. I know that he has a really good in-depth understanding of C#. He's translated that over to another language and made that applicable to another language. So many of these different aspects of an interview that we are looking for were solved with one question saying, “Tell us about when you solved a really big problem and how you felt about it.”

MATT: Memory management. [crosstalk 33:26] You know, the interview I just did, I had a list of about nine questions that I was going to ask. I asked my first question. By asking my first question, he answered all nine of my questions that I was going to ask throughout the interview. Because he was very passionate about what he did, he demonstrated his knowledge of what he did. And I didn't have to ask him any technical questions because it was really clear he had a great understanding of technology and the technology he needed to use.

He talked about memory leaks. He talked about resource management. He talked about interfaces with APIs, like all of these things I was going to ask him about. My first question, which was one of the questions Mike likes to ask, and it's, you know, “Tell me about what you've been doing and what about in your career have you done that you're really excited about?” And he just went off. And about a half an hour later, I said, “I don't have any more questions.”

[laughter]

MIKE: I found another question that kind of goes the same direction but flips it. “Tell me about a time something you did failed.”

[laughter]

RYAN: It's kind of like [inaudible 34:56] break production [laughs]. You need to break production to earn that [inaudible 35:02] [laughter].

MIKE: And I asked that one. “Tell me about a time you broke production.” If they say, “I haven't,” you know you don't hire them because either they haven't been in the industry very long, or they're lying.

[laughter]

MATT: Yep. Anyone with any amount of experience has brought down production. And we all wear that badge [laughter]. It's when you do it often when it becomes a problem.

RYAN: Right. With the same scar over and over again.

[laughter]

MIKE: That question's great because, again, is somebody going to Google, get on ChatGPT, find, you know [laughs], examples of how bad somebody did? It's not something that's...you can't make a shallow replica, right? You kind of have to go into your human experience. And you can hear about the problem-solving skills. A lot of times if they're not talking about who they collaborated with, then that's suspicious, right? So, you broke it alone. You solved it alone. Why was nobody else involved? There's a lot of things that come up.

MATT: Yeah, and if you ask them, “Tell me about a time you brought down production,” and part of that response isn't how you fixed production, also a red flag.

MIKE: Yes [laughs].

RYAN: Troubleshooting is a skill that can be taught. You're just going to have to learn over time with the work that you've done. There are people who have that instinct. You can hear them go through the problem one by one. There are people who’s like, “I don't know where to start.” You've got to start somewhere. I mean, they can fix the problem if you tell them what it is, but they just don't know how to start.

MATT: Yep. And starting somewhere is better than not starting at all.

RYAN: Right. So, back to the AI topic with interview, the junior definition is going to change a lot, right? The junior dev definition. We no longer need to ask people what is the definition of, you know, and something anymore, right [laughs]? Use ChatGPT and [inaudible 37:23] [laughs].

MIKE: Yeah, it expands your brain.

RYAN: Right [laughs]? Crazy.

MIKE: And there's technologies that have been doing that for millennia. Writing expanded our brains because it allowed us to expand our memories. Now your memory can last, not just short-term, but for years, and even across generations. And across geographic differences, you can send the book somewhere. So, writing was able to expand our brains and has had a dramatic influence on human culture.

And think about the printing press that allowed that to be expanded dramatically and computing. The rise of computing has allowed people to expand their mental capacity, same process, I think. Now we get to expand our functional intelligence. Some of the things that before we had to rely on amassing this deep well of knowledge are less important. And we can use our problem-solving skills more quickly. That's a big deal, right? That’s a big deal.

MATT: Yeah, what we do isn't going away. It's just evolving.

MIKE: Right. It's interesting you say the definition of a junior engineer changes. That's true. They don't have to have as much of the, like you said, the specifics. They don't even have to know it that well because they can use some tools to help...to bring about, how do I do this? Well, it’ll show you right away. But the problem-solving skill, that's something that you have to practice.

RYAN: Yeah. I don't know if you guys have done this before, but like, I would print out an exception stack and have somebody read it and tell me what they think. I've done that before in the interview because, like, if you can't follow this mumbo-jumbo blob of text to find out where things break, you're not quite at the mid or the senior level yet. You're not even there. You're mid-level. Expect to at least know how to read an exception stack and figure out where things break [laughs].

MATT: I love looking at Grafana logs every day, Ryan.

[laughter]

RYAN: Well, that's the thing, though. When you see a problem, what do you do first, right? You hear that, I would go look at log, or I would go look at the exception return. Where do they start? And that's important.

MIKE: I've done something similar by just showing a file. You’re doing a remote interview, pull up a file in your code, right? And say, “What does this do?”

RYAN: Yeah. You can --

MIKE: And it goes a long way. Go ahead.

RYAN: Yeah, you can read between the lines, and most of the time you can read between the lines and just guess 90% of the function there, unless it's a trick question.

MATT: Well, if you understand code, right?

MIKE: Exactly.

MATT: And it weeds people out really quickly. And I've seen Mike do it. I mean, we've been in so many interviews together. I probably can’t even count. But some of the answers you see with that question are really surprising. And it's a great question because it shows if someone can follow code, understand where things are going, what the dependencies are, what's coming in and out, some key things that you just have to understand.

MIKE: And the questions they ask are sometimes more important than what they tell you.

MATT: Almost always.

KYLE: It makes me think back to...I had an interview several years back, now at this point, but somebody did that. They threw out code in front of me and they said, “Tell me what's wrong here.” And I looked at it, and I was just kind of like, “I have no idea.” And I looked at it a little bit closer, and finally I was just like, “I don't know that there's anything wrong with the code, but this comment doesn't make sense.” You know, they’d commented the code.

And he's like, “Oh, that's what I was looking for.” He'd thrown code in front of me, and he's just like, “Yeah, we wanted to make sure, basically, that you were paying attention to all aspects of the coding window.” And I was just like, okay [laughs], you know. But he was literally looking for that I would read the comment and then read the code and determine if they actually fit together. And stuff like that can even be helpful.

MIKE: Nice. We've covered a lot of ground here on what matters for interviews. We've talked about not just doing a coding challenge because it doesn't really get, many times, to the right kind of information that you're looking for. But if you do...and sometimes it makes sense to talk about bigger problems and then do it in pseudocode. Do it at a higher level where people are talking through their thinking. Ask why rather than what.

MATT: Some advice for those of you out there listening that may not have a lot of experience with this. Be honest. Be transparent. Ask questions. Those are the things that are going to get you hired.

MIKE: And that's the main thing we've ended up focusing on here, right, is asking people to talk about what engages them, and then, let them do the interview [laughs]. Let the person you're interviewing speak for themselves and reveal what they care about, what they're good at. And those kinds of probing questions work and will continue to work for a long time.

I think it's been a great session. Hopefully, you can take this with you in your own interviews.

Until next time on the Acima Development Podcast.

In this episode of the Acima Development Podcast, host Mike welcomes returning contributors Kyle, Will, and Dave for a lively and insightful discussion on communication and its failure modes—especially in engineering and development teams. Mike kicks things off with a humorous but illustrative story involving his children, hoverboards, cactus spines, and a major context mismatch with his wife. This leads into the first failure mode: people interpreting the same situation differently due to differing contexts. The group agrees that a lack of shared understanding often derails conversations and projects, especially when assumptions go unspoken or expectations aren’t clarified. Will and Kyle emphasize the importance of providing full context when asking for help or collaborating remotely, noting that even minor omissions can significantly delay progress.

The conversation then shifts to another failure mode: assuming communication is complete after initial planning. They highlight how this mindset leads to integration issues near the end of projects—when it becomes clear that vital tasks or dependencies were overlooked. Dave tells a memorable story about a sound engineer who foresaw this issue and left a self-contained module called “OYWNS” (“Oh Yeah We Need Sound”) that could be plugged in later, exemplifying proactive thinking. However, the team debates whether these are truly communication issues or just planning failures, ultimately agreeing that both planning and communication must be iterative and responsive, especially in complex, cross-functional environments. Mike brings in the Agile principle of “customer collaboration over contract negotiation” as a more effective framework to reduce these last-minute failures.

Toward the end, the group introduces a third major communication failure: speaking without tailoring the message to the audience. Whether it’s explaining unit testing to a non-technical relative using car analogies or trying to influence C-suite executives without drowning them in technical jargon, they agree that effective communication requires strategic translation, not just transmission. They also discuss how hierarchy and communication chains create distortion, using examples like long managerial handoffs or segregated teams that never speak directly. The episode closes with a call to action: be deliberate about context-sharing, break down silos, speak in your audience’s language, and ensure someone owns the responsibility of facilitating true collaboration.

Transcript

MIKE: Hello and welcome to another episode of the Acima Development Podcast. I am Mike, and I am hosting again today.

With us, we have long-standing contributor Kyle and Will Archer. Kyle Archer, Will Archer-no relationship to each other [laughs]. We have Dave Brady--

DAVE: Howdy, howdy.

MIKE: Who hasn't been here for a little while, but he's here today. He's been out on leave because of some medical challenges, so we're really happy to have him here with us today.

DAVE: I'm delighted to be vertical and to be here.

MIKE: [laughs]

DAVE: Yes, that was optional. That was an elective for me, was being vertical. So...

MIKE: So, great having you with us, Dave. We are looking forward to having a good conversation. And speaking of conversation [chuckles], that's what we're talking about today. We're going to talk about communication and communication failure modes.

I'm going to start, as usual, with a story, but I’ll introduce the topic first. To tell you a story...A few weeks ago, I got to do a little bit of setup here. So, you probably all know about hoverboards. They're not hoverboards, right? But it's like a Segway without a --

DAVE: Segway with no sticks?

MIKE: With no stick, exactly. The self-balancing scooter that you stand on and move around. They're ubiquitous, especially among young people and kids, and they're all over the place. We’d had one in my house for several years, and the kids played with it, and they enjoyed it.

My wife discovered something. You can put a seat on it, with a bar that comes out from it, and you put your feet on it. And it's got handlebars that you can lift up and down, two bars that you can lift up and down. So, you can adjust, you know, you can move it forward or back like you normally would by using your hands. And it makes it into, essentially, a little electric go-kart [laughs] because you've got a really low center of mass, you can just go full speed and have all kinds of fun.

So, my kids at home, at Christmas, that’s what they all got, and they love it [chuckles]. There's a ring around my house that’s...[inaudible 02:32] dead. [laughs] They have flattened it so much, not every day, but often. And [laughs] they have a great time. So, that’s the first line. So, I’ve got to tell a couple of things that are going on here.

In my office, I actually have a lot of cactus plants. Back during the depths of the pandemic, I had a little extra time and started doing some grafting experiments with cactus. It was fun. I've got quite a few cacti that I had some fun with that are now growing [chuckles] and taking up a lot of space in my office. But I like to take them outside in the summer.

So, a few weeks ago, late spring, yeah, the weather was going to be good, so I started bringing out these cacti. And I've got the gloves on, but sometimes you get the spines and especially the little fine...they're called glochids. I don't know if you're familiar with them, but they're fine, little spines, and they fall off really easily. And they get in your gloves, and they get in everywhere. And they're really hard to pull out, and they're itchy and awful [laughs]. So, they just get in everything. They are very hard to get rid of. It's like sand in the car after you've been to the beach, you know, it’s everywhere. Same kind of thing, it gets in your gloves.

So, I was bringing out these cactus plants, and I was getting more and more in my gloves, and my hands were getting itchier and itchier. And I got most of them upstairs, and my kids were out [inaudible 03:49] around the house on their hoverboards. And [chuckles] my four-year-old he comes up a lot in these stories because he’s a great source of stories. He ran his into a raspberry bush [laughs].

DAVE: Those are spiny. Oh.

MIKE: They are very spiny. I think he even did it deliberately, like, "Oh, I'm going to run into the bush, see what happens." Ooh, and then he realized that was not good. And then, he blamed the scooter because he thought...and he blamed the bush, “That shouldn’t be there [laughs].” So, he was angry at his scooter and refused to get on it because he thought it would hurt him again. And so, he started chasing my daughter, wanting her scooter. And --

DAVE: Because her scooter has never driven him into the bushes. His logic checks out, yeah.

MIKE: That’s exactly right. So, he's chasing her around the house, screaming. And I’m like, I should do something about this. But my hands are full of spines. And the thing is, I’ve been going up and down. My belt was kind of loose, and my pants had been falling down. So, I’ve got this series of problems: If I start running around the house, my pants are going to fall off. But if I try to take my gloves off and redo my belt or pull my pants up, I’m going to get spines all over my belly, which I don’t want. So, urgent situation. I can do nothing about it.

So, I take off my gloves [laughs], set them down, lift up my pants, adjust the belt, and I’m about ready to go out. And my wife comes out, like, “Um... what are you doing?” I’m standing there with my belly hanging out [laughs] with my belt while my son is screaming. He’s really yelling out there. Somebody’s going to, like, call the authorities.

There was a lot of context that was missed in this situation. And the way she was perceiving this situation was very different than the way I was perceiving the situation. We both knew there was a problem and me standing there I was trying to solve the problem. And [chuckles] it certainly did not look that way to her.

DAVE: Right. “What are you doing?” “I'm spanking our boy. What does it look like?”

[laughter]

MIKE: Not the topic—not spanking—but that kind of situation is incredibly common, where one person has context, the other person doesn’t. And you may be thinking you’re talking about the same thing, but you’re not, because both of you see the situation so differently. And neither of us was wrong. She looked “There is a real problem going on here. Why aren’t you doing something about it?” I was thinking, “I am doing something about this real serious problem, and you’re not seeing it.” It was very cordial, by the way—no yelling at each other. She just was wondering, “What are you doing? [chuckles]” We worked it out.

But this kind of idea where people see things differently it’s just everywhere. And I think it's one of these failures of communication, not just this kind, but there are a variety of these failures of communication failures that happen in engineering. And they are the problem, I would say [chuckles], with almost anything. It almost always comes down to some sort of communication failure. And I could tell stories, a variety of stories, and I probably will tell a few here. We're going to talk about, what are these communication failure modes, and what do we do about them?

My introductory story gives an example, you know, different context. You may be talking about the same thing, but your context means you're seeing things very differently, and unless you resolve that, you’re not going to see eye to eye. There is one example. Feel free to run with that. I would like to hear from the panel here: What are some communication failure modes you’ve experienced, and what’s the underlying reason behind it, and what can you do about it?

WILL: The biggest thing that jumps out at me when I think about sort of failure modes of communication is a lack of, I don't want to say respect, like, regard, care given to making sure that people have the same context for a problem that you do, right? Like, when you'll be working on something, it'll be all day, every day. You've been chewing on this bone for a couple of days now, right?

You narrow it down to some external team, some other partner, some other person. And you ask them a question, but you're sort of like...it isn't that your question doesn't make sense; it's the question assumes a certain level of context they don't have, right? And then, you run into all kinds of problems, which is sort of like a bunch of follow-up questions, like, sometimes people can get defensive, right, where it's like, “Hey, I've got a problem with your API.” “Well, if you've got a problem with my API, you've got a problem with me,” right?

And I think you can mitigate a lot of that defensiveness with, I mean, that's a separate problem, right, like, it's a separate issue. But if you can really lay out your case and your context and what's been going on, why you're doing it, why that path has led them to my door or your door or whoever, right, like, what brought you here and the reason that you think this person can help, right, like, there's just a lot there.

And really breaking that down is pretty laborious, but it improves outcomes dramatically, especially in, like, a distributed workforce. Like, you’ll find yourself in a situation where, like, you'll drop some Slack message on somebody. And to a very large degree, whether you get a reply in five minutes, or an hour, or maybe today, depends directly on how much context you provide, how much “What the hell did they even mean by that?” that you're asking them to do in this public help Slack channel.

MIKE: I've seen that to be so true. I'm thinking about a junior developer saying...it doesn't even have to be a junior, but it could be a senior developer. “I'm getting an error when I try to sign in,” and be like, well --

WILL: Story of my life, buddy [laughter].

KYLE: You just described every message that comes into the DevOps channel [laughter].

MIKE: What you've just done is you've asked other people to go and find all that context for you. And it's a real problem. It means that...I could go as far as to mention this happens in the DevOps channel. It's not very polite, but even if you're doing your best, I mean, if you're ignorant of it, you're not going to get a good answer. But if you come with somebody who says, “Well, I can't log in, and this is what I tried, and this is the error message that I got,” well, then that's catnip to an engineer who's like, oh, there's a problem. What am I going to do about that? And they'll go in and they'll solve it. It's just a totally different experience when you provide that context.

DAVE: If you can't give me context, at least give me steps to reproduce the problem, right?

KYLE: Yeah.

DAVE: Because then you need context.

KYLE: Links. Links are huge.

DAVE: Links, logs.

KYLE: Let me know what you're looking at.

DAVE: There's a reverse to that, though, right? Because it's not just that if you don't provide all the context, it's bad. It's more like, you have to do an impedance mismatch because we've all had the problem where you're fighting with the compiler, and what the heck is going on? You just moved into a new machine, and everything was fine, and now everything is just thrown up on this one service. And you sit back and you're like, why won't this stupid thing work? And somebody two desks over will go, “You got a new machine?” “Yeah.” “Disable pthreading.” And they go back to typing. And you do it and it works.

Because just the sound of your error, the tone of your voice, the time of day you were having this problem, right? You just got out of your sync meeting, and just the mention of “Why doesn't this work?” was enough for this person to grasp all the other context, assemble it, and hand you the correct answer.

MIKE: So, context is huge and one of the key failure modes, right, I've seen in communication. I think we'll hit on some other critical ones here, but I think that that is absolutely one of the important ones. That’s what I led out with, right? And the way to solve it, as we've been talking about, is give people those clues, right? Give people the breadcrumbs they need to be able to do something. If you don't build that shared understanding, then one of you is going to be standing there fixing their belt, and everyone's going to...”What are you doing out there [laughs]?” This is just going to happen.

I think one thing that I've taken to doing is in almost any group conversation where we're talking about a project, I will start by laying out the reason that we're working on that project and a summary of what we're doing and why. A lot of people in the meeting, I'm assuming, already know that, but almost I always get expressions of appreciation. Because even if you have been working on it, maybe it's been a week, and developing that foundation for the conversation is useful. Even if it feels a little silly, even if it feels a little redundant, even if everybody knows it, it gives you something to run with. Without that shared understanding, you don't really have language.

KYLE: That's an interesting one, because we actually had a leader that posted a topic at our work here a while back, and it really kind of stuck with me. He talked about how his father would start conversations with people, and he would provide that context, like, hey, I'm such and such from this interaction, from when we did yard work together, or just to kind of give context so that actually clues people in on who you are and where you're coming from, why you're talking to them.

You gave a very specific answer to engineering, which is great for this topic but I'm just saying, like, in general, for everything, just providing context, even in introductory conversations with people, it's very helpful and can save embarrassment, even.

MIKE: Yeah. I loved that also. Like, “Hi, I'm Mike. This is where we met.” Because they're thinking the same thing, like, you may not remember their name, and maybe you don't. And that way, you're giving them permission to do the same, right? You're saying, it's okay if you don't remember me. That's normal. Let's build some shared understanding.

DAVE: How do we make sure...so, I've got this idea bouncing around in my head. It might be a dangerous one or a dumb one. We won't know until we release it into the world.

I've been fascinated for decades on unit testing my own brain, like, coming up with things that are external to my meatware because a telescope can look at anything but itself, right? A microscope can examine anything but itself, right? Same idea. And the brain, it's hard to diagnose what's going on, especially because the order of operation is always back...like, we always think that we reason our way through something, and then we feel good about the decision. No, you integrated it; you synthesized the emotion, and then you back-justified it in your brain. We've got the receipts on the fMRI to show that that's actually what you did. But your brain tells yourself, no, I did it the other way around. You literally give yourself this reverse receipt that it's backwards because that would be logical.

Context: How to debug context is starting to feel, to me, like an integration test for my brain, where there are things I can do with my unit test brain. Like, I can pull up a video game that I've played tons and tons and tons that's challenging for me. And if it kicks my trash, I know I'm done. I know it's late at night, and I've lost my dopamine from the day, and I've lost my focus. I should put the computer down. If I open it, and I play it, and I just get a high score, I'm like, okay, I'm as locked in as I think I am. Let's go be productive.

I'm wondering about integration tests, where if we are following these rules, we should see this change. We should see this thing happen. You know, given a situation where I know this, and you don't...and this is the beauty of it: going into the problem, you never know what the other person doesn't know, right? So, it is a challenging unit test. So yeah, I'm just thinking through because it does feel like that, to me, like an integration test where you say, you log into the website, go here, and in the back end, you're bouncing back through seven different services, da, da, da, and at the end, you end up with a lease or with the end product of our system.

And it makes me wonder, like, if you are in a leadership role, and you want to affect a certain change in the organization, and that has got to be passed down by telephone through five layers, good luck, right? I'm not even sure how to approach that. I'm at the phase where I just thought of this thought, and now I have some interesting questions, but I have no interesting answers. It's just kind of a spicy thought.

MIKE: Well, it's an interesting idea. A way to get that shared understanding is that you test it. It’s like, do you know...and that's something I've seen work as well. Like, have you worked with this API before? It's a non-judgmental question. If you haven't worked with it, that doesn't judge you. It doesn't say you're a bad person. It just means you haven't gotten to it yet. So, asking those kinds of questions, “Well, have you worked with this API before? Are you familiar with this one?”

And if you ask a few of those questions and some of the answers are blank stares or “No,” “Well, let me talk about it a little bit, and then explain how it works.” If you're not running those tests, right, then you're almost certainly not going to have the kind of shared understanding that you want, because you haven't taken the time to check whether it's there. I like that. Testing in production, not so great. Not so great.

DAVE: There was a thing that Marty Seligman did. Marty Seligman wrote Authentic Happiness and Raising the Optimistic Child, which I think should be required reading for every parent ever because there's things that you can put into a child's wetware before the age of seven that will bake in and will stay with them, and that you can dramatically affect their health care, their mental health outcomes over their life with that.

WILL: Dang. My kid’s seven. Oh, well, I could save one of them maybe.

DAVE: What’s that?

WILL: I have a seven-year-old, so maybe I can save one of them.

DAVE: Right. It's on the bubble. Yeah, speed-read it. Speed-read it.

DAVE: But he did...I wish I could remember what it was called. I want to say it was, like, a case, like, it was a case study, but, like, CASE was an acronym. It was, like, Context-Aware Semantic Evaluation or something like that, where they were taking dead people and analyzing their speeches, like presidential stump speeches and that sort of thing. Because it’s science, you have to come up with a hypothesis, make a prediction, and then test your prediction. And so, what you do is you gather statistically improbable phrases.

And so, where am I going with this [laughter]? What's the context that you have on this? When I sit down to work on a ticket with a coworker and I ask them, “How does this tie into the rest of the system?” if the answer is, “I don't know,” and they're the senior and they're the person who knows that system, then that's significant. We should maybe put a tick mark on that, right? It's like, I'm supposed to get this from you. I would expect you to have this. Who would you expect to get this from? And that's kind of an interesting idea.

I certainly, like, hanging out with Ramses, like, Ramses knows where...I don't know if Ramses knows where all of the bodies are buried, but he certainly knows where all the memorial services are held. And he and I worked on something a couple of months ago, where it was going to send an email. The system was going to send an email. And that piece got written eight years ago and never got tested. I mean, we went on production; we verified manually that it works, and then we walked away. And so, I'm like, “So how do we test this?” And he's like, “Um...” And when Ramses says, “Um,” I get real nervous. My tail goes way bushy because that means we're in the dark times in the code. And you do still find those. You do still find those.

And so, anyway, the point being, coming up with, like, identifying specific phrases, think back on this...machine learning, you can examine your past life for data. Come up with the phrases that you think were maybe telling a certain thing, and then watch for them. I don't know if this is a genius idea or not. I'm just telling you the experiment that I'm going to run next, and I'll tell you guys later how it went, so...

MIKE: Well, the principle you're suggesting here is to test it out, right?

DAVE: Yeah, test it, yeah.

MIKE: Verify it. It makes perfect sense. You verify it, and then you fill in the gaps.

DAVE: Yeah. We measure this as a cadence run rate, right? Like, somewhere in the org there's somebody whose job it is to just, you know, obsequiously, fastidiously, you know, slick up Jira like a fat daughter for a beauty contest. Sorry, that's a weird southwestern phrase. But just somebody's job who it is to just fuss with Jira all day. And this is the person who knows how to make a burnedout chart and a burnup chart and an Eigen chart and a Gantt chart and all this stuff. And these are the people that, you know, in every organization, there's somebody that can tell you these teams have this much velocity, and if we ask them for this feature, it will take them this long to get it, right? And we aggregate this over time.

And I'm actually suggesting so that you think that's, like, the wave function, right? We aggregate and sum them all together. I'm actually talking about, like, taking one interaction and putting a point on it and saying, “Okay, start the stopwatch. We're going to measure this one, and we're going to watch and see how this input affected the output. Did this one run long, or did it run short?” You'll have to do it multiple times because, you know, there's going to be 73 external circumstances to everything.

MIKE: So, I'd like to maybe, if you all are good, because we've talked a lot about how to deal with this specific problem about context, to talk about another communication failure mode. I think that we've all experienced this. So, you've got, like, five teams working on something, and the project is down to the wire. You've got about a week left, and then somebody realizes that the API between two of these systems isn't working. And everybody knew it had to be built. It was in the requirements six months ago, right?

But you reach this moment like, oh, we didn't do that. Because you all got in your silo, and you didn't talk to each other about it because you thought, oh yeah, we’ve figured this all out. We've done the communication. We're done with that. Let's go work on our stuff. And it's somewhat related to the missing context, but it's a little different, right? This is assuming that the communication is done and not iterating on it.

Big projects, as you're coming near to the end, it seems like they always have something like this, right? Like, oh, wait, we missed that. We forgot to either...well, there's always the one thing, right, the thing that doesn't work and takes way longer than everything else. And there's that, but a lot of times there's also that, oh, wait, how did we miss that? And we didn't miss it a lot of times. A lot of times you knew about it, and just somebody didn't write it down in the stories that can get done. So, have you seen this? And what do we do about it?

DAVE: I have a fun, short story about this. Years and years ago, I worked at Acclaim Entertainment. I worked on video games for them, Jeremy McGrath Supercross. And when you work on a video game 25 years ago, all you care about is 3D pixels, texels, voxels. How much graphics can we stream? And you can literally have a AAA title get nine months down the road on a 12-month dev cycle, and there's no sound effects anywhere in the game. There's no music; there's no sound, nothing.

And the friend I had at Acclaim that I most closely worked with was the sound guy, and he was a sound programmer. That's what he did all the time. And he put a module in there called Oywns, O-Y-W-N-S.cpp, right? And we would compile it, these libraries, and he would just check it into the repo, and he's like, “When you need it, let me know.” And we walked off.

And he actually left the company. And time went on, and it's like, do we need? Oh yeah, we need sound. We should go do this. And we pulled it up, and we realized, O-Y-W-N-S. Oh yeah, we need sound. He called it from the get-go that you were going to...like, I'm going to tell you in advance the thing you're going to forget. There you go. And bless his heart, he wrote the sound engine to have no dependencies so that you could just pick it up and plug it in. You only had to fight with the upstream code. You didn't have to fight with the sound. That was a genius-level, like, master development. Like, to see it, to call it, to walk away and leave working code in the future for somebody, it's amazing to me.

MIKE: And he'd learned that because he'd seen it happen [laughs].

DAVE: The other one is, like, you get to the last week, and it's not that you forgot to write the API but that you never tested the API at scale. And if it's a third-party company...I won't name names, but this has happened recently where we connected up to the remote API, and not only did it not work at scale, but it didn't work at all. Like, they had written it, and they had signed off their acceptance test plan for their unit test.

They had never talked to us. We were the customer. They had never spoken to us. And we connected to it, and it just...you know, to this day, we still see messages come through from the QA team saying, “Hey, just so you know, this guy's API is down, so you're going to be seeing a problem, you know, coming back from this.” So, I'm not going to name names, but I'm seeing nodding around the table, like, we've seen this. We've seen this.

And I like this because...like is the wrong word. There's not an obvious scapegoat that you can point to and just shriek at and say, “This is all your fault,” right? It's all our fault. This was too complicated, and we left one of our PFAs, our Potentially Fatal Assumptions, we left it unexplored until way, way, way, way late. And then, that's when you have late-night coding sessions to code around the problem.

MIKE: So, I gave, like, the overall idea. Dave, you gave a couple of specific examples. I'm guessing, Kyle and Will, you've seen similar problems in your careers, and if so, what do we do about it? Whether or not you've seen it, how do you address that problem? Because it's everywhere. I have some thoughts here, but I'm going to be quiet for a minute and let you all --

WILL: Is that a communication failure, really, or is it a planning failure? No offense. That's one of the hard parts of, like, just planning, and there’s things that are...I don't see it as, like, we could express this, or we didn't express this, or we said it, and it didn't land. Everybody was like, “Oh yeah, it’s sound.” Everybody was like, “Sound,” but everybody's focused on, oh, voxels, character design, game player, whatever, right? And everybody's just like “Sound,” right? Yeah, yeah, it’s sound.

But then, I mean, it's an organizational strategic sort of, like, planning kind of an issue. And that's just, you know, in my experience, it's just the nature of very complex plans and integration. So, it doesn't matter. Something is always last on the list, and that is, like, the further down the list you go, like, the more strategic risk, let's say, that it's not going to get looked at. And, you know, the reason it always comes at the end of the project when everything is integrating is that's when everything on the list is getting checked off. To call it a communication failure, I think, is not how I experience it, at least.

MIKE: Well, let me --

DAVE: I think I agree with you. It's both, right? It was a planning failure, but it's exacerbated by the communication failure around it. It's kind of how I see that.

MIKE: Well, let me give a take on this because there's a lot of truth in what you're saying, right, is planning. So, should we all do waterfall-style planning, where we just really lean heavily into planning, make sure we get the planning right?

DAVE: If you've built this thing five times before, yes, you should. It's a proven method.

MIKE: Good point. There is an approach that gets away from some of the upfront contract negotiation and makes it an iterative communication process, where does this work? Does this work? Does this work? I have just pulled up the Agile manifesto, and one of the principles it says, “Customer collaboration over contract negotiation.” One of the key items in there is that if you lean heavily into the plan, well, that is a really expensive way to solve the problem. And some of those things are going to be left out because it's just so hard to get right.

And it is a planning exercise, but to a degree, planning is all about getting that shared context, right? It's how are we going to be working together? And if you change that to having a repeated, iterative conversation, then you're still going to miss stuff, but the costs are lower.

WILL: Sure. I mean, everything's a waterfall at a small enough scale.

MIKE: Sure.

WILL: I don't know. I mean, still, yeah, I still see it as planning. I still see it as a planning problem more than a communication problem. I don't know. I think six of one half [inaudible 30:29] the other, right? Because what's planning other than communicating, right? Like, planning is communication of an idea.

DAVE: I think I can split this correctly. You're seeing this as...I'm not seeing...in the communication that happened, I'm not seeing a defect in that communication, and I think that's valid. I'm saying that the communication should have happened here, and it failed to happen at all, and that's the slice that I'm kind of approaching this as. Like, it just didn't happen, and you can make an argument that it wasn't anybody's job to do it. It was all of our job, but nobody...I don't know if that makes sense.

I agree with you that these, like, much of this communication that I described absolutely fell down on planning. And the oywns, my friend, Sean, he wrote oywn, and he had brought up sound at the team meetings, like, every single week for, like, six months, and everyone would just, like, you know, whatever, the sound guy is talking while they're doing it, and, to me, that's a communication failure.

But it led to, yeah, failure to plan of, like, when are we going to...there was nothing on the Gantt chart of, hey, let's make sound effects work, and you can better believe that when we released the game, the sound was not a tight fit. It wasn't Crazy Taxi where you could rock out to the game because the sound was tight and had been up since the beginning. It was just like, yeah, it's a soundtrack. We bolted it on, and you could feel it. You could definitely feel it.

KYLE: But in that scenario, the communication was attempted, right? Is that a problem in communication, or is that a problem in that we didn't have a project manager or somebody else that wasn't prioritizing it correctly?

DAVE: Good question. In my opinion, the answer is yes. I have kind of a...I don't want to get too many southwestern colloquialisms in there, but I grew up with a phrase called tough titty [chuckles]. And what that meant was suck it up. Yeah, this sucks. I can literally go back to an interaction and say, “You were the victim in this interaction, but you were also the only agent that you had control over what you would do. So, this is not your fault, but it's your responsibility. If you go back to this team meeting again in a week, you are the one who gets to present this.”

And, at some point, what do you do? It's like, I presented it to the team. I said, “This is getting urgent.” I communicated that as well as I could, but nobody wanted to pick it up. And, at some point, you just have to drop it at other people's feet and say, “It was sitting on your porch. All you had to do was open the door and pick it up.”

WILL: Yeah, I'm assuming that communication failure mode, right, if I explain, like, I need a context or a task, something, clearly, concisely, to the point in a constructive fashion and you just don't care, we are definitely in a communication failure mode, but I can only go so far [inaudible 33:23] don't care [laughs].

DAVE: Is that communication, or is that culture, or is it psychological safety? This is a beast with many symptoms, right?

MIKE: Well, I think it was Kyle who touched on this a little bit. If you don't have somebody who owns it, who is going to listen, then you have everybody caring about their own silo of stuff, and they're not going to care because they have no reason to care because they are worrying about their own responsibility. And if you say, “Well, it's everybody's responsibility,” then everybody has a tiny piece of the responsibility. If you get a crime that happens in a large city and lots of people are watching, well, everybody thinks somebody else is going to do something about it, right, so nobody does anything.

And if you don't have that designated person who's like, “Oh, I'm going to go in and do something,” you can hope people will be proactive and fill your world with heroes. Well, that's great, but it goes against human nature a lot of times. You need somebody who says, “Oh, yeah, this is my job. I am going to listen to this person, and I'm going to figure out if that fits in the plan.” And I think if you don't have the designated person, and I've seen this over and over again, if you don't have somebody who knows it is their responsibility to care about what happens, then nobody cares.

KYLE: This reminds me. I had an old manager. He was always in meetings, and he was always very...what I called silver-tongued, and I always teased him. I said, “You're quite a salesman. You're quite a salesman.” Finally, he stopped me one day, and he says, “You realize you have to be this way, right?” I was like, “Why do I have to be a salesman and an engineer? What are you even talking about?” He kind of explained it to me. And I have wondered if this is what the sound guy, you know, maybe it was above, they weren't willing to listen, or maybe it was the sound guy not selling it correctly.

Because what you have to explain to get an engineer to understand is going to be massively different than, say, a C-level or somebody in sales. You have to sell your idea completely different. You have to sell the priority of your task. And I guess maybe that's what I'm recommending is take a salesman's approach in your communication just to get your idea sold, to get your priority sold in these kinds of situations.

MIKE: Okay, so you didn't say it directly, but you just identified another failure mode of communication that's even bigger than that. If you don't speak to your audience, you've lost them. Oh, this is simplistic. If my four-year-old asked me, “Why does the sun come up in the morning?” And there's multiple ways I can answer that, right? Many of which would be ineffective. And if a professor asked me [chuckles], “Why does the sun come up in the morning?” They're probably looking for a very different answer.

And if I'm not changing my answer...and sometimes people get uncomfortable about this. They're like, no, I should just tell the truth. Well, that's a lot squishier concept than that, right? Your goal is to communicate something to somebody, and language is not very precise. So, communicating that message is not going to be the same, depending on your audience, because you have to meet them where they are. So, that's not a matter of breaking the message to filter it in some way, right? It's a matter of actually attempting to share it better, to carefully tailor the message to the audience. And if you don't do that, the communication does not happen, or at least does not happen well.

KYLE: Yeah, I feel like I do it all the time, because, for example, everybody you're speaking to, when you're explaining a problem, right? My father is a mechanic, so when I'm talking to him about anything computer-related, I relate it back to a car, you know? How does this work with a car? How does this work with the systems he's aware of? And yeah, just breaking that silo, I guess, we brought the word silo up earlier, breaking down that silo so that you are able to communicate on a commonality. Without that, I mean, right then, he's finally interested, before then, he doesn't care at all. It's just computers to him, right? But turn it into a car, and he's engaged.

DAVE: My father-in-law was a plant engineer for years and years and years. He had a full machine shop in his garage. He would make engines, like, actual engines that would burn gas or steam. He liked making steam boilers that would run. He was working with that level of precision and tolerance. And he didn't understand computers, bless his heart. He was born in the 1930s. He was a Depression child, and he worked at one company his entire life. He was the last of the boomer career, one-job career people.

And I remember trying to explain to him testing and finally saying, “You know when you went to make this cut, and you stepped back, and you made a jig so that you could calibrate the cut exactly where you needed it?” He's like, “Yeah.” “The unit test is that jig.” “Oh, okay,” and then he was on board. He's like, “Okay, it’s going to tell you this is going to be exactly where you want it, when you want it, at the time you want it.” I’m like, “You got it exactly.”

MIKE: Absolutely. And, you know, you mentioned the C-suite, Kyle. I've seen this. I've even done it. I've helped somebody write a slide, right, you know, make a PowerPoint, and put in some language that I thought was right. And I go, “Yeah, this is exactly what happened. And, you know, here's the technical reasons.” And they're like, “No, that doesn't convey it at all. If you have any technical details in your description, you're doing it wrong.” “Oh, okay [chuckles], but that's my job is to provide the technical answer, right? I'm the technical person.” “No, no, your job is to translate it to somebody who's not looking for a technical answer.”

And thinking about that differently is a critical realization if you want to fix that communication. Otherwise, you know, eyes glaze over, and communication did not happen. Like you said, you don't engage somebody. But then if you engage with something they actually do care about, it makes all of the difference.

DAVE: There was a thing --

MIKE: Go ahead --

DAVE: Something that I got taught years and years and years ago was when you're writing, like, an email to somebody where you want them to do something or you want to answer their question and it's detailed, you write out the answer. Then you read through your answer and drill it down to, like, one sentence. And back then, we called it the executive summary, right? And so, it was literally like, “Hey, can we use Hibernate to, you know, backdate the JVM with the database this way?” Executive summary, “No.” [laughs]. And then you give them, you know, three pages explaining why.

And we still do this to this day. You'll see people write this post and then at the bottom they go, TL;DR, too long, didn't read. That's perfect, except the TL;DR should have been at the beginning. Write your message, write your TL;DR, and then move it to the top so that somebody can read the TL;DR and then decide, “Oh, yeah, that is too long. I don't want to read that.”

MIKE: Some of the best blog posts I see do exactly that. They'll say TL;DR, a semicolon, and then give you the two-sentence answer. And, honestly, if they do that, I'm more likely to read it sometimes.

DAVE: Absolutely.

MIKE: Even though I've already got the answer. Because here's somebody who respects their communication.

DAVE: Yeah. I’ve wanted to --

MIKE: Go ahead.

DAVE: I've wanted to do, like, a YouTube channel where literally every video starts off with, “Hey, YouTube. I'm Dave Brady, and today I'm going to teach you how to...” you know, and in the first three seconds, I've told you what the video is going to be about. And you won't see this in the wild because the YouTube algorithm is explicitly designed to punish that. It incentivizes watch time. And what’s the easiest way to drag out the watch time? Is not give you the answer you need for as long as possible. And it's evil. It's awful.

MIKE: Which is why every recipe on the internet is a nightmare. Have you noticed this?

DAVE: Mm-hmm.

MIKE: How many pages do you have to scroll down to? The nice people at least put a link, “Take me to the recipe.”

DAVE: Yeah. I tweeted this, like, a few years back that if you have a problem with your computer, like, “I can't get Windows to resize, you know, when I'm in this mode. And when I'm in full-screen mode, I can't get this window to resize.” And you search it, and if the top-level search begins with an explanation of why you might be wanting that answer, in other words, like, “Full-screen apps are a great way to use your computer to its maximum potential, but some people...” you're not reading the answer; you're reading a content farm. And somebody is pumping up their SEO to get out in front of that, and they're doing the same thing. They're just dragging out the watch time.

MIKE: Let's not do that. The Internet is becoming a sea of AI-generated slob and declining in usefulness. We can do better in our individual communication, and we can be concise and tailor our message to our audiences.

So, we've covered three failure modes of communication. We've talked about lacking context. We've talked about everybody owns it, and so nobody does problem. And so, you know, how do you get somebody who will make sure that things don't get dropped and is keeping that iteration going? “Well, did we miss this? Did we miss something? Did we miss something?” And we've talked about not talking to your audience. Are there any other key failure modes in communication you'd like to talk through today?

KYLE: I've got one that I've been thinking about bringing up and trying to figure out when to [inaudible 43:28] in. But that is the chain, the chain of the communication. And what I mean by that, the best examples that I can think of right now is when you've got people lined up and you draw on their back. And then, the person there, the next person will draw what they think was drawn on their back, and it goes down and down and down. By the time it gets to that last person, the context is completely different. And the longer that chain is, the more context that is either changed or is missing from the original context.

And so, point being, in, like, the engineering world, that's how many chains of leadership something will go through before it actually gets to you or how many team members it will go through before it gets to you, meaning, like, you go too far, and by the time it gets to you, it's not actually what was wanted. And that's where I'm just thinking of situations where I've completed tickets only to find out, “Oh, that's not at all what they wanted,” and I had to go back and talk to the person directly and could have saved a day's worth of effort going directly to the individual rather than having it go down the chain to me.

And I've seen this problem be a concern even just one level where team members aren't allowed or are discouraged from communicating with one another. And they have to go up through their manager over to the other person's manager and back down to the other person to get the communication done, and it just sits there and runs through. You don't have that direct communication, and things just start to break down.

DAVE: Microsoft did an amazing study in the late ‘90s, early noughties, where they ran all the project data longitudinally for, like, Windows 3.1 all the way up through Windows 2000, which was excellent at the time. And they were basically, “What causes bugs? What causes bugs to take forever to fix?” And they found that it wasn't team size or project complexity. It was the number of layers of management you have to go up and back down to get into the right leg of the trousers to get the solution that you need, and it's exponential. So, if you have to go up one level, it's twice as hard. But if you have to go up two levels, it's four times as hard. If you go up three, it's nine times as hard, and it gets insane.

And going back to contract negotiation versus collaborating with customers, when you're stuck in that hierarchy, what you have to do is say, “Draw it again,” and then somebody comes and draws on your back again. You're going to argue over how much resolution of data I need on my back in order to fit, right? You're negotiating a contract. Just turn around and talk to the person, right, which is against the rules of the game.

But the point of the game is to show you that the game is broken, right? It's like, this is literally an exercise in doing it badly. So, when that happens, attack the hierarchy. Don't go after better contracts. Don't try to make it so, “Well, if we could just get it so that the DBA team and the engineering team and the ops team, if they could just pass everything back together as tickets, this would all be solved.” Nope. Nope.

MIKE: [laughs] Oh man. I've seen that one as well. “Oh, yeah, just make a ticket system then we won't have to talk to each other, right? It's all just organized.” And, of course, the opposite happens. It's a nightmare. But you said the solution is embedded in the description of the problem. And I've seen this, and I've seen it a lot from good, well-run organizations is the first thing you do is when you identify the teams that are dealing with this, you get people on the ground, and you connect them together, and then you step away.

If somebody is going to claim ownership of that communication and say, “No, everything's got to go through me,” then they are destroying their organization because you are preventing communication. And people who will instead say, “My job is to facilitate the right conversations. So, my job is to make conversation happen. That doesn't mean it has to come through me. In fact, I'm probably the wrong way for that to happen. I'm going to find the people who need to talk and get them talking.” And that kind of leader will make great changes in their organization, make things move forward. Again, I've seen this happen.

So, I've seen that kind of leader fairly often, and I deeply appreciate when I do because everybody's lives are so much better. And the best negotiations I've seen are exactly that. You might have some leaders who get in like, “Oh, yeah, we need to do this. Let's get the engineers and put them together and have them figure it out.” And when you don't see that happen, you're like, “Okay, this is going to take a few months.”

DAVE: We've talked about this on the podcast in the past that one of the most amazing things you can do if you're on a siloed team is to violate the borders of the silo and just go talk to somebody. We would do team swaps, and now you know somebody over there that you can just poke.

I'm just thinking about this, that we have call processors that talk to the customers, and they have to use our website on a side of the website that I hadn't seen anybody use because I was always facing another set of customers, and all of a sudden, I'm facing these people as they are my customer.

So, it's all remote. I jumped in on a remote meeting with one of the call processors to just shadow them on some phone calls. And if I had been in office, I would have picked up my laptop and gone and sat next to them, like, physically next to them because I want to actually create rapport. I want them to know what I look like and what my name is and that I'm a nice guy so they know they can come talk to me.

And just sitting with them, I immediately realized, “Oh my gosh, we are making you type with thumbtacks on your keyboard. It does not have to hurt this much.” I added, like, a dot sort. There was a thing, four or five merchants in it, right, and it wasn't sorted. And that's fine if you need to read through five things, but these poor processors were seeing 500 stores unsorted, and they had to find them by name. And they were just...and when you're a call processor, you're making minimum wage or barely above, and when you're in that job, you don't complain, right? You just put your head down, and you just do what it is.

I'm watching this, and I'm like, “Oh, my gosh, you're working in a Dickens novel. Who did this to you?” And I'm like, “I did this to you. I'll be right back,” right? And so, like, the next day, I, like, poked him, and I said, “Hey, just go to Jira,” and he went and he looked, and there was a ticket to fix the thing, to just sort the merchants. And that's a one-line fix, right? It's literally dot sort. It's a seven-character fix.

And I have friends on the processing team now because I took the one thing that hurt them the most in their eyeballs, and my point is it was collaboration. It was crossing through the DMZ and saying, “Show me what you're working on, and show me how you use this product because, in theory, we built it for you. Let's collaborate.”

KYLE: See, and without that direct communication, how slow would that have been? Because that's processing up to some project manager. Then it would be prioritized down to you, and you wouldn't care. You'd be like, “Sure, yeah, this ticket, throw it back over,” you know? And you would have never heard any update. But now you get that personal interaction, and you've got friends over there that are just like, “Thank you, Dave. Like, this is great.”

DAVE: The truest proof of that is the fact that this bug had been in their system, and with this particular...you guys know we have one merchant that's, like, an aggregate. They've got thousands of merchants right under them. It was that merchant, that super merchant, right? And we never checked it on the engineering side, and they had been dealing with this for years.

KYLE: Wow.

DAVE: And no ticket had been created. Because if you're processing, you don't get to complain about the software. You don't get to have input on what would make it easier. And just having an engineer sit down and go, “Oh my gosh, we need to fix this for you,” it was world-changing, right? Literally, there was somebody going, “Let me make this not hurt so much.”

And we talked about this a few minutes ago. A gift card is so much better than just more salary, and a nice bonus is so much better than, you know...I mean, to be fair, I like getting cash as a bonus. I love that. It's fantastic. But those aren't the things that I remember, right?

You'll find anybody on WordPerfect the second year that they were in business, and when the company took all of them to Hawaii for two weeks as the Christmas bonus, right? They all remember that because they were wildly profitable and spending like stupid because it was the middle of the ‘90s. You get the idea.

MIKE: Well, I think that is a great place to tie things up today. We could probably go and find a variety of --

DAVE: Oh, I've got eight more topics we could go into on this. There's different ways that communication can falter down [inaudible 52:11]. This is an evergreen topic. We could easily circle back to this [crosstalk 52:16].

MIKE: Absolutely. But we hit some big ones. And importantly, these all have relatively straightforward solutions. You just have to do them. You just have to use the solution, right? Test the context. Make sure that you have it. Cut through the hierarchy and go meet with somebody directly. Speak to your audience, right? Think about who you're talking to. Or designate somebody. Designate the person when you have a large group that needs to communicate who is going to facilitate that communication.

Let's work on that and make our communication better because, in the end, it's most of what we do is trying to figure out how to do things together as humans. And with that, until next time on the Acima Development Podcast.

The episode opens with Mike sharing two formative stories that highlight the importance of speaking up against inappropriate behavior. The first story recounts a moment in the late 1990s when a manager made a blatantly racist comment during a hiring discussion, leaving Mike stunned and regretful for not speaking out. The second story contrasts that moment with a more positive example—Jonathan Schwartz, then a Sun Microsystems executive, swiftly and calmly shutting down a sexist remark during a public tech event. These stories set the stage for a broader discussion on moral courage, preparation, and the power of speaking up in professional settings.

The panel—Mike, Will, Justin, Kyle, and Tim—then share personal anecdotes ranging from experiences with crass coworkers to moral gray areas in technical and corporate decision-making. Justin recounts a story of an inappropriate joke at a company meeting and the swift exit of the offending employee, highlighting the importance of understanding professional boundaries. Will reflects on his past confrontations, including a fiery exchange with Alan Kay and a more recent situation where he lost a technical argument at work but chose to “disagree and commit” to preserve team cohesion. Tim and others emphasize how reading the room, respecting legal and ethical boundaries, and choosing the right hills to die on are essential strategies for navigating workplace culture.

The conversation closes with strategies to foster healthy debate and encourage junior team members to voice their thoughts. The group stresses the importance of frameworks like DACI for decision-making, teaching engineers to argue both sides of an issue, and maintaining psychological safety in teams. Ultimately, the takeaway is that speaking up—whether against clear misconduct or in nuanced technical debates—requires preparation, emotional intelligence, and humility. And sometimes, the bravest choice is knowing when to push, when to let go, and when to walk away.

Transcript:

 MIKE: Hello and welcome to another episode of the Acima Development Podcast. I’m Mike, and I'm hosting today. Today, I've got with me Tim, joining us from security. We've got Will Archer, as is usually joining us. Thank you, Will. We've got Kyle, and we've got Justin. We've got a great panel here, and we've got a good topic.

I've been bouncing this one around for a while. I'm excited to talk about it. And I'm going to start with a couple of stories about...well, I was present for both of them. One of them is more about me, and one is more about somebody else. The context for the first one...and let's start by making this a little awkward. So, I'm a white guy [laughs].

WILL: Oh no.

JUSTIN: No, no.

[laughter]

MIKE: So, my dad's family's mostly British. My mom is an immigrant from Norway. So, white guy, right? This...value-neutral on that one, but it affects what comes next. So, I lived in Louisiana back in the late ‘90s, and I was working at a construction company, upholstery [inaudible 01:24], an interior construction company. So, we’d work in office buildings, so we’d tear out the previous setup. When one tenant moved out, another tenant moved in. It's not all we did, but it's a lot of what we did, and it worked.

I was working in the warehouse as a warehouse manager at the time. So, I was downstairs. People would walk in and ask for a job sometimes, and I couldn't make the choice. I'd refer them to the manager who worked upstairs in the offices. So, I was sitting at my desk one day and somebody walks in, asked for a job. So, I called upstairs [chuckles] to my manager upstairs who could make this decision. And his response was, “Are they white or black?”

And I was so flummoxed [laughs]. I couldn't believe that he'd said that. I was just so floored. I just stared at the phone [laughs]. I just stared at the phone for 20 seconds. And then, he goes on. He says, “Okay, just say W or B”. He thought the reason that I wasn't saying anything is because I was afraid of incriminating myself. So, I just stared at the phone, like, I didn't say anything else. I just stared at the phone, and, honestly, to my regret [chuckles] because I should have said something, but I had no idea what to say. I was so, so completely gobsmacked. I couldn't react. He came down.

And the race of the person who was applying doesn't really apply here, right? What matters is, I wish I would have said something. And this was going on 30 years ago, and I still think about that because I wish I had had something better to say. I wish that I had prepared ahead of time, you know, had gone through this scenario in my mind. I just had not gone through this scenario because I didn't expect somebody to say something like this, and so no good reaction. It certainly changed my perception [chuckles] of the manager upstairs, but I didn't know what to say. So, there is my first story.

Second story. I was at a Java Users Group in the early 2000s. And Jonathan, I made sure I looked this up, and Jonathan Schwartz, who later became CEO of Sun Microsystems, was coming to speak at this Java Users Group. So, there was a lot of people there, hundreds of people there probably from the Java community. And Sun Microsystems was pushing their own distribution of Linux at the time. It didn't go anywhere, by the way [laughs], and neither did Sun for that...well, they were eaten by Oracle a few years later [laughs]. But --

WILL: Solaris was all right. Solaris wasn't so bad.

MIKE: They were a meaningful company at the time. I was actually doing a little research on this before the call thinking about it. I guess they had a lot of their business in finance, and the crisis in 2008 did not serve them well [laughs].

WILL: They were pretty well cooked by then, though.

MIKE: I think they were. They were stretching a bit. Which is not really the topic of the story. I was at a Java Users Group...but, you know, it gives you a little context. I don't think he was CEO. I think he was, like, a VP at the time. But he came there, and he was representing the company certainly. And he was going to give out free copies of their new distro of Linux. So, I was like, oh, cool, the Sun distribution of Linux. And he used it...I got one, and it was, like, they hadn't even removed all the branding. I think it was SUSE Linux. They changed a little bit. But they were going to repackage and sell it.

And the person introducing Mr. Schwartz, talking about getting things started, was talking along and made just this badly sexist remark, shockingly sexist joke in front of hundreds of people, right, and thought that this would somehow be okay. And, interestingly, Jonathan Schwartz immediately shut it down. “No.” He didn't escalate it. He just, “No,” and then moved on.

And I've always been impressed with that. He clearly had prepared for that moment, and he knew if somebody said something that was out of line, he would put them in place. And this wasn't somebody who worked for him. He just had practiced this muscle that he stopped what the presenter was saying, you know, that's not okay, and then he went on. And here I am again going on 30 years later, and I remember that. I remember that because I was so impressed with how well he handled that situation. He said something, and he handled it with grace. He didn't make a big deal about it. He just made sure that's not acceptable behavior. We're going to move on.

Both of these are for very specific kind of speaking up. There's an instant, clearly something out of line. Right in the moment, say something. There are lots of other kinds of moments where maybe it's not quite so clear, right? Maybe there's a decision that is complicated, that's not quite clear. And a lot of times, there's technical decisions. Do I get involved in this? And they're hard. But I'm going to reference what I didn't do, which was prepare for that kind of situation ahead of time, and what Jonathan Schwartz clearly did do, which was think about it ahead of time, prepare, and practice so that when he was in a moment where the decision came, he was able to say the right thing at the right time.

Oh, I'm going to start by asking my panel here. What are some times that you wish you had spoken up and you didn't, and maybe sometimes, you know, maybe the opposite? You spoke up when you shouldn't have.

JUSTIN: So, I've got two instances, one that I was present at and one I was not. One I was present at. A company I worked for this guy was very, like, I worked directly with this guy, and he was very, very crass. Like, if there was a dirty joke, he knew it, and he would say it. And most of the time, like, I had known this guy for, like, you know, I was working with this guy for a couple of months. And most of the time, it was like, oh, okay, he's a decent engineer, but, man, is he crass. He's not going to last long at the company. Somebody's going to make a complaint.

Well, the next month rolls around, and they have a company meeting. There was the CTO who was a woman, and it was a meeting of the engineering department. So, there was probably, like, 30, 40 people in this room. And this guy says a joke really loudly, and it was very, very crass, something about humping somebody else, I don't know. It was really, really bad. And the room deafens, and everybody looks around. And the CTO she just starts the meeting right there. She didn't say anything to him right then, but she probably should have.

But, in any case, he was gone within two weeks. So, the next time that she flew out, it was to talk to him specifically and escort him out. So, that was a case of reading the room and understanding who you're talking to. I think that's a skillset that people need to have. It's like, understand the audience of who you're talking to.

And then, the next story I have I wasn't at, but I definitely heard about it. I work for a large bank now. They recently had a large initiative to return to office, and the CEO was having a couple of town halls to talk about it, and somebody very senior...well, not very senior, somebody who was probably two levels above me, but they were still an IC. It's basically architect-level position, super architect-level position. He asked directly to the CEO something to the effect of, you know, you do realize this is going to really ruin morale. And I don't see how this can't ruin morale.

And so, what happened next the town hall went on because the CEO didn't have a good answer for that. And what happened next was that guy's boss's boss somehow heard about that question and fired him in the elevator later that day. Then the story doesn't end there.

WILL: Ask me almost anything.

JUSTIN: Ask me almost anything [laughter]. And the guy later on got his job back.

WILL: Whoa.

JUSTIN: Because it was specifically, you know, it was an ask-anything type of situation. And the boss's boss had overstepped his authority and, well, I guess he fired him for the wrong reason, so...And that sort of --

WILL: Yeah, you better get that constructive dismissal. There are other steps. You got to make him [inaudible 11:17] and do the hokey-pokey [chuckles].

JUSTIN: So, I guess my two stories were examples of be aware of what you're speaking, and sometimes it's worth it to speak truth to power, and sometimes it's not worth it to be very crass.

WILL: Yeah. Damn, I'm taken aback at just the stories of people just being just woefully out of line. I have not had a lot of experiences with people talking like that, I mean, I don't know. I believe you guys. I know these guys are out here. I just haven't...I haven't had that experience.

For me, most of the stuff that I've been exposed to that's been really difficult has been within the bounds of professional communication, right? Like engineering teams, they're all hard problems, and very often, the solution to those problems is not immediately apparent. And you're choosing without prior foreknowledge between two equally good strategies, and that's where you see a lot of people butting heads. Although I will say I talk a lot of shit, and I've gotten better at it. I've gotten a lot better at it. But I have a ridiculous story if you guys want to hear it.

MIKE: It's story time.

WILL: When I was younger, like, when I was young, right, I was, like, 25 and I was at the University of Utah. I was at the Graduate Student Advisory Committee. So, we were a little selected group of grad students that got to sit in faculty meetings, which was a profound waste of time but very interesting. And we got to talk to, like, faculty candidates would come in and be like, “Okay, well what are the grad students like?” And we had no power. It was like a nice little pat on the head committee, but I appreciated it.

And we got to meet people. And one of the people that we got to meet when he buzzed by was very famous in certain circles, University of Utah alumnus Alan Kay, who is just a wretched, dried-up, useless, blowhard. Like, you know Alan Kay's name because, Alan Kay, for the entirety of my lifetime, has completely abandoned any form of useless work to tell the story of Alan Kay ad nauseam.

We had a lunch meeting, me and a bunch of grad students. Like, you had the privilege of interacting with diet Will, who most of his fangs have been pulled...I am so much...I'm sweetness and light. 25-year-old punk rock Will was not as retiring and diplomatic as I am now, and it did not go good. He spent...I don't even remember what we were arguing about. We were arguing about, like, operating systems, like, in the weeds of the programming languages. And I was a pretty serious, like, compiler operating systems nerd at that time in my life. I didn't have a lot of time for Smalltalk, Smalltalk the language, rather than small talk, like, chitchat. Like, we were deep in it, and Alan Kay he was ridiculous, really bad.

And it wasn't only me that he made a really poor impression on, but I'm not going to out anybody else because they had the good sense to keep their mouth shut. And he spent the rest of his lecture in front of the entire university...I got, out of a 45-minute lecture, I got about 15 minutes directed at how bad I suck. He didn't mention me by name, but everybody who knew...and there was a blast radius among my faculty advisor. And other people were just, like, Will, what did you do?

And the right thing to do is to just let the old man flap his gums at you because it doesn't matter. Oh, he sucked so bad. Sorry, I can't read Alan Kay articles anymore having interacted with the guy and seen the absolute wasteman of his professional contributions for the last 50 years. Anyway, moving on.

MIKE: So, you and Alan Kay don't get along [laughs].

WILL: There is no we [laughter]. I doubt that he has thought about me twice in the intervening 20 years. He’s probably dead now.

MIKE: That's the worst moral to take from this story [laughs]. The real moral...total sideline, Calvin and Hobbes, the greatest comic ever written, there's an arc about Snow Goons, where he builds monsters, and they go out and wreak havoc. And [chuckles] the final line of the panel...[inaudible 17:07] if it's a panel. The final strip of the arc is Calvin learning his lesson. And Hobbes asks him, “So, what did you learn from this?” He says, “Snow goons are bad news [laughs].” And Hobbes observes that he's actually learning nothing from this [laughs], and that's how he likes it.

WILL: Well, I actually think it segues into the general strategery for these sorts of disagreements, which is, at that time, I hadn't developed this muscle yet, but, like, now I've got it, and I still talk a lot of shit. But I get along pretty well with engineering organizations, even when we don't agree and I don't get my way...is when you can tell that things aren't going your way.

This conversation that I had with this rich, old fossil, who's completely high on his own supply, and he was in an orbit somewhere completely detached from the actual state of the art in 2006 or whatever, there was no reaching that guy. There was no communicating with that guy. There's no conversation we were going to have. So, just vibe out, eat my expensive, free lunch, let him ramble, and move on.

More recently, I have learned my lesson. And there was a big debate that happened a few years ago. And I was on the losing end of it, where they were getting rid of...we had this Java server, like, I work for a big retailer, like, a big retailer. We shop there. And they were getting rid of their...they had this big initiative to get rid of this Spring Boot application server, which was sort of, like, our gateway behind the firewall downstream services. They wanted to get rid of all of this old, crusty Spring Boot infrastructure and replace it with team-specific Node JS microservices, I guess you'd call it, like, gateway microservices. And every team, every functional group would have their own.

And it was a dumb idea. It was dumb. Like, why do this? Like, why...it would make it slow. Because it was based on the false premise because that was in JavaScript and many of our other systems were in JavaScript, it would make the engineers fungible across these radically different domains, which has never happened. It's two years on. It never worked. I was like, why? This is going to be slow. It's not going to buy us anything. It's going to slow down application development. You guys don't have the appetite to pay down the tech debt of moving this stuff over. So, you're going to build a new service, but you're not going to get rid of the old one. And they haven't.

MIKE: Oh [laughs].

WILL: You know? All this stuff happened, and I was right about everything. And I told them, and they said, “We don't care.” And I said, “Cool.” [laughter] Disagree and commit. Because, in the end, if I had continued paddling the boat the wrong way, we would just spin in circles. And it is an absolute guarantee that we will fail. If we can't disagree and debate and get at everything back and forth, and then we're going to say, we're going to go, and, like, we're going to go. And, as such, we did not get an optimal solution, and there have been consequences for that solution, but we are moving forward slowly.

And I still have a good reputation in the organization. I got my ‘I told you so’ points. I got to dig in a time or two privately. But I'm fine, and they're fine, and the project is fine. And was I right? Yes, I was. Were they wrong? Yes, they were. Does it matter in the end? I don't know. Maybe, maybe not. But that's how you have to deal with it. If you are on the losing team, take your L, and shut up [laughs]. And that's how it goes.

I have a friend, a good friend of mine. He got into a similar slapfight on some GraphQL implementation architecture, and he couldn't take his L. And he was a contractor, really senior guy, made a lot of money, but he was still a contractor. And contractors can be fired very cheaply, and he's gone. And when he was going to get hired back...he subsequently got hired back. He's an excellent engineer, and he has been doing great work, both before and since. But he got into this fight with a staff engineer and a director, and they got his ass. He was done. He was out of work for three months.

And when he was trying to get back, they were on the phone with a manager who was trying to bring him back on. He's like no, man, get rid of this guy. He's a jerk. Get him gone. We don't want him. You don't want this dude. And it's a fine line to walk. If you want to have a career and still talk your shit [laughs], you got to figure out how to balance it.

MIKE: Interesting. You talk about reading the room, and you say that's the key. And the room may be you and one other person. Being attentive to the person or people you're talking with is a huge amount of the communication, and that's not easy for everybody. I mean, there's people who are neurodivergent who have a really hard time with that. And so, that's going to be something you're going to have to work on and work on and work on because it matters. That's where a lot of the communication is. I think there was a ton of insight from your stories, Will.

WILL: [chuckles] Well, I've been talking shit for a long time.

[laughter]

TIM: I do think these kind of moral boundaries are a little bit endemic with the tech industry, and these companies are in a public position where it gets...like, okay, so to go back to your original story there, Mike, how you've got a kind of construction related business.

MIKE: Yes.

TIM: And I hate to say it, but it is kind of like a stereotype sometimes, you know, the street construction workers, there's a certain level of crass that just comes with that industry. In some places, it's just more publicly accepted, and that's unfortunate. And that's not at all desirable or a place where you want to work. But within the tech industry, you just go back 10 years, and there's patterns of it getting...okay, so, Uber, the bro culture in Uber with...I can't remember...there was a couple of executives that ended up getting canned because of sexual harassment and discrimination and just kind of dehumanizing pattern of events.

The same thing happened in Activision, if I recall. They had that bro culture in their executive leadership that eventually got outed, and people were let go. There was Google. I think Google got caught about 10 years ago just kind of...it was an internal memo that got leaked. And one of the executive leaders were disparaging the gender wage basis, kind of things like that. And they made some really insensitive comments in ink that got leaked in a memo that...it's just something that happens in the tech industry. And when it gets out, there's a zero-tolerance policy about how it is accepted in the industry. I think that's one good thing [laughs].

Like, if someone in our executive structure were to do something stupid like that and it got leaked out, they'd probably be on their ass in a couple of weeks. And it's just when to speak out. Sometimes in the tech industry, the industry will speak out for you [laughs]. You will not be accepted otherwise. There's a pattern within the tech industry of this kind of stuff of getting, how would you call it, a Darwinian approach to how you're going to be received [laughs].

WILL: I would say it's corporate size, like, entrepreneurs are a different breed of cat, they are. And they're not going to act right, but they started the company. So, you could kiss their grits, and it's good and bad. Madness and genius go hand in hand. And people with an entrepreneurial gene, or personality disorder, or whatever, they're out there, they just are. But you have to leave a substantial portion of who you are at work.

Like, you guys have never heard me tell you a dirty joke. And let me tell you right now, I love dirty jokes [laughter]. Like, I love them. I love them. You've never heard one out of my mouth. Even now, like, I'm not on the clock. I'm doing this one for free, but sorry, nope. Not going to do it. Sorry. Like, I might have gotten this podcast a PG-13 rating with my trash talk. But I'm actually not going to swear over the rest of the podcast because that was enough. That was enough. That PG-13, woo. Mm-mm. Not a word.

TIM: I don't know. I think you get the Silicon Valley or slopes around here. You get that attitude with a lot of these kind of startups. But I've met some really impressive people in the leadership realm who are...I think my favorite one that I've ever met was Scott Farquhar from Atlassian. At the time, he was the co-CEO of Atlassian. We had an opportunity, me and a couple of my peers, to have lunch with him. And I've never met a more straight-to-the-point, like, honest, no-BS kind of person.

He wasn't propped up. He wasn't inflated, wasn't anything like that. I was like, “I really like your stuff, like, what you guys are doing and how well you guys have developed. But you guys just don't support HIPAA certifications,” at the time. And I said, “What do you guys want to do about that?” And without even skipping a beat, his answer was, “Nope. We're not going to do that, at least not right now. I can tell you right now that's not on my roadmap, and I have no interest in doing it.” Complete respect.

I've never had a CEO executive reply in a straightforward manner. They're always diplomatic. They're always politicians with how they give you their answer, the majority of them, what I've got. But that guy, he and, I think, Mike Cannon, those guys, straight shooters, just some of the best guys I've ever seen. Really a good pleasure to be with those guys.

WILL: But that's a unicorn of a unicorn because I think that guy was the founder, right? Like, he was the dude, right? And so, entrepreneur. He got his business to, like, the ramen profitability, paying the bills. This is a thing. And then, he took it to a multi-billion-dollar level. And so, I mean, that's a unicorn of a unicorn. But even that guy there's a difference between the CEO that comes up through the ranks, and I was a director here, and now I'm the VP here. Now I'm a CTO here. Now I'm a CEO here. And you just sort of jump up, jump up, jump up. Like, this guy, it was his shop. And it is such a refreshing difference.

And it's why Founders-CEOs are still coveted to the degree they are. Because a professional manager who was appointed by the board and beholden to the board is just going to be like, “Well, we have a lot of quarterly initiatives, and we can get that on our roadmap. All our partners are valued,” and just some more word vomit.

So, your eyes glaze over and you wander off, whereas the founder is just like, this is my shop. Yeah, I can't do it strategically. It doesn't make sense for our company to take on this amount of infrastructure to support HIPAA, which I know a little bit about, and it's a tremendous amount. It's a tremendous lift, and he's just like, I can't afford it. And he'd just be like, no. But yeah, I don't know, man. There are a few entrepreneurs, and there are ducks [laughter].

MIKE: Let me observe something that's come from a number of these stories. You talk about complicity. Tim, you talked bro culture taking over sometimes. When that happens, there are people feeling uncomfortable. Obviously, there are people being mistreated, which is why it eventually surfaces and becomes a scandal is because there are bad things going on. But one person starts seeing it happen, and they don't say anything, and then they decide to join in. They follow the pattern, and then it grows. You end up with a bunch of people...you come in, and humans we absorb the norms around us, and we tend to follow them.

And some of us have an inherent punk rock attitude [laughs] and maybe naturally resist those norms. And some people are more prone to say, “Hey, I'm going to follow what's going on here.” In any case, there's a human tendency, and we can see this outside of...we can take this to extreme examples. Obviously, people cite fascism in Europe, you know, a hundred years ago, where a lot of people just kind of went along with it, even if they were uncomfortable.

There's this tendency where when a group's doing it, you don't say something. And it's interesting because even if you're reading the room [chuckles], maybe the room says, “We're all okay with this, and you're not.” And that's an uncomfortable situation to be in. So, what are the boundaries? It's not just about protecting your job, right? Because maybe it's worth leaving your job. When is the time when you say...well, so I'll take this further. If you were working for Enron and you didn't say anything, that probably is a black mark on your resume. There's times when there's clearly a line that's been crossed, and you should have said something. It was worth losing your job over. So, where do you find those boundaries?

WILL: Yeah, that's tough. I mean, I think, for most commerce, like, a good bright line is legality, right? I will not go to jail for any job. There's no paycheck. Like, I won't do it. There's not a chance. I have worked myself...personally, I've worked for defense contractors, many weapons of war, you know, in the hands of the United States military. We're killing people, and I have some sliver of accountability for that. So, is it morally gray? I have to say yes. I have to say that's a moral gray area because it's just the act of war, the act of violence, of taking human lives. But is it wrong? Ooh, no, I can't say that. Can I say it's right? Ooh, not purely. And everything exists somewhere in that continuum.

So, I'd say legality, like, that's a bright line. If you find yourself being asked to do something that is not legal for a job, for a paycheck, you should stop. And so, then it becomes a situation where if you disagree with something, I think you should argue, like, you should raise up your hand and say, “Hey, I don't think this is cool. I don't think this is okay. We shouldn't say anything.” And then, to the degree that you're empowered to act, I think you should act to do the right thing.

And then, when it's out of your hands, when it goes to your boss's boss, and then you're like, nope, this is this, then it becomes a situation where you have to ask yourself, do I stay, or do I go? And that's, I don't know, man. I have left jobs, and it was very, very difficult because...well, I'll tell this story.

So, I ran my own software company for 10 years. We made, a bunch of mobile app fitness products. I started it with a good friend of mine. We built it up. It was a seven-figure company. It is probably a seven-figure company. I had a significant equity stake in it. And we had some business relationships and some business arrangements such that I thought that the company was basically limited.

We had an artificial ceiling, where it was never going to be anything more than a job. And I had to ask myself, is this a job that I want? And the answer, to me, was, no, we should sell the thing, get a nest egg, get, like, a modest middle-class retirement out of it. That's what we would wind up with. We would wind up with a robust 401k. It was on the order of, like, [inaudible 34:55], think about it like that.

And me and my co-founder we had diverging of opinions. My co-founder saw it as this is the job I want. I'm the CEO. I can do anything I want. I'm the king of my little kingdom, right, and I love it. And I'm like, ah, but I'm not the king, not that it wasn't cool. I did a lot of cool things that you can't do in a big company. But we had this fundamental diverging of opinions. I had to leap. I had to go, and it did not go good. But that's sort of one of those forks in the road where it's like, okay, this is dumb.

There were other things that I mentioned where it's like, yo, we're making these technical strategic decisions that I think are really bad. I don't think they're going to work out, and they wound up not working out. But still, I'm just like, okay, well you're writing the checks, and that gives you the right to say, Yes, No, or Maybe. Like, I'm delegating that to you for the paycheck. So, you get to make the call, and I let you make the call. And then, this other thing where it's a strategic line of the road where it's like, I get this one life, and I'm not living it the way I want to. If we can't come to an agreement, I got to go, and that's that.

And so, how do you differentiate between, like, I don't think this is a good idea, but okay, which happens I think to most of us a lot. You can think about something you did today, where I'm just like, ah, I wouldn't have gone that way. How do you make that determination?

TIM: I see it happen all the time, but it never escalates to the Uber level of the bro culture. But I think every software development shop we always know there's a group of brogrammers that they act a certain way, and most of the time it’s just harmless. They're their own kind of way. But what happens is it just turns into a click, just like every other workplace. Whereas this time it happens to be software engineers, and they feel that they have the momentum and the numbers to act a certain way without consequence, and that turns into the bro culture.

And those always self-destruct over time because, eventually, one of those guys is going to do something stupid that HR is going to come down or the law and order of public...your PR is going to kill you when you do something that bad. But I think when you're in the engineering workplace, you just have to be self-aware enough to be able identify who the brogrammers are that may have a negative click approach or impact on their peers, fellow employees. And sometimes you just kind of have to steer clear of those people. Otherwise, you just add more fuel to the click, and then you're just, okay, great. We've got that going on, and yeah.

WILL: I don't know. I think the click-ish culture is, I think, I mean, I’ve run into some brogrammers here and there [laughter]. They’re really few though, to be perfectly honest with you. I’ve run into the bro boys. I’ve run into the bro boys in the sales team. I’ve run into the bro boys [inaudible 38:22]in marketing.

TIM: The sales engineers.

WILL: Less so MBA dudes, those guys can bro down. But devs, like, bro devs, pretty rare. It's just, like, the psychological skillset that you need to be an effective developer it doesn't sit well with the bro dogs. There's a lot of sitting still that you need to master if you want to do that job and brogrammers do not, as a general rule, excel at sitting quietly for many hours in a row.

MIKE: Well, I think that there may be some simple rules here. You said legality, easy. Don't break the law. There's another one, and this goes back to something we've talked about a lot: psychological safety, treating people with respect. If somebody's doing something that is dehumanizing another person, I think that can be a bright line. Like, no, I'm not okay with that. I'm going to say something, and that's my cue to leave.

I did leave a company a number of years ago because their marketing tactics were maybe legal, but certainly not ethical, taking advantage of people to the point where I just couldn't stomach it, so I left. When you join in that, by proxy, once you're dehumanizing, you've lost some of your humanity [chuckles]. And you've said, well, this kind of person is better than this kind of person. That kind of thinking doesn't have a good end. And I think that you can recognize that, look for that, and see that as a line.

I think there's a lot of things though that we haven't talked much about. It's just a technical decision, and we're going to have a disagreement. And, in most of those cases, you don't have to win [laughs]. You’ve been saying, Will, I will raise my opinion. I will share it. I will emphatically argue for it, and then I will agree with the group and [laughs] go with the decision that's made. A lot of times it's more important to have a decision than the right one, than the optimum one. Better to be going...you talked about rowing in the same direction.

If you're generally going in the direction that will get you where you need to go, you'll eventually get there, and it's a whole lot better than going in circles. So, I think there's a lot to be said for put in your paddle and just rowing. After you have clearly and effectively shared your guidance as to which way you think we should go, you get down and you just follow the leader.

WILL: Yeah. I mean, that's it. The only way it works is if you trust me to do my job and I trust you to do yours, and that's it. Like, as long as you understand where I'm coming from and why I wanted to do it, and, hopefully, I understand where you're coming from and why you're choosing this other thing, then that's what we got to do. And in terms of like, yeah, you can't work at an abusive or dehumanizing place. You can't do it for long. If you are in one of those places, you've got to get out.

I was not in this meeting, but a meeting was reported to me very recently at the big corporation that I’m consulting for that one of the VPs went in an AMA, and he said, “There's not a single individual contributor in this org that is operating at a staff level.” So, this is, like, hundreds of engineers, including many staff engineers on a level. And he's like, there's not a single one of you that's at that level right now. And I'm just like, woo, that's spicy, man. That's bold. Sure that's not you? [laughter] It was tough. It was a tough meeting. I was just like, wow.

MIKE: It's interesting you've said maybe it's you [chuckles]. If you're leading an organization and everybody is underperforming, it's usually something systemic. And you can blame people, or you can do something about it [laughs], and that's on you, I think.

WILL: If you gave me a hundred chances to either develop, recruit, or retain, like, a staff-level engineer, I could get one. I could give one of them, man, come on [chuckles]. That's just me. It is the little, old me [chuckles].

MIKE: That goes to the creating a culture of respect, that it’s just absolutely vital to have healthy dialogue.

TIM: There was one thing you brought up, Mike, the process of every time...so I had the experience. We had a new mobile app up and running. It was doing well. It was a primary mechanism during COVID for patients to get their COVID-19 testing everything. It blew up bigger than we thought. After we kind of hit a peak run time, the principal engineer brought up the idea that they wanted to move all the microservices into one monorepo. And the logic behind it, which was baffling to me...because I was on the DevOps and the CloudOps side of it at the time. The logic behind it was because Google that's how they do it. And this individual pointed out multiple big people out there who do everything they do in a monorepo.

But from a DevOps perspective, we had built custom pipelines in this whole ship-it mechanism that was fine-tuned, and we were easily bootstrapping a new microservice, getting it up and running, delivering a feature, all within a sprint, which is mind-blowing. The system is working. And then, they're like, oh, let's flip it to a monorepo because we don't want to have to keep doing patches on each repo on...And I was like, okay, just use Dependabot. Do something to help automate this thing. And it got to the point where engineers were getting very colored and very open about their feelings one way or the other.

And I don't think there's any argument in the world that will swing one way or the other that a monorepo or a microservice model is the right way to go when they're so religious about, you know, they have these stakes in the ground. If you've done this for a while, you just need to learn to step back and know your role as either a contributor to the conversation...this is my opinion, and this is why I think we should do X, Y, and Z. But, at the end of the day, it's the decision maker who's going to make the call, and you've given them the best information that you can. The decision maker makes the call, and you run with it.

And some people got so bent out of shape that one guy decided to quit because it didn't go his way, and it wasn't because...I mean technical reasons one way or the other, but it was more of an emotional decision for him to get out of there. And it's like, I don't think that was the right hill to die on because every piece of software expires in 10 years. We always replace something with a new something after a certain amount of time. So, what's the point? Just do the job and then move forward and let the decision maker who's hired to do that do their job.

WILL: Well, I'd say...I agree with you and, yeah, totally agree with you. But I would say this, if there is a series of technical decisions, strategic decisions at the management level where they're making these broad strategic decisions that just make your job suck, it just sucks to go into work over and over and over because it's just, like, this sucks, then you should find a new job. You should find someplace where you will thrive, and you will feel comfortable. And sometimes it's just a reset, where you just need different tech debt [laughter].

TIM: Yeah, that’s true.

WILL: [inaudible 47:27] Because we've all got our stink. But at the same time, you shouldn't do it in a huff. You should do it because, like, you made this decision, and I said, “Don't do it,” but you did it anyway. And now I'm stuck in this living nightmare every day. And that's a great reason to quit a job.

JUSTIN: So, I know we're towards the end of the podcast. But I did want to kind of get you guys’ feel for, you know, all of us are kind of more senior, right?

WILL: Allegedly.

JUSTIN: Allegedly. What are some good strategies to encourage discussions like that? Because, oftentimes, we could get to the point where we think we know what the idea is and everything and where we want to go. And we're trying to steer the direction of the discussion and go for the decision-making process that we think is correct, but oftentimes, we need other people's input. We need to encourage the next generation of engineers to speak up. So, what can we do? What are the strategies there?

WILL: I was in the debate team when I was in high school, and when we would do our little practices, everybody was expected to be able to take either side of the argument. And I think in an engineering discussion, it's almost never black and white. There are going to be pretty clearly good and pretty clearly bad decisions, but it's almost always a trade-off between two differing sets of ideals.

And I think in every good-faith engineering discussion, you should be able to articulate both sides. This is why I think your idea makes sense. This is why I think my idea makes sense. These are the fundamental values that we are fusing between for this and this. If we go this way, we get this. If we go that way, we get that. What is it that we value? And if we value set A or set B, then that dictates our strategic direction. But I think everybody should be able to do that in a good-faith discussion.

TIM: I had an awesome SVP once, and the way he did it when it came to these big decisions that had implications about framework changes, or big wide impact, he said, “Okay, I'm going to set the discussion up with the DACI framework.” So, Driver, Approver, Contributor, and Informed. “Whoever wants to do this thing, that's the driver. I'm the SVP, so I'm going to approve one direction or the other. And then, I want subject matter experts to weigh in as a contributor.”

And his advice to everyone who was a contributor was, if you tell me all the pros about your idea and you can't tell me one con, then you're way too religious about your perspective, and I don't want your information. You need to be able to tell me the pros and the cons of your position and remain objective. Otherwise, you're not furthering the conversation and giving me valuable information. And just setting that framework right up front really led to some productive decisions that maybe they were wrong, that's fine. But we were able to get through that process so fast and realize we were wrong quicker than debating it for four months and then finally getting there.

JUSTIN: I like that. I’ve kind of heard about that sort of thing before, but you put it together really well. Like, you have certain roles, and you expect people to play those roles, and encouraging people to play those roles makes sure that you get that variety of opinion, and it encourages people to fulfill that role. If you have a junior engineer, it would be like, “Hey, guess what? You are going to be doing this role here. I want you to prep for this meeting, and this is what we're going to be talking about.”

TIM: Yeah, 100%.

MIKE: I really like that, too. Come prepared to take the stance against your position.

JUSTIN: [laughs]

MIKE: And I'll often do that for myself in a meeting because I don't feel good about a decision that's not had any negative feedback. It may still be the right decision, but a lot of times you know it's the right decision because you hear the arguments against it, and they're all bad [laughter]. You have to hear both sides and sometimes that actually...in fact, oftentimes, that makes it really clear what the right decision is because you start arguing against it and being like, yeah, you're not making a very good argument, you know, it helps. And sometimes you dig and you find something that genuinely is a good argument and you realize, hey, well, maybe I wasn't coming in with the right idea.

WILL: I just default to constructive laziness. Like, what's the least amount of effort that I could do to get my job done over the long term, over broad, like, whatever my planning horizon is, which is not sprint level but quarter, if not year. And whatever's going to win me the quarter with the least amount of heartbreak and stomach acid, I'm doing that. Past a year planning becomes real hopes and dreams and moonbeams [laughter]. But you can plan a year. Like, a big organization can plan a year. [inaudible 53:23] plan two years. Don't [inaudible 53:25] your fairy tales [inaudible 53:26] [laughter]. Tell that to the investors. Save it for the board meeting, dude.

MIKE: I think we've covered some good material here. We've talked some about how to deal with just straight-up inappropriate things that shouldn't be said. It's okay. Speak up. Shut it down [laughs], and it's worth it. We've talked about a lot of these areas where it's not so clear and that we should say something politely [laughs] but emphatically.

There's a recent idea that maybe take a counterpoint so that you're clearly looking at both sides of the issue. And just the importance of not dying on every hill, of getting with the program, that a large organization or an organization of any size is only going to go somewhere if you're all going in the same direction, and push it forward even if it's not quite the way you want to go. And as long as it's not making you hate your life and your job, it's probably okay. You can live with it.

WILL: Yeah. If you hate your job, you should go. Nobody should spend all day, every day at a job they hate. You should go. If you hate your job because you didn't get your way, that is a deeper concern. I think you should probably introspect.

TIM: And you're still going to get paid. So...cool [laughter].

WILL: Yeah. Yeah. I mean, many times I have returned to my fundamental axiom, which is that I get paid either way.

MIKE: This is some good stuff because this is hard. Going back to the beginning of the call, I was in a situation. I didn't know what to do. I was young [laughs]. I was young and foolish. And I wish that I had today's experience then but you don't.

Hopefully, you've learned some things. When you're in your situation where you don't want to be regretting it 30 years later, thinking about what you could have said. Thank you everybody for sharing. I think we're in a good spot.

Until next time on the Acima Development Podcast.

In this episode of the Acima Development Podcast, Mike, Will, and Kyle explore the intersection of language learning and programming. Mike kicks off the discussion with a personal anecdote about studying German in school, noting how little he’s used it since, and how learning Spanish as an adult through Duolingo was more rewarding. This transitions into a broader discussion about how the process of learning any language—human or programming—reshapes cognitive patterns and makes it easier to acquire future languages. They draw parallels to AI, explaining how training models on multiple languages makes them better at understanding new ones, suggesting that the mental “training effect” of learning is more important than the specific content being learned.

The conversation then shifts toward practical advice for aspiring developers. Will emphasizes the concept of “the lore”—the unwritten, experiential knowledge in software development that can’t be easily taught in schools or bootcamps but must be passed down through mentorship and hands-on practice. He shares his own journey from fixing computers to compiling Linux source code at 15, which gave him a massive edge later. They all agree that building projects, no matter how scrappy or simple, and putting them out into the world is the most effective path for beginners. This “build and share” approach helps attract feedback, builds credibility, and accelerates learning in a way that no tutorial can match. The group also acknowledges the tough job market, especially for juniors, and suggests finding mentorship even in “sketchy startups” as a legitimate way to gain real experience.

Finally, the hosts reflect on how the landscape of software development is evolving in the era of generative AI, emphasizing that while tools are getting smarter, the core need remains unchanged: people who can think clearly, break down problems, and express logical solutions. They argue that learning how things work under the hood—whether it’s operating systems, threading models, or build pipelines—is still essential for becoming a great developer. Will humorously champions “vibe coding” and encourages makers to lean into their curiosity and creativity, while Mike reminds listeners that even though the languages may change, the deep thinking and hands-on experience involved in building and debugging remain the bedrock of the profession.

Transcript:

MIKE: Hello, and welcome to another episode of the Acima Development Podcast. I am Mike, and I am hosting today, as I have usually been doing. With me today, I have Kyle, and I have Will Archer. Actually, I have Kyle Archer and Will Archer [laughs].

WILL: No relation.

KYLE: No relation here.

MIKE: No relation, but they sound alike [laughs]. And, sound like, speaking of sounding like, I'm going to start today by talking about something I did long ago in high school. I took a German class. I took four years of German. I started in junior high, and then got to four years of German. And you know how many opportunities I've had to use German since I got out of high school? I think the answer is zero [laughs]. I think the answer is zero.

Actually, no. I ran into...maybe four years ago, I was visiting my parents, and I was going on a walk with my parents. And my dad ran into somebody who he knew who was from Germany, and he said, “Hey, Mike, so you speak German.” And I froze. It had been, like, 30 years [laughs]. I’m like, “Probably not right now [laughs].” I didn't have much software. If I'd taken a little bit of time, I could have done something, but no.

If I see, like, on a packaging, like, shampoo bottle or something, you know, and they've got instructions in German, you see that, and you don't even think about it. You just start trying to read it. You’re like, oh, I recognize that, and you start reading and try to make sense of it. That's about my entire experience using German is reading the back packaging that happens to have a translation in German, you know, since I got out of high school.

However, a few years ago, I said, you know what? I want to learn a language that I might actually use [chuckles]. I started using Duolingo, the language learning app. I even have a paid account now, because after a while, I started using it more. I've been doing that pretty consistently. I think, daily for the last four years, I've been learning Spanish. And I actually completed the course a while back, but they give you just review, and I've been doing that. And I got everybody in my family doing it, too. So, we're doing it together. And I went through it the fastest.

And I've read this: it turns out that after you've learned one language, it's easier to learn another. And, in fact, they actually recommend that if you want to learn a language very different from your native language, for example, for an English speaker, if you want to learn an East Asian language, if want to learn Chinese or Japanese, you learn it faster by learning a European language first.

WILL: I believe that.

MIKE: Because it trains your brain to think about things in a different way, to think about language differently. And once you've done that, you're primed, and it's easier to switch to --

WILL: So, you’re saying if I wanted to learn Chinese, right, I want to learn Chinese, it has very few loanwords or commonalities, then I would go faster if I learned Spanish, and then went to learn Chinese, than I would if I had only learned Chinese.

MIKE: That’s right. Which sounds crazy, right?

WILL: It does sound pretty crazy.

MIKE: It sounds crazy. Like, how could you possibly learn Chinese faster by learning another language first? But I've read this. I don't have the source. I don't have it right in front of me, but I have read this before that it actually is faster to learn a different language that’s more similar to your native language. So, you know, again, if you were a native Chinese speaker, it might be faster to learn maybe Korean first and then go learn a European language because it changes the way your brain thinks. I think that's fascinating.

Maybe the source that I read is totally wrong and leading me astray, but there's some intuitive sense to it that it trains the way you think. And regardless of whether it's universally true, there's probably some meaningful truth there that there's some training about how to think that is different from learning the specific syntax.

To take this further, AI models have been doing...they've run some of them in lots of different languages. So, they just train a text on lots of languages, and specifically, they train on text that's been translated from one language to another. And the fascinating thing that they find is the models that have been trained on one language or on many languages actually are better at learning a new language than they were at learning the first language.

So, if you have a language that can translate between English and German, and English and Spanish, and Spanish and Turkish, and just name your language pairs, and then you you add a new language to the mix, like Igbo or something, then if you've already had those other languages, your model will be faster and even do better than it was if it was just trained on that language. And there are a couple of reasons for that.

First of all, you have more data from commonly represented language pairs. There are probably really common ones, like English-Chinese, for example, you know, where there's a lot of stuff translated out there. And so, you have a higher body of language between really commonly used languages out on the internet, but it's more than that. There seems to also be just some universality of learning to translate that makes the models better. When you've learned between a lot of languages, you actually improve them all together.

WILL: That makes sense. That makes a lot of sense, to me, actually, I think. Because you're dealing with, how do I put it, like, you're dealing with...I don't want to use the word means but ideas and a more robust mapping of linguistic phrase and the poor, I would say, human idea. We're all people, and to a degree, I think language shapes the way we think, but only to a limited degree. And so, the more mappings, the more cross references you have between a turn of phrase, or a word, or something and the fundamental idea of the human condition that you're trying to express, then I think the more, you know, evocative and robust your translation between the two things would be.

MIKE: And it's interesting you say that. You can say there's this human experience that it gets closer to you. And they actually talk about, also in AI research, the embodiment problem, which is that some concepts have been really hard for AI to learn, partly because they don't have the human experience. I believe the...I can’t remember the philosopher who did this...was it Plato? Plato and the cave? I'd have to go look it up.

WILL: That sounds about right. That sounds right.

MIKE: So, imagine you're sitting in a cave, and people are broadcasting pictures of something up in the wall in front of you. And so, your whole experience comes from that broadcast, you know, looking at this shadow show. It’d be difficult to learn some aspects of the world. You could learn a lot, but it'd still always be second-hand, and that's how these AI models are learning. They don't have the body, typically. I mean, there are some robots now with, you know, physical bodies that can go out and live or experience the world. I'll use the word live very loosely [laughter]...to go out and experience. And so, there's some ability to learn there.

And, in fact, simulation is hard for robotics. You know, it's a lot better if you use a real robot, but then you break your robot. It’s expensive [laughter]. That embodiment problem is tricky. It's hard to learn if you don't have those experiences. You're suggesting there's this human experience that is independent from the modeling we use in language, and that if you model it several different ways, you actually get closer to the human experience, which is a fascinating idea, which we won't dig too deep into, but it's still a fascinating idea you said there [laughs]. I like that.

I mean, we haven't even hit the core topic that we're talking about today because this initial idea is so interesting that leads to it. I wanted to ask, so present it, if you were going back, if you could go talk to your old self that was first learning to code, what would you tell them? How would you have them do different things differently? And I think this idea of language and this idea of experience and language being different is a fascinating analog because it's another language, kind of language, that you're using to accomplish a similar purpose, but in a totally different context.

WILL: So, it's been a long time since I've learned how to do this, and I'm always, I don't know, I'm aware of, like, you know, how much unconscious knowledge that I've just sort of absorbed and accumulated and I deploy without thinking in just the daily process because I'm learning all the time, right? The machine never stops. The treadmill never stops [chuckles]. I'm always learning new, different more horrible ways to break machines.

But I have brought on board a fair number of junior developers. And the one thing that I look back on the most is what I refer to, colloquially, as the lore of just these are things that you have to know because they're not your job. But you're never going to be able to effectively work and to make an artifact that functions without the lore, which is just, like, anecdotally, I remember...

So, my first job I was working in a computer shop. I was a computer repairman. And I got exposed to a lot of the lore, you know, back when it was really like a dark art. It was really like a guild. When I started programming computers, there was no internet. I mean, it existed, obviously, but, like, there was no internet. There was literally no Google, nothing.

And so, I got in this computer shop, and I got, like, some graybeards who taught me some stuff. And then, I got exposed to Linux, like, as it was rolling out. Like, as it was coming out, I was getting on Linux, and I had to, like, I ordered...I purchased a slug of six or seven CD-ROMs with the source code to Linux that I had to compile myself [laughter] as a 15-year-old kid to have Linux. And it was such a hateful, abusive process.

But then I went and I got into college, and I got onto the programming team. We had a little competitive programming team. And I was wildly more effective than any of my peers because I had this lore around Linux and making the shell do things, reading files from the shell like basic commands. How does a directory work? How do you mount a device? Like, all this crazy stuff that is both absolutely foundational to the process of creating a running, functioning artifact, at least for me, and is not the kind of thing that anybody would bother to write down.

And so, that, to me, is the biggest and craziest thing, and, like, how do you get? How do you get the lore? And that is the thing that almost...I don't know of another way other than being in the company of professionals. It has to be passed, almost literally, from hand to hand. You need to be, not necessarily in the room, but in a distributed workforce, we're finding you need to be either in the room or in a very, very close virtual analog. I mean, it has to be live, one-on-one transfer of information between you and me. I think a one-on-one Zoom call would be sufficient, but I think a webinar or some analog that wouldn't do it.

And so, then it sort of leads into this problem of, like, I think, in 2025, we're being extremely hard on junior developers in this year, at least. At this moment in our industry, we are being very, very, very hard on junior developers because productivity is going up. And people, you know, productivity wants to go up. Salaries want to go up, which means the demands to get a salary job are also going up.

So, if I'm starting from zero, all that in mind, this is sort of, like, the background of like, okay, this is the problem that I need to solve. I know nothing. I need to get up enough so that I can demonstrate enough value so that I can sort of start the flywheel turning where I work, and I get paid to work, and then I get to work more, and I get paid more, and then that virtuous cycle where I can sort of put energy in, and I get enough money to, like, go full-time, or part-time, or something like that.

So, what I would say to do is...I suppose that’s the goal. It does assume the goal is to go pro. The goal is to go pro, to have a job, have a career, you know, like, really do it. If you just want to go out and have fun, I think there's never been a better time to go out and have fun doing programming. I think there are a lot of places, a lot of frameworks and stuff, or a nominal fee people will be happy to just get your stuff out there in the world. I know C was a Rails shop, and, like, what are the guys who, like, you just sort of put your artifacts up, and they just manage all the service stuff for you.

MIKE: Oh, Heroku?

WILL: Yeah, Heroku. There are places like that, I mean, analogous ones, I'm sure, for, like, you know, JavaScript, or Node, or React, or whatever you want to do. And so, those kind of places are really good. You've got your generative AI sort of coding assistance that could go out and do it. So, honestly, I would just make things and get them out into the world in the dumbest way possible. And this would be really hard for me, especially as, like, a broke kid. Like, I would spend some money to, like...I'm the guy who compiled Linux from source at, like, 15 years old, so obviously, I'm a little bit of a psycho, but that's the biggest thing that I could see.

And I’ll say that I think I'm monologuing a little bit, so I should take a step back. But, like, one of the patterns that I see in terms of people who are trying to bootstrap a software engineering career successfully is people who are trying stuff, putting it out on the internet, writing a blog, making a thing, like, I'm trying to make this thing. And then, you just put it up, put it up, put it up, put it up, put it up. Here I'm building this thing; here you go. Here I'm building this thing; here you go. Here I'm building this thing; here you go. And you put it out into the world. You're proactively doing the work. You're asking good questions.

Because most developers, most engineers, are very, very, very, very, very forthcoming with helping anybody out there who’s actually doing the work. If you come to somebody like, “This doesn't work. I don’t know.” Then, you know, you may get a cool response. But if you come out like, “Hey, I was trying to do this thing, and this is what I tried, and then this thing happened. And so, then I tried this, and this, and this, but then that happened. And I'm worried about, and I'm thinking, like, this is maybe a thing that I could try. But I have this question and, like, how do I implement it?” People will do backflips for you. [crosstalk 16:54]

MIKE: These are professional problem solvers. They can't help [laughs] but want to just solve the problem.

WILL: Exactly. And so, the cycle that I'll see in terms of people doing it right is they'll go out. They'll start working. They'll put their work out into the world, at whatever level of quality that they're doing. And they'll start, and then they'll get some feedback. And they'll put some more out, and they'll get some more feedback. And they'll pull some more out. They'll get some more feedback. They'll put some more out. They'll get some more feedback. And then, the blog is done because they have a job, and they're too busy to write the blog anymore. And I've seen that happen over and over and over. And so, that's kind of what I would...

Generative AI, I think, is a little bit of a tangent, because, I mean, really, it's just, like, start making things and put it out into the world publicly, and share it with everybody. There has never been, nor do I think there will ever be, a surplus of people creating things and pushing them out in the world. People who are willing and able to do that stuff are rare gems. We've never had enough of them in the world. We never will. And if you demonstrate yourself as that kind of person, there's a spot for you. You just have to, like, raise your hand and say, “Here, I'm out here. I'm doing it. Here I am. Look at all the stuff I'm doing.” And you’ll get scooped up.

MIKE: And I've seen that repeatedly. I’ll give a couple of examples, you know, go and build stuff. So, a woman...I think we may all know her, but I don't want to, you know, like, speak for somebody without, you know, tell their story without talking to them. Home school --

WILL: Is it bad [laughs]?

MIKE: No, it's not bad at all.

WILL: I’m kidding. I’m kidding.

MIKE: It's actually really good. So, she was a homeschooler. She had been teaching her kids for years and found that she needed to organize her stuff. And she’d wanted to code for a while, so she learned to code to build an app to organize her stuff. She was not out working professionally. She just built something because she knew she needed it. And she learned so much about how to get something done by solving this need that she joined an apprenticeship, quickly got to speed, and became an effective, professional developer.

Another woman I knew was working at the front desk at a company, started working with the engineers and writing some stuff. Like, 10 years later, she's running an engineering shop as a professional, you know, she became an engineer, ended up running her own shop, like, very much so. That process of building something, and, you know, from whatever background you're coming from, learning that lore, building, building, building, absolutely, I’ve seen to be the case. And the people who are really effective tend to be people who've done it for a while.

And you're talking about doing it as a kid. I started doing some coding in...well, I had done a little bit, like, as a really little kid, but then I didn't have a computer [laughs], so I didn't do any for a while. But then, when I was in, like, junior high, I got a computer, pulled out the big manual, GW-BASIC, and started writing some stuff, starting from those things, and building it, and then I didn't do it for a while.

But then I came back to coding in college. It went much more quickly. It went much more quickly because, man, I'd done this. I had practiced. I knew what I was dealing with, and it just pushes it forward so much. I also did some...talking about fixing computers, I did tech support for Windows 98 and Me [laughs]. And you learn some things about how a computer works and how to solve things that serve you well in the industry.

WILL: Absolutely, yeah. And I remember selling a lot of Windows 95s. I remember like, yeah, I did a lot of weird stuff, a lot of weird stuff, because I predate Windows 95. I was doing, like, Windows for Workgroups, or before or earlier, a lot of like, MS-DOS type stuff.

MIKE: I remember Windows came out thinking, oh, do you need that [laughs]? I've been doing all this on the command line [laughs].

WILL: Yeah, yeah, yeah. I don't know. And there's a big piece of me, I mean, this is a little bit of a tangent. But, like, I do wonder, to a degree, whether...I wonder how, you know, generations coming up right now, where, like, all the really big bad, like, gross stuff has kind of been watered down. It's been, you know, it's been tamed, more or less. I think there are a lot of computers that are effectively appliances, and there's really nothing to worry about.

If you’ve got a Chromebook, if you’ve got an iPad, you don't have to worry about all that. You don't have to worry about it, unless you want to be a professional, in which case, you know, here there'd be dragons. It's like when everybody opens up their car these days. You pop the hood on your car, and it's nothing but plastic shrouds. You can't see any of the weird hoses, and belts, and cables, and stuff like that. It's all in there, but you don't look at it anymore. You have to take the cover off, and it's like, oh yeah, it's the same nuts and bolts. It always [inaudible 22:13]. So, I mean, I just don't know. I don't know --

MIKE: You're touching something. You’re implying that you do know. You're implying that if you really want to learn this, that learning a language isn't necessarily, like, I mean, it's a part of the process. But you're talking about this lore that you had to learn, even more important than language, because the language of today is probably not going to be the language of 10 years from now. It might be. Java is still around. It’s been around for a while.

WILL: Yeah, Java is going strong. It ain’t going nowhere [laughter]. Well, and Kotlin, right?

MIKE: But it's not the number one anymore. There's new kids on the block. You know, there's Python, JavaScript, and I'm sure Python and JavaScript will be here 10 years from now. But there'll also be another new kid on the block, probably, and particularly talking about GenAI, you know, there's new tools, and that's always been the case. There's always been new tools, and those tools are going to change. And that doesn't change the fact that somebody who is trying to build stuff, and has grappled with hard problems and thought about how to address those problems in a systematic way, has developed skills that are universal.

WILL: Absolutely. There’s sort of, like, call it, like, cognitive, simple machines, like, where there's sort of processes that, like, I could explain to you in five minutes, and I will, and it won't even take me five minutes, but, like, foundational ideas. Like, always debug from the known to the unknown. Always debug from the known to the unknown. And if your stuff isn't working, dumb it down until it works.

I have had a professional day, where I contracted for my full rate, which is substantial, and they hate it, and they were happy to do it. And the butter would have melted in my mouth when I cashed the check, where all I did was make the damn thing turn on. That's it. It didn't do nothing. And that was [laughs] a hard day.

And everybody's had days like that, where you're just getting going. You talk about the lore. That's the lore right there, where it's just like, oh no, no no, just make it work, right? That is a dead, simple idea. Go from the known to the unknown, and if it's not working, dumb it down until it's working. Make it stupider. Like, how much stupider? As stupid as it takes, and then you move forward from there.

I could get a tattoo of that [laughter]. I could write it on my body in indelible ink. And you learn, sort of, like, simple cognitive machines like that conceptually, intellectually. It's a lever, a pulley, things like that. How do you break down these problems? I don't know. I don't know. I mean, nobody ever taught me any of this stuff. A lot of it was just, like, I just had a knack for it from day one.

MIKE: Well, so you say you had a knack. I'm sure there's some truth to that. Well, you say you sat around with a bunch of gray beards [laughs]. I think that software engineering has historically been an apprenticeship industry, and I think that's underappreciated. I think I've said it before on this podcast. I think it's true. You know, a few hundred years ago...there's a reason the most common surname in the United States is Smith is because there's a whole lot of people who are Smiths. There was a whole lot of need for metal workers who could, you know, a kind of cottage industry, who could take the metal and make it into important implement.

And you learn that by going and spending time with somebody who did it and working with them for a few years, maybe 10 years, for some extended period of time, looking over their shoulder and learning all the things that you learn by just being there. And I feel like software engineering is very much the same way. We try to short circuit it, and you think, oh yeah, we can teach this in school, hmm, or, you know, bootcamp. And there's some truth to that, but I think a lot of what you're learning in those places is coming from trial and error and watching the people around you.

WILL: I mean, the principal problem of teaching programming in schools is, I can teach you programming if I know it, but if I know it, do I really want to be teaching a bunch of high school kids, ehhh, you know [laughter], I mean, they pay me pretty good to just do it. And that's problematic if you want to teach people.

I think I've said this before. I think I've said this before on this podcast, and I don't really think of programming as a four-year bachelor's degree type of subject. I think if you graduate from school with a bachelor's in computer science, you have proved yourself to be smart enough and dedicated enough to be trainable [laughs]. The industry, historically, has been willing to absorb the sort of finishing costs to take this raw [inaudible 27:40] and turn it into an actual developer. And I feel like that appetite might be waning.

But if you compare analogous professions in terms of like, skill, you know, which I will make up out of whole cloth in my head, I think what you're expected to know is a lot more like somebody with a master's degree than a bachelor's or a high school diploma, or something like that, you know, all this stuff. Because it's an apprenticeship thing, you could do with a high school diploma or not even. You just have to sit with somebody to fill your apprenticeship. And companies, large companies, especially, they can afford to post that sort of a gate on you.

I'll say this, too, if we're worried about apprenticeships, you know, it's like, I'm a junior developer. I need an apprenticeship. You guys just took the graybeard’s advice on faith. And it's like, you got to have an apprenticeship, you know, to really learn the trade. Well, how do I get an apprenticeship then? The three little magic words: sketchy startups [laughter]. Sketchy startups. Sketchy startups. They're going to pay you in Amazon gift cards. Like, it's going to be in somebody's house that smells funny. Like, you're going to get a paycheck that you can't necessarily cash on the same day [laughter], like, all of that stuff, all of that stuff. But you will get your apprenticeship in.

And this is, like, one thing. There's a lot of stuff that you can and probably will have to let go, like a W-2 form, or health insurance, or anything like that. But the one thing you can't afford to let go is you do have to have at least a journeyman to be learning from. Like, there are a lot of sketchy startups who are like, uh, we don't have anything. You, did you get a web page up one time? You're the CTO/co-founder.

And you got to have somebody better than you, like, a couple of steps ahead of you so you can learn from them, or else it's just going to be a disaster, like, a bad experience for everybody. But, in the end, yeah, sketchy startups, man, so many people, so many people have gotten their start on some, like, really, rinky dink fly by night what do we even sell startup.

MIKE: Another wonderful thing about said startups is you wear a lot of hats. You do a lot of things, and you get to work in a lot of aspects of the business that you're not going to get so much in a large enterprise [chuckles].

WILL: Absolutely.

MIKE: You're setting up the infrastructure [laughs]. You don't have a DevOps team. You are the DevOps team and the engineer. And you've got to do both, and you learn it, or else it doesn't get done [laughs]. Having been at multiple places where they laid off the rest of the team [laughter] and I was doing everything, you learn a whole lot that way, when it's you or nothing.

WILL: That's the truth. But, I mean, it is pretty sketchy. It's pretty intense to go in without any sort of support, especially when you're just a baby dev. I think it's difficult, and when you find yourself in situations like that, the fundamental trade off you make is a lot of people...it's a really high risk, high reward situation, where there are a lot of people who would have made it if they just had anything, anything backing them up, anything at all. They would have made it, but they had nothing. And so, the ones that make it through those guys are going to be serious.

MIKE: Sure.

WILL: But you're going to find a lot of people who, ordinarily, they could have made it, you know. They could have done it. They just didn't have anything, you know. I believe in taking risks and stuff like that, but I don't believe it takes stupid risks. And I think you can get yourself in a bad situation like that.

KYLE: That's a good point, where you can’t get discouraged too fast, like, I don't think I took a necessarily traditional approach. I wanted to go straight into dev, and that backing that you're talking about, or that support, I don't think I ever got it. It just was not available to me. And I ended up finding myself in QA. I found myself in QA for seven years before I really found my niche in DevOps. And saying that, I'm saying, you kind of have to keep at it like. Maybe you don't get the opportunity, you know, right away. But if you stick around and it's something that you really want to go for, maybe the opportunity will present itself. Like, not everybody gets it at first try.

WILL: Oh, absolutely, yeah, completely.

MIKE: Looking for work during the dot-com bust was not fun [laughs]. That was not a great time.

WILL: Oh, it was so rough. Yeah, I worked at a gas station. I worked at, like, straight up, I worked at a gas station, like, I was the ninth shift manager at a gas station.

MIKE: [laughs] I ran up my credit card bill pretty high [laughs], and I very nearly went back to doing construction, because I made money doing construction.

WILL: [laughs] It’s the truth, man, yep. Yep.

MIKE: But on the other side, it got better, which, you know, kind of going back to the theme, you notice how we're talking very little about the language. We said, what did I wish somebody had taught me? I think that you should learn a language. If you want to learn to code, yes, go learn a language. You might not code in it in your job; in fact, good chance you won't, but you're going to learn some things. You are going to learn some things about yourself. You're going to learn some things about the computer. You're going to learn some things about the computer you wish you didn't know [laughs]. You're going to learn those simple tools.

You're going to learn, maybe if I cut this problem in half, then I can very quickly do a binary search and get to the root of the problem. You're going to learn how to write a configuration file, which you will do an endless number of times in your career. You're going to learn how to do all of these lore sorts of things that change your perception of the world. You're watching the master blacksmith banging on that metal, and you're learning what the colorless sparks mean. And it's going to change you more than the language.

Although the language is going to be important, you got to learn something. You need to go out and start building something. And I will express that strongly. I think you said it really well in your monologue [laughter], Will, that by building those things, you're going to learn a different set of skills. And, you know, I came into this, this podcast thing because I wanted to express something about you got to learn the language, but that's not the most important thing you're going to learn.

WILL: Like, I almost think...one thing, and this is something that I’ve thought for a while, maybe I'm just out of touch, but I really feel like...how do I put it? In the age of generative AI, ChatGPT, and stuff like that, like, I've never thought all that much about programming languages, and I plan to think so much less in the future, because, like, what’s a language? A language is a way to express a series of procedures, a series of thoughts, very precisely, which human language does not do, will never do, but, you know, sort of even a high-level programming language. It is a language to express logic, procedures, stuff like that, you know, really specific way.

And ChatGPT, generative AI is amazing at translation, the translation of ideas across, you know, from English to Spanish, from Spanish to Japanese, to Korean, to Smalltalk, you know, whatever you want to call it. And so, you know, language Kotlin, Node, Python, Ruby, Swift, whatever, the ideas are important. The language is nearly meaningless at this point. Maybe that's just me, because I understand the concepts really, really well, and I know exactly what I want. I know exactly what I want. And so, how do I say that in Python? I almost don't even care, you know.

MIKE: Will, I'm suggesting that you've got to pay your dues, that it is helpful. It's helpful to take that stack of CD-ROMs and of source code and compile your operating system. Because you know what? CD-ROMs don't matter. And that particular build of Linux you probably used for a little while and forgot about [chuckles], you know, the specific problems you were solving were irrelevant. But you had to do some work grappling with it, and you have to learn something, I think.

You know, we all, as a child, we don't know how to speak, so we watch people around us speak. And I don't think it really matters what natural language we learn. You got to learn one, and then you can communicate. And then, if you need to go learn another language, that's what you do, another natural language. Likewise, here, you've got to learn how to speak. And I do think that you have to go through some work, and, like you say, work for sketchy startups. Go write a bunch of stuff. Go do a bunch of vibe coding, you know, go use a bunch of generative AI and see what you can build, and see how it's broken, fix it, and [laughs] push it out there. Work on something.

And it's that working on it where you learn those skills. So, I do think you need to learn something specific. You've got to drill down enough to be able to understand. So, if you're writing apps using generative AI, they're going to write it in the language. Underneath, there's an interpreter, compiler. There's pieces that you're still going to have to understand because it's going to do it wrong, and you're going to have to go and learn...say it's Python. You're going to have to go and learn Python enough to fix its bugs to know, hey, this is wrong. Where did it go wrong? And go and fix it, or, you know, ask it to rewrite it again, figure out which one looks best.

You're going to have to learn the underlying structure as you go. I think learning one matters. I would recommend that. Learn one thing well before you generalize. Maybe not perfectly, but I do think the learning one thing really well does help you generalize. Going back to what we talked about at the beginning, [inaudible 39:22] one thing further.

WILL: For sure, for sure. Make something, build something, and put it out in the world. Make something and put it out in the world. Vibe coding is great. I am pro vibe coding, you know, like, just making things. I love making things. Like, that's the foundational thing. There's a big piece of this where it's like, if the idea of making things just for fun, to scratch your own itch, because I want to make stuff, because I like making things, and I'm just going to do it and put it out there, put it out there because there are people who desperately want to find you, but you have to give them an opportunity, some avenue to reach you.

If you want to work, you have to find some way so that somebody who's searching for you desperately but they don't know your name could find you in some way. I think that's absolutely foundational. But if the idea of just making stuff just to make it doesn't really sit well with you, where it's just like, well, but who's going to pay me? There is a big question, I think, you need to ask yourself, which is this for you? Because that is the fundamental thing.

If you don't just love making stuff, then is this really a good idea for you to waste your one precious one-time-only life doing? I don’t know, there's lots of wasted, like, just make a bug. And that's fundamentally why, like, you know, ultimately, like, that foundational loop of making things and putting things out in the world it always works if you're just really into making stuff because not everybody is and, like, we could smell our own.

MIKE: [laughs] It is true.

KYLE: I would tack on, too, to go along with making stuff, understand how stuff is made or what is already out there. Because I feel like that's one thing that I missed going through my career is I didn't take the time to stop and understand what it was that I was coding against. Like, what was the underlying operating system like? How is this running on Linux versus Windows versus, you know, the other OSs? You know, just having that fundamental understanding underneath the hood just helps you understand what you're coding as well.

WILL: That is such an important concept. And this is one thing that I think we're getting very, very, very, very, very wrong in terms of the vibe coding, like, generative AI thing. Like, hey, man, I understand that we all want to make things. We want to push these things out in the world. We want to make things, and that's, like, the goal. But one of the other cognitive levers, the cognitive simple machines, is like, when you do something and you're like, I don't know how that worked, read the manual, man. Just read the manual. You did something, and it's like, oh, I'm just sort of trying, like, I’m just trying something. Oh, that worked. Whoa, whoa, whoa, what did you do and why?

Like, taking that step back, that is another one of those intellectual, simple machines that you really need to master. If the other one is a lever, then this one's a pulley, where you integrate these things, and you get a larger understanding of how stuff works. That's why I don't care about languages anymore, because I've done this so many times that I understand the foundational things. Like, oh, we're doing a [inaudible 42:53]? It's like, oh yeah, that's a thread, but it's got rules around it. I'm like, okay, that's cool, you know, because I understand how that works.

And, like, you know, oh, promise. It's like, oh, this is how a promise works. This is how an async function rewrites my JavaScript with all these freaky returns and stuff that it puts in there, where I'm like, okay, these are the rules for taking this async promise away thing and turning it into what appears to be an imperative function, just all that kind of stuff and, like, that's another one of those fundamental things. And it's one of the things we get so, so wrong about generative AI, because generative AI goes both ways. Like, not only can you ask it, “How do I do this?” You can ask it, “Whoa, what the hell did I just do?”

KYLE: Yeah, we need to do that more often.

WILL: Yeah, we need to do that way more often, way more often. You're talking about the lore. That's it, the generative AI it could do that for you, just basic stuff, where it's just like, you know, like, oh, I Googled this thing on Stack Overflow, which is still really, really relevant, although some of those Stack Overflow answers are getting pretty creaky, where it's like, the accepted answer is, like, 10 years old. And I'm like, I don't know whether that's best practices anymore, homie. I don't even think that'll compile. That's from 2009. We're in a brave, new world.

Like, that Stack Overflow answer can drive a car [laughter] because we’ve moved on. But you can read it. You could feed it into generative AI and say, like, “What is this?” And it'll tell you, and, like, it works actually a lot better. Like, what the hell is this? So, it'll be like, it's this, and this, and this, and there will be references. And you should look at the references because sometimes it'll hallucinate [laughter], but the references are usually pretty good. There's all kinds of stuff out there.

I think it’s never been a better time to learn how to make software, learn to code, learn how to make things using software. I don't think there’s ever been a better time of your life. And I think for the people that have the knack and really want to do this, I think the doors are as open as they've ever been, really. I don't think it's that much harder than when I graduated, you know, college at the middle of the dark days of the dot bomb era, you know, and I had to work graveyard shift at a race track, sell my plasma in the hopes of getting into the industry.

MIKE: I did that, too [laughs].

WILL: I don’t get...Pro tip: drink a lot of water. I'll water that down. You're not stealing my vital fluids for nothing. They gave me the side eye. They're just like, “This is looking pretty thin, dude,” and I'm just like, whatever. I don’t care [laughter].

MIKE: Another pro tip: tums.

WILL: Tums.

MIKE: Calcium. They use an anticoagulant that they'll make you feel funny, but if you have lots of calcium, it counteracts it.

WILL: Interesting, interesting, okay.

MIKE: Totally off topic [laughs].

WILL: I mean, you know [laughter] --

MIKE: Make sure you’re giving blood, too.

KYLE: You need to know how to pay for it. You need to know how to pay for your education.

[laughter]

WILL: That’s right. That’s right. It wasn't for free. Yeah, it wasn’t for free. But, anyway, I don't know, I remain optimistic. Nobody is throwing their phone in the trash, you know, nobody's spending less time on the internet. I think I'm probably going to stay fed, and you can, too. There's room for you. If you're listening to this, get in here.

MIKE: Yeah. So, during the pandemic era, interest rates were really low, perhaps artificially low, and so big tech companies invested a whole bunch of money in themselves because they couldn't invest anywhere else and make money, as much money as they could by investing themselves. Interest rates go up, and they look and say, hey, I can take this cash I've got, and I can go make whatever percent off of it really consistently. I don't have to risk hiring a bunch of people and hope they build something that'll make me money, so they let a whole bunch of people go.

WILL: It’s true.

MIKE: So, there is a flood of engineers out on the market right now, and it makes it really hard to find a job. I actually read a data analysis just this week about this topic. And there is --

WILL: Send it to me, would you? Will you email it to me?

MIKE: Yeah. I think it was in The Washington Post, if I remember right. Basically, they found no evidence thus far that generative AI has had a meaningful impact on hiring in software engineering.

WILL: I believe it. I think there's a lot of weird stuff in terms of big tech. This is a little bit of a...no, it's a substantial diversion from what's going on. So, I work with a lot of people out on the West Coast. And what I have found is that big tech let a lot of people go, but it's having a very local impact, because big tech it's all on the coasts, you know, a lot on the West Coast, to a small degree, on the East Coast. Well, all these guys required people in the office, so you have all these people who are unemployed on the West Coast. But they live on the West Coast, and they need West Coast money. And they can't make Midwestern money, Utah money, mm-mm, that won't do because they live in San Francisco, and they have to pay their San Francisco mortgage.

And so, I feel like a lot of the unemployment effects from big tech losing a lot of jobs are very localized, where, you know, like, I’m pretty senior, and I get paid pretty good, but I don't get paid pretty good San Francisco money. Like, nobody's cracking half a mil. For me, that's not the dough I'm making. You can make some, you know, you can get on levels.fyi, and they'll tell you the Bay Area is a very lucrative place to be an engineer. But you don't get that money, you know, telecommuting from Idaho anymore. They kind of shut that down because they've been doing layoffs.

And so, demand is very high, and they can say, like, no, if you want Facebook money, you're coming in Facebook. You're going to have to get on the half-pipe and skate around every day, or whatever they do [laughter], you know. You're going to have to come and eat our sushi if you want that money. So, I don't know, it's one of those things, that very top tier, very expensive market, very, very, very high pressure stuff. Like, yeah, those guys are hurting pretty good. But I actually don't know whether it's so bad if you fly out of the country.

MIKE: For new hires, it's real bad. Yeah, it's real bad. I was interviewing recently for an internship position, and just the number of applicants we got alone.

WILL: Interesting.

MIKE: I think we posted it, and we had...I don't remember the number. I think it was over 300 applicants in one or two days for an internship position, not even, you know, like, a full-time, long-term job, just to get an internship.

WILL: For how many positions? For one?

MIKE: Two, yeah.

WILL: Two. Mercy, that's brutal.

MIKE: Yeah, it's tough out there for people starting.

WILL: So, I should start, you know, exploiting these [laughter] beautiful young minds for my sketchy startup. You’re saying, Will, start a sketchy startup, like, there are people to be trafficked.

[laughter]

KYLE: Oh geez.

MIKE: And on a more positive note here, it’s always --

WILL: That was great. I’m very optimistic [laughter]. I have so many sketchy ideas. I'm going to be a head in the box. You'll never see me. I'll just Venmo you sporadically [laughs].

MIKE: I think Will said it really well, not very long ago: the world needs people who can think clearly about problems, who can break them down and express unambiguously and in a well-structured way about what the problem is that needs to be solved. This is the problem I need to solve and solve it. That need is great today, just like it was every other time. People have been hiring software engineers for the last, what, 70 years [chuckles]?

WILL: Yeah. All the way back to Grace Hopper, man. Like, who's going to fix all this crap [laughter]? Yeah, fascinating. All right, I'll have to cook up some sketchy startup ideas.

MIKE: Oh, any final words?

WILL: No, no, that was it. That’s the final thought. Like, I’m going to start cooking up some sketchy startup ideas [laughter]. There are exploitable minors out there.

MIKE: Expect some job listings in the near future [laughter]. Until next time on the Acima Development Podcast.

In this episode of the Acima Development Podcast, host Mike kicks things off with a personal story about his wife’s jewelry-making hobby to introduce the episode’s main theme: the classic “build vs. buy vs. open source” dilemma in software development. Using analogies like 3D printing and sourcing beads, Mike illustrates how creators often rely on third-party resources rather than building from scratch. The conversation, joined by guests Justin, Kyle, and Will, highlights how this decision-making process mirrors software strategy—balancing time, cost, scale, and business priorities when choosing whether to build proprietary solutions, buy from vendors, or leverage open source tools.

As the discussion unfolds, the group explores real-world applications of these choices, especially at scale. Will draws from his experience in academia and enterprise to explain how even cutting-edge innovations are often built atop existing open-source frameworks. Justin stresses the importance of aligning technical investments with what drives a company’s revenue, while Kyle adds that factors like security and infrastructure complexity often make buying solutions the safer choice. The team discusses examples like login systems and reporting platforms, agreeing that while DIY solutions might work early on, scale and security typically justify outsourcing or adopting proven services like Auth0 or Snowflake.

Toward the end, the conversation veers into the implications of AI and large language models (LLMs) on the future of development. They debate how junior engineers might adapt, how architectural decoupling can future-proof integrations, and how the rise of AI could alter productivity expectations. With humor and insight, they emphasize that every decision—whether to build, buy, or adapt—requires thoughtful consideration of timing, scale, and evolving business needs. The episode wraps by reinforcing the idea that good architecture, adaptability, and a keen sense of timing are essential for sustainable tech development.

Transcript:

MIKE:  Hello, and welcome to another episode of the Acima Development Podcast. I am Mike, and I am hosting again today. I apologize: my voice is a little bit raspy. I've had a little bit of probably a virus [chuckles] going through the family. I feel much better today than yesterday, which is good [laughs]. But I may be a little bit raspy.

I've got a small group with me today. We're actually moving offices at Acima, moving to nicer, I guess, you know, a little bit updated offices just a couple of buildings down. So, same complex, it's not much of a move, but [laughs] upscaling a little bit today. So, a lot of people are tied up in that. But I've got with me here Justin and Kyle. And we should have a great, more close personal conversation today than we sometimes do. That'd be great.

I'm going to start by talking a little bit about my wife [chuckles].

JUSTIN: I'm a little curious how you're going to spin this because this is the build or [laughs] buy --

MIKE: That's right [laughs]. So, my wife is an artist. She doesn't paint very much. She makes stuff. She's a maker, and she's made a lot of things. But the thing that she does most, and she's done this for years, she makes jewelry. I cleared this with her beforehand [laughter], you know.

JUSTIN: You’re going to be internet famous, hun.

MIKE: Yeah, exactly [laughs]. I've mentioned her before. She makes stuff. She makes jewelry, not, like, diamonds necessarily. She uses natural stones...diamonds are natural stones. But she generally does more, like, beadwork. She'll do complex subbead things, you know, of patterns of natural stones. There's one she uses a lot called labradorite that is just this beautiful stone. Kyle's nodding, like, I know the one [Chuckles]. It's translucent, but if you look at it in the light, it has this blue shine in it. It's really pretty. But she uses other stones as well. And she'll make these patterns and put, you know, she uses a lot of, like, so it's like a larger stone as the centerpiece. It’s kind of chunkier sort of stuff, really pretty.

And she spends a lot of time, you know, like, hand tying it between the beads [chuckles], and she'll say, “Oh no, I messed something up. Now I got to undo, like, two hours of work [laughter].” And so, she'll have to undo two hours of work to go and retie it and move on. And she spends a lot of time building this stuff. Really cool. And so, you know, each is one of a kind; that's how she does it, which is really cool. You get these one-of-a-kind things that she makes.

But there's some interesting aspects about this. She doesn't make the beads. She doesn't make the stone. I mean, the earth made the stone [laughter], and then you have somebody in the supply chain, you know, somebody whose job it is to go collect the stone from a quarry somewhere and distributes that out to suppliers. Some of the suppliers will buy the stones. They'll take those stones and make them into...and they'll make lots of different things, but some of them will make them into beads of different sizes. So, she'll buy it several steps down from the original source.

She buys the stones. She doesn't make those herself because we don't have the manufacturing equipment, nor would we. I mean [laughs], it would require a lot of upfront costs. Now, if that was your thing, then that would be a very reasonable thing to do. She was actually talking to me the other day like, “You know, I see these people who have their own stone shops, and I'm tempted to [chuckles] start small and go into that.” I don’t know if we'll do that or not [laughs]. But she's thinking about expanding and doing some of that, you know, to actually start getting some of that stuff wholesale. We'll see.

This ties into our topic, by the way. This isn't just an aside. It's very relevant [laughter]. So, we could potentially buy some of that equipment, but it has not been worth it thus far based on the business that she does. She doesn't do this, but I know people who have 3D printers, so who do buy the supplies and start making it themselves. Justin's, like, pointing [laughs] to 3D printing equipment over [laughter] in the corner. We only publish this in audio, but we record it with video. Maybe someday we'll do the recording, you know, post the video, but [laughs] we only post the audio.

So, he's got the 3D printer, and 3D printing is super cool. You can print out all kinds of cool stuff, but, like, the stuff for making the beads out of the stone, you have to buy the equipment. And you can download the designs. They're great. I mean, you can download all kinds of cool open-source stuff on the internet, where people have published the 3D models, and you can print them out yourself. But you've got to buy the stuff.

You've got to buy the resin. You know, you're taking responsibility for everything in the process. Whereas if you're not doing that, you just order from somebody who does it online, and they ship it to you unless you get to a certain scale. Justin, have you reached the scale where it's actually been economical for you?

KYLE: [laughs]

MIKE: Hear the laughter? Hear the laughter, listeners [laughs]? Of course not, but it was cool. So, you did; you got to do it [laughs]. I've got Matt who's been on the podcast quite a bit. He has a 3D printer. I've got a nephew who does it. I don't think it's ever been economical for any of these people, but you do get to do...I know Matt just does a lot of his own 3D designs. I don’t know if you do that, Justin, or not.

JUSTIN: Not as much as I'd like but, I mean, that's the intent, right?

MIKE: Right. The intent. So, it's one of those, someday [laughs]. It's like my big, beefy computer I've got on my desk here for machine learning that I'm not using right now [laughter]. When I get a moment, we'll use. So, we talked about...oh, and Will is joining us. He's been a bit delayed, but Will Archer is joining us, which brings us to our topic for the day. And you're joining us right on time. We're going to talk about, in software, when do you build; when do you buy; when do you use open source?

And we've just talked about very practical, tangible items, but software follows a lot of the same rules, right? And there are times when any one of those three is appropriate. And you think about things in the tangible world, and a lot of the same rules still apply. And I'm going to stop talking for a little bit because I've set the stage, and I'm curious what your thoughts are based on kind of the stage that I have set.

WILL: I feel like I've had a little bit of a unique career in that I've spent a decent amount of time kind of on the edge of technology, like, actually doing brand-new stuff that has never been done. Because I came from academia, where we're, like, hardcore research, like, stuff that isn't commercial, and it's not going to be commercial for a while, and then brand-new technologies where there weren't libraries. It wasn't, you know, necessarily sort of your standard web CRUD systems kind of integration stuff.

And what I have found, personally, is you will almost never be going soup to nuts, brand-new stuff. The most out there, woo-woo stuff that I ever developed was a static analyzer for embedded device drivers. And what we were trying to do is we were trying to prove memory safety, where it was this low-level super tiny code set, and we were trying to use...and what we were doing there was, like, we were using a publicly available open source, another woo-woo research project theorem prover. And so, we were sort of trying to analyze the code, you know, get a formal specification for this embedded network, you know, firmware driving...I'm, like, throwing bits on your, like, you know, like, real, real nitty gritty thing.

But I was only doing this within the ecosystem of this previously available open-source stuff. I did a bunch of, you know, kind of pushing audio processing forward on the iPhone when it came out just, like, real cutting-edge stuff. But even then, I was taking a bunch of existing open-source stuff for Linux like C++ drivers, and files, and processing stuff, porting it over to this thing, making it run there, then adding my tiny, little secret sauce, which was cool. It was cool. I did stuff that pushed hardware forward a little bit.

But even then, I was doing just a little bit, like, just this extra thing. There was this decoder thing. There was the analyzer thing, and none of them worked for this, and I made them work for that. And then, I added on some sauce. As far as I've gone, it's just incremental improvement. And that's in a case that I think, you know, outside of those two gigs, I don't do stuff like that anymore. I'm doing big enterprise line of business stuff, where I'm, you know...I hate to call it an assembler, systems integrator, you know, because I write plenty of custom code, nothing like that. Even as far out onto the ice as I've gotten, that's all it's been, you know what I mean, so...

MIKE: Absolutely. So, standing on the shoulders of giants.

WILL: Well, exactly. Absolutely.

JUSTIN: Personally, I think, you know, the question is, you know, build, buy, or open source? It comes down to what makes your business special? Because you need to determine what is making you money, and you got to maximize that and minimize everything else. And if what is making you money is something unique that you have to write custom software for, yeah, go do that. But if it's something that's not core to your business, buy it or open source it. Push everything else away. Just focus on what makes you money. At the end of the day, if you aren't making money, you know, what are you doing as a business [laughs]? I mean, maybe you got a lot of money pouring in from investors, but long-term, you got to maximize your, you know, the thing that's making you money.

WILL: Okay. So, I'll play devil's advocate here because one of the things, you know, when I was working with Acima, and one of the things that I do, like, right now for my large retailer, is at big enterprise scale, small efficiencies add up. One of the things that like, you know, like, I don't feel like I'm talking too much out of school here, but I spent a lot of time dealing with various, you know, financial infrastructure providers trying to get a couple of basis points on a billion dollars’ worth of credit cards.

JUSTIN: That's making you money.

MIKE: That’s making --

WILL: That's not Acima’s business. It's never been Acima’s business. It is not core business. That's not core business. That's just like, yo, at this scale, if you can get me a couple of basis points, you just paid for your whole team forever. But that isn't why Acima exists. It's nothing to do with Acima’s core intrinsic value proposition, whether I can run cards for, you know, 3.5% or 3.45% has nothing to do with whether the business lives or dies. But if I can get it, you know, 500s of a percent more efficient, that justifies the entire team's salary, [chuckles] forever because it is kind of cross-purpose, you know. It's not going to double the business. That's not going to --

JUSTIN: But like you said, it's going to pay for stuff.

WILL: Sure.

JUSTIN: So, if you can identify something that you can write that is custom that is worth a lot to you or to the company in some way, you focus on that, and that's what makes you unique, probably. And, at the end of the day, though, it's like, do you have to write your own code to optimize for that, or can you buy something that optimizes for that? And so, you're always going to have to do this juggling, this evaluation of is this ultimately going to be worth it and make more money for the company? You know, buying versus building.

And a really good example for this, where I'm currently at, or every place that has a login, to have a good, secure login, that takes a lot of skill. And there are companies that have figured out how to do that at scale and sell it as a third-party provider, you know, Okta, OAuth, you know, Okta, sorry, OAuth.

WILL: Auth0.

JUSTIN: Auth0. And, you know, there's a number of other providers and Omni, for example, or, you know, where I'm currently working, we're not in the business of login. So, obviously, we're going to be, you know, hey, this is very important that we get something that did it right. And so, we will shift the risk and the responsibility to this third party, and we will pay them gobs of money to own that. Because, at the end of the day, that's still going to be more secure and cheaper for us than writing it ourselves.

KYLE: I think you brought up another variable in what you were saying there, too, depending on what it is that you're looking at, different variables play a larger or a lesser role, and in login, security is massive. You can roll out your own login stuff, but it's like, do I want to risk it, or do I just want to go with the golden standard? I'll pay that premium for that golden standard, the tried and true that we know works.

And I think we run into that with a lot of SaaS products, where there might be a really good, even open-source product out there, but it may not have the name. It may not have the GitHub stars or whatever you're using to qualify it. You know, you're not willing to risk your business. Even though it's a massive cost difference, you're not willing to risk your business in going for that open-source solution versus the paid-for solution.

MIKE: Well, you're not just paying for the software. Going back to what we talked about with the 3D printer [chuckles], software is free with open source, but that is not the only cost. You now have to own everything about this, make sure that every configuration is right, because if you get it wrong, then your business gets compromised [laughs]. And now you're paying for the security experts. Now you're paying for, you know, the liability that one of them has done something wrong. And you start adding up those costs, and it's not actually free [chuckles]. It's not actually anywhere near free, and the cost is likely higher than just paying for it.

KYLE: Well, and, to your point, how many of these larger companies have we seen spawn where none of their software really is proprietary? They're just using open-source software, and they're selling it to you as a service. There are so many companies out there right now that, you know, they'll use the ELK stack. And they're using, assumably, some of the more open-source type portions of that. Maybe it's licensed to some extent. But I'm saying you are just paying for the support. You are paying for them to manage your infrastructure for you and to have that reliability.

WILL: Got it [inaudible 17:19] not like us. We would never just put together a bunch of open-source software and sell it to people [laughter].

KYLE: No, never.

MIKE: Or [laughs], as you say, shoulders of giants. Does anybody build an operating system from scratch if you're not in the operating system business? No. And even the people who are in the operating business, a system business, if you're Red Hat, you didn't build it. I mean, you built some of it [laughs].

KYLE: You've contributed to the community.

MIKE: You contributed to the community. And even Windows is moving a little bit in that direction. It’s not a great model [laughs] to have to build the infrastructure. You know, everybody lives on top of that. And the infrastructure works really well when it's shared. And you look at all the cloud platforms. Yes, AWS, you could replicate a lot of what they do pretty easily, but you don't [laughs].

KYLE: No.

MIKE: Or, you know, I mentioned one cloud provider. There are others, of course, that are also very competitive in there, you know, Google Cloud Platform or Azure. They provide that.

WILL: Well, but AWS and Oracle, let's say, you know, some other ones, they do provide a counter narrative here in that, I mean, Oracle is maybe the furthest along the curve where their opacity has...like, when I run into some ops team who's just like, “Yeah, we're deep in Oracle,” I'm like, “Oh man, I'm so sorry.” Because Oracle runs...everybody I've ever talked to says that Oracle runs a real type database. They run good database.

MIKE: 100%.

WILL: Their stuff is as good as you'll find. But I don't know a lot of new people, a lot of new builds that are getting on them because they really charge you for it. And so, what you are running into...oh, maybe Kyle could speak to this better than I can. But I know a lot of people who have been pushing back on AWS pricing because you get real locked in real deep to AWS. I don't know, I have personally, in my own personal little micro projects, looked real hard at some of the other providers, you know, other than AWS for some of the stuff. There is a lot of people who are taking it back to the colo, the independently run colo just because AWS has been tightening in the screws bit by bit by bit. And that is the counter-narrative.

You get locked into a system, and there's some very smart MBAs. And if they don't have very smart MBAs, some hedge fund will buy them and install a team of very smart MBAs to [laughter] slowly bleed you dry, you know.

JUSTIN: I think we've talked a couple of kind of extreme examples, but I do want to look at one that may be a little harder to evaluate, and we all know what Snowflake does.

WILL: I don't know what Snowflake does.

KYLE: Well. [laughter] Yeah, we should probably touch on it for our listeners.

JUSTIN: I think Mike could say what Snowflake does.

WILL: [laughs]

MIKE: They provide data warehouse services, so backend, not real-time reporting data, real-time transactional databases, but business intelligence reporting, you know, all of the post-transactional database stuff and the whole ecosystem for making that tie in together.

JUSTIN: Yeah. And so, that data, Snowflake itself didn't generate that data. That data lives in our data stores, you know, in our databases. And, theoretically, business could come to the engineers and say, “Hey, I need this report.” And the engineers could go in and generate the report for them, right? And so, there's nothing that Snowflake is doing that is absolutely necessary for the business to run necessarily, you know. But it does [chuckles]...it does make the engineer's lives a little better since they don't have to interact as much with the business folks. It gives them tools to dissect that data. So, the question is, do we build a reporting system? Do we buy Snowflake, or do we use some sort of open-source thing? Like, what are the pros and cons there?

MIKE: The answer is don't ever build your own reporting.

[laughter]

KYLE: Please don't.

JUSTIN: Why not? It's our data.

MIKE: I've done it. I've done it before. Have you ever seen what happens when said business person runs a report against your production database that has grown a little bit bigger than it was a year ago [laughter]?

JUSTIN: Why isn't the website responsive [laughter]?

MIKE: That's what happens. Yes. And then, you end up using a data warehouse service like AWS Redshift. And [chuckles] next thing you know, you're rebuilding something like Snowflake. Now, you don't have to go to Snowflake. You don't have to go that far. But building reporting yourself, I mean, if you've been living under a rock for the last 20 years, data has become important [laughter], and it drives a lot of business and is actually, you know, what's driven all of this AI revolution that we've had over the last few years. It's the reason we have smart devices. You can talk to them, assistance, because there's a lot of data around. Data has become this huge, huge thing, and well-run businesses use that to run. They need that data.

And so, reporting has been done before. I don't know how clever you think you are. Just like you're not going to build your operating system from scratch because you're not actually that clever, it takes a team of very smart people years to build that. You're not going to build a better reporting stack for cheaper than what's out there. So, there is my strongly held opinion. If you are building the reporting, except perhaps if you're building some special visual chart that nobody else is doing, your own little special sauce on top of the framework that others have built, then there might be a business there.

But, generally, if you're not in the business of specifically reporting, I think the answer is if somebody asks you to build a report, you say, “Okay, what tool are you going to use for that?” That is the answer [laughs]. The answer is not, “I will build that.” Now, push back if you think I'm wrong, because I could be wrong.

WILL: I don’t know.

KYLE: I’ll tuck onto what Mike said there and say that, when you do that, what I've seen is you have a database that is then dual purpose like that. And as a platform, DevOps, whatever you want to call me, engineer, trying to support that infrastructure for my engineers, it's one of those things where scaling for those different types of workloads is vastly different. And so, when you have that dual-purpose database, how do you scale it? So, if you're not using another solution like this, it's almost impossible.

WILL: Well, no, I suppose I could say, you know, push back because I think all this stuff there are arguments for both. And the biggest argument for doing it yourself...and there’s maybe two. One is there's this super special thing that only I can do, and no one is doing it. It's a niche thing that's just unique to me. And then, what do you do? You take an open-source ecosystem and solution, and you build on your little thing on top of it. That's usually how it gets done. And then, two is you just need a little thing. You just need this one thing. You just need to run this one report. It’s just one. I just need to run, you know, sales for the end of the week, just tally it up, right? And everybody's laughing. It’s like, oh, doing it yourself is a dumb idea.

No, doing Snowflake for some tiny ass thing is the bad idea because it’s only a bad idea when the thing snowballs. And you have a goer, and this thing is working, and, like, you're doing more and more. Instead of every day, it's every night. Instead of every night, it's two times a day, and then things get bad, right?

But no one's counting the times where it was just a thing that you built, and then it didn't get any bigger, or you stopped using it in the first place. It was just something where it's just like, oh, it's a thing. It's like, ah, yeah, it was dumb. Whatever. Like, it is definitely worth investing in something that's a goer, and you're using it a lot, and it's starting to impact the organization. But, like, most of my ideas are bad, you know, I just try it out. Just try it out. You’re like, oh, that was a dumb idea. It's like, yeah, put it with the others [chuckles].

KYLE: Just, like, so maybe as a proof of concept or as a start, it's not a terrible idea, but if it grows at all, you probably want another solution.

WILL: Exactly. And then, the real devil becomes, oh, it's growing. But like, were you watching it? How big did it grow? How long has it been? How many people are running that report and how often? [inaudible 26:46] how that’s going to go [laughs].

JUSTIN: I think this goes back to if you are in a startup, like, a small startup, and you're trying to figure out how to make money, and you're, like, my budget is zero, and, you know, whatever needs to happen, you know, depends on me, and the money that squeezes out of my wallet that, you know, I made doing something else for 10 years. So, if you're in that startup mode, I think it goes back to what you said, you know, you are doing everything by yourself to see, you know, does it make any sense, you know, to run it at least once? Because a report, like you said, you know, I could run an SQL statement and done, and, you know, I can automate that and done then, you know, it grows and so on and so forth.

But it's like, a lot of this time, in startup mode, you are exploring everything because there's some sort of business need and, usually, if you can identify that business need, and it's some small X unit of time to execute it once, you're probably going to do it and try it out just so you understand what's going on. And then, once you’ve ran that proof of concept, then you can decide, oh, this took me, you know, half a day to do, but I could see it growing a lot. Now I can evaluate the open source, or I can evaluate the vendors, because I know what's required. I don't think you know enough to make those other decisions until you've tried it yourself. It's really hard to do that until you've tried it yourself.

MIKE: I should put an asterisk on what I said before, at scale [laughter], at scale, don't build reporting yourself. But I am completely in line with what's been said and figuring out what that line is. Where does this count as scale is the whole reason we're having this conversation; otherwise, it would be so obvious. We wouldn't have it in the first place. Because when you are small, you've got to build it yourself. You've got no other choice.

WILL: Yeah, everybody starts small. You know, everybody starts with nothing. Even in a big company, like, if you don't already have, you know, your Snowflake, your Red Shift, whatever, whatever you're doing...Even in a big company, if you've got a Snowflake contract, no problem. If you've got OAuth contract, no problem. All good.

But the first person to be like, yo, this is out of hand. Like, we've lost our minds. And then, the six-month grind to move all the old junky stuff onto the new, shiny infrastructure that's brutal. But we've all migrated, where it's just like, this can't go on, and it's just like, oh, we have to drain the swamp now. Okay, nuts. Well, that's going to take a while, and that's just the nature of the business.

It's like, was it Chesters + Spence? If you go back, you know, you don't assume perfect knowledge as you go back in time. All of the terrible engineering architectural infrastructure decisions that we've made that are actively ruining our lives on a day-to-day basis were not only reasonable, but the optimal decision at the time, you know?

JUSTIN: I was thinking about this earlier this week, and everything that we write will eventually be tech debt.

MIKE: I've heard it defined a little differently. All code is tech debt. It has a cost, right? It's infrastructure that you now have to maintain. It doesn't matter how awesome it is. It could be the most beautiful thing ever. It's still costing you money. You've got to maintain that thing. Unless it's out there bringing in revenue, it's pure cost [laughs], so it's only hurting you. And there's always...it's never in the middle. It's never purely making you money. There was a cost to building it. There's a cost to maintaining it.

WILL: Yeah. All my commits will die a hero or live long enough to become the villain.

[laughter]

JUSTIN: I'll put that on a t-shirt, Will, and send it to you.

WILL: Right [laughs]? Yeah. All that tickets, man [inaudible 31:07] nothing. Oh, dude, it is terrible when your tenure at a company has extended long enough that you start popping up in the git blames, where it's like, who is responsible for this?

MIKE: And you're the one doing the git blame [chuckles]?

WILL: Yeah. I’m like that fucker. [chuckles] God.

JUSTIN: So, I guess I got a question for this. It's like, given the fact that everything's going to change, I mean, you got your buy, your build, and your open source. I think there comes a time when you are going to be, you know, you're going to be switching in between those, especially, you know, you probably built a lot when the company was small, and now you're moving to buy and build. So, what can you do if you are small, or even if you're big, you know, what can you do to minimize the cost of that switching?

WILL: I remember reading this, and it sounded good, so I never checked their sources. But successful companies spend, I want to say, like, 10% of revenue on research and development. It could be 20, but I think it was 10. It was 10% is R&D money. If you're successful, over the long term, you're spending 10% in research and development. And I think, you know, you should be spending 20% on technical debt. If you're spending less, you’re screwing up. If you're spending more, then, hopefully, that's sort of you're paying off the old credit card bill, and it's going to get down to a slim amount.

But I think that's probably [laughs], like, probably, you know, 10% on research and development, new stuff, 20% on paying off the mortgage that, you know, your company makes its living on. And then, the rest is just sort of operations and line of business stuff. I think that's probably the right mix. Because if you're much under that, you know, I mean this is, like, long-term, broadly over the scope of a year, I think if you're much under that, you're running up your balance.

KYLE: What company gives you 20% for that? Because [laughs] I've never been in one.

MIKE: Acima, historically, said 30% for tech debt.

KYLE: Really? Okay.

MIKE: Yeah. There's been a bit of a tightening of the belt over the last year, but I think that that's going to be...we’re going to go out, like, another couple notches [chuckles] in the coming year, which is I think an important thing because I actually am aligned with Will. I think those numbers are about right.

KYLE: They sound good.

MIKE: You could argue is it 15? Is it 25%? But it's somewhere in that range. It's pretty close to the right amount. If you're not spending about that much time improving your stuff, it's not going to get better. But then, going back to the question, how do you avoid that problem? You know, sometimes you can't see the future, and you shouldn't try. But good software architecture never goes out of style [chuckles]. Decouple your things. Don't make a big, tangled mess. Build discrete components. I don't care if it's in a monolith or it's in separate services.

Make sure that you have separate components that can be separately maintained. There's a clear boundary between them, and [chuckles] they don't know about each other's business. Don't let things grow too big. And then, when you have to replace your component with the thing that you buy, it's not very hard.

JUSTIN: I totally agree, and I think that lends itself to not being a single-shop company. Like, oh, we're a Microsoft shop. Everything we own is Microsoft. Or we're, you know, a Salesforce shop. Everything we have is Salesforce, and it drives our business, and it drives our website, and things like that. So --

KYLE: Or an AWS shop.

JUSTIN: Yeah [laughs]. Or an AWS shop. The funny thing is, I was chatting...like, with all this AI stuff, and I hesitate to bring it in here, but the thing is Google and Microsoft both just released frameworks for having an Uber AI agent that understands all the systems that it's plugged into and --

MIKE: And so did Anthropic, by the way.

JUSTIN: And so did Anthropic, yeah. So, you have this mother chatbot that can do all these things that's owned by a single company. And, obviously, if it's like Microsoft, everything will be tied together in a single chatbot. And it gives my little security heart nightmares to think of that because if you crack that one, you got everything. But the thing is, can you just imagine, if you were a Microsoft shop and you had the Microsoft chatbot, whatever copilot they deem it be, it's going to be so hard to move off Microsoft because everything will be tied together.

KYLE: Yeah. You're at their mercy.

WILL: I think that's their hope. But I actually don't think it's going to go that way. I actually think a consistently well-designed AI-engineered sort of system is going to be predictable in a way that the coupling is going to be very portable from one chatbot to another. Because, yeah, Microsoft they're going to make their lock-in chatbot, but then Google is going to make their unlocking chatbot. And I’ve seen this thing is like the kind of thing that an LLM is going to be very good at porting over and translating from one to the other.

MIKE: I've got a case study at Acima. Not going to go into all the details, but we happen to have integration with an LLM because it's running part of our business. I’m not going to go too much into the details of the business [laughter]. But there's been some really important things we've built that use some of that AI tech, and we built a generic interface that none of our clients, internal clients have any idea which company is powering it behind the scenes.

Nobody is integrating directly with the provider, with, you know, Microsoft, Google, Anthropic. They integrate with our internal client. It makes you have to do a little more work, right? Now you have to build this in between, but because we have that, we could switch providers just like that. We'd have to probably rewrite some queries and do some validation, but none of the clients would have to change a thing. You absolutely can build your infrastructure. You can build with good architecture. If you always put that layer there to decouple, now you are not committed to that one provider, and you can swap it out.

WILL: I would say even if you're running on vibes, if you're running on pure vibes, say, okay, the LLM task is, okay, let's say it's my messaging queue. Okay, I'm an AWS guy, and I want to go out, you know, my...what is it? SNS queue. That's my event queue. And it's like, “Hey, LLM, write me a facade over this thing so that I have a uniform, not AWS-specific interface.” And that task is a task that an LLM, you know, coding buddy can take on and win.

MIKE: That's true.

WILL: Like, sophisticated refactoring? Maybe not for them. Because we're just moving data payloads around. This data goes in this pipe, you know, format the data for this pipe, format the data for that pipe. It's a translation. It's a translation problem scheme, you know, that you're coming through. The LLMs are really great at translating stuff. Really great.

I see the vibe, but I also see the win for, you know, empowering the vibes because you could define this ecosystem. It's not everything under the sun. It's our stuff that we fully understand, and we've exhaustively tested. I kind of like it. I'm kind of into this, you know, for just hammering out web apps? Oof.

MIKE: But that still led to good...I mean, you're doing vibe coding on an LLM, cool idea, but you're still architecting the system at a high level to be, you know...

WILL: Decoupled.

MIKE: What’s that? I thought I heard a word there.

WILL: Decoupled

MIKE: Decoupled. Exactly. Decoupled.

WILL: I mean, no, no, the vibe guy is on the beach with a pina colada. He's paying me to do that. Now you've transcended the vibes. The vibes are not enough. Like, now you need to elevate your game to somebody who actually does something. That's fine. You go prospecting, put your shovel down. If you strike gold, then you call me. Just be like, “I would like to keep this mine operating.” Why go to a [inaudible 40:32]? And I’ll be like, “Yes sir. Right away, sir. Make the check out to cash that.”

MIKE: So, you're saying the same thing we've been saying all along, though. You start small. You are willing to do things differently than when you hit that scale, and you should, and you should.

WILL: I love it, man. I love it. I see clearly the distance, my dream of the golden 30, where I could just get, like, 30 hours a week, just coming in, getting my stuff done, bang, bang, bang, being the overseer to a team of AIs. They're doing stuff, but somebody at some point in the chain needs to know what the hell is going on. That'll be me. God help a junior developer, you know, in the year of our Lord 2025. But I'm not that guy. I love it, man.

JUSTIN: My kids are all going through high school. Well, no. I got two kids in college and two kids in high school. And the high schooler, the youngest one, he is going through math class, and he has LLM helping him out. I mean, he basically really dislikes the teacher because he doesn't learn well from her for whatever reason. But he comes home, pops it in ChatGPT. ChatGPT solves it, and he's like, “Okay, really teach me how to do this.” ChatGPT goes through each step, teaches him half a dozen ways to figure it out. And then, he knows he learned better from ChatGPT than from that teacher. And my other kids have had similar experiences.

So, it's like, I'm kind of optimistic about the junior developer because they could go in, ask ChatGPT, “Explain this code to me, like I'm 5. Explain it like I'm 10. Explain it like I'm 5.” And they gain a deeper understanding of that code than a senior engineer perhaps would have the patience to teach them. So, I don't think it's all bad for the junior developer, other than the fact that the senior developer is suddenly going to be, like, 10 times more productive, and there may not be work for the junior developer. But for them learning, it’s --

WILL: Yeah. Well, I mean, that is effectively it. If I get more productive, then all the low-hanging fruit is gone. And the kinds of problems that I've worked with day to day, and the kinds of problems that I don't really see going away, LLMs are going to struggle because they're going to hurt. They’re going to hurt on the kinds of issues...because it's never just a repo or just a codebase or, like, how does this work together? It's just weird crap, you know? And that is --

JUSTIN: They may struggle now, but we're still early days.

MIKE: We are.

WILL: Yeah. You know, who knows?

MIKE: But, well, we talked about this a few weeks ago, that software engineers have become more productive consistently for at least 60 years, and the industry has only grown. The need for people who can think clearly about problems, and I'm going to say explain those, so be able to understand the problem that needs to get solved and understand the solution at a high level, is not going away. People who think clearly about problems are going to be needed, I think, for the foreseeable future.

Now, the tools that they use have been changing for all of 60-plus years, and I expect will change maybe dramatically with these AI tools. But businesses have never said...I've never heard of a business saying, “I'm making a bunch of money,” and then said, “Stop. Okay, that's enough. That’s enough. I'm making enough money. I don't need to do this anymore.” If we become more productive, so is every other company out there, and that competitive landscape just raises the stakes. And they're going to need smart people to drive the business further, or else they go out of business.

WILL: It's true.

MIKE: How is this any different than it was 10 years ago, or 20 years ago, or 50 years ago? You still have a competitive landscape, and now your competitor is using these tools, then they're way more effective. Well, you better be doing the same thing, and you better be hiring some experts who know how to do that.

WILL: That's true. I think it's true that we may be under...so, the technical sophistication that you need to quote, unquote, “vibe code,” it's like, oh yeah, I'm a senior software engineer, holy shit. Vibe coding is going to get rid of all of our jobs. And I'm like, yeah, but let's get a senior product manager and see whether the vibes are still flowing so freely, you know, not exactly the same level of vibes. I think Mike put it precisely. Our productivity has been going up and up and up and up and up decade by decade by decade by decade.. This is nothing new. And if I double my productivity, you are going to want three times as much output.

[laughter]

KYLE: The one thing that was pointed out to me on this was, you know, when I was coming up through school, it was, “Oh, you can't use your calculator. You have to do all this without a calculator.” And then, you start really thinking about that and how many places you'd never have a job where they say, “You can't use a calculator.” I mean, that's just ridiculous.

And to kind of go with what Will is pointing out there, it depends on your calculator as to how good you are. Some of the stuff that I've seen people do with those old TI-82s, or whatever, like, it was brilliant. I wasn't able to do, you know, some of the things and some of the equations and saving some of that stuff, you know, that they were doing. I was clueless. And I think we're going to run into that same thing with juniors, you know, they're going to be clueless what to really do, even vibe coding. You're going to need a senior. You're going to need somebody, an architect that actually understands what's going on in order to actually be able to efficiently use the tool.

WILL: I actually think...so, this is a little bit of a tangent. I'm going to go old man shakes fist at the sky. But, like, generationally, I think the late ex, early millennial, that we may have actually peaked in terms of technical sophistication because you still had to do it. If you're a teenager, do you know what a C drive is? You know, God help me, do you know what slash vol is? You do not.

You don't know that because you don't know...why would you invest in this arcana? But it matters. If you want to be in the business, that matters to you in ways that you don't understand. My generation we were both exposed to everything hot off the presses, like, every good technical innovation, like, up until this LLM, you know, it's a bunch of Gen Xers. We made you. We made LLM for you, Rugrats. You're welcome. It's our final gift to you, you know?

[laughter]

We got it all right as it was. People my age, like, 45, you know, thereabouts, early to mid-forties, we're not a bunch of COBOL programmers. All that hot stuff that the kids are getting, we made it for you. We made this, not you guys. It's not rock and roll or fashion or whatever. There's no new slang. This is just us.

MIKE: We're on a tangent. I'm going to roll with it and tie it back.

WILL: Yeah, I get it.

MIKE: So, I think there's something you said. Those of us who fit in that category, especially maybe the higher end of that category [laughs] in terms of age, you may also notice that there aren't so many car guys out there anymore. There used to be all these car guys. I'm saying guys. There are women. I'm using this generically. But there was a stereotype --

WILL: They exist. But you counted --

MIKE: They exist. But gender norms of the era [laughs]. There were a lot of these people who were car people, and it seemed like they were all over the place, and they’re becoming a rare breed. Because you don't need to have that expertise to keep your car running, because cars are better than they used to be, and they're becoming more technology-driven. So, you bring it to somebody who's an expert in that. And so, that generation is going. We hit peak car guy, right?

WILL: Oh yeah. Like way back ‘80s, maybe ‘70s, OBD2, baby. Just plug it in. Listen, tell me where it hurts, baby [laughs].

MIKE: So, you know, I think that there's something there. I mean, we've seen that happen at that other technology. We had this other technology where it's not like cars have gone away, but the person who can take care of everything end to end they don't exist much anymore. They don't even really exist. Those people work on classic cars. They don't work on the new cars anymore. And they might even shake their fist to the, you know, the sky around the new cars because the technology has moved on. It doesn’t mean the technology is worse. But yeah, I think that there's something to be said for that, that the end-to-end person is going to probably fall out of existence.

To bring it back to our topic, that build versus buy, you know, our expertise becomes specialized. And you've got to build the thing that actually provides value, and everything else you buy or you're working with another system. You build on that existing system and recognize that's how this works. You can't build everything yourself, and that's okay. In a complex civilization [chuckles], you wouldn't want to build everything yourself because there's a huge loss that you incur by saying you're going to do everything yourself.

WILL: I'm not smelting aluminum, man. I just want my Coke.

MIKE: Exactly [laughs]. Exactly. There are times you're literally part of the world that you should build. You should build your art and buy the rest, or use open source when you're at the appropriate level of scale.

WILL: Yeah. Get some samples. Get you a drum machine, some turntables.

KYLE: I think as we've hit some of these newer technologies, and kind of like you're saying with the car analogy, it's the same thing is happening in software where you can't have one...I guess things are more complex than the generalized do you buy, build, or open source? Because, in the platform world, we go between what's the appropriate option? And you got to look at all these things, too, where, oh, this is a great buy. But then we've got to employ the person that maintains that software for us. And then, you've also got to tack on, like, so that's been around, but the newer portion of this is infrastructure as code.

Well, this might be the top of the industry, but the next one down may look like a better option because, well, this one has infrastructure as code support. And maybe we don't want somebody that, you know, their entire job is to actually click ops this entire solution. We want somebody that's going to go through and automate all of this.

And so, now it's like options that maybe you wouldn't have taken advantage of prior you're taking advantage of because of other variables like infrastructure as code, or something like that. Because now you've got to weigh the costs of the offering, the costs of what it's going to cost to maintain, and then the cost of the code behind it. And those weren't things we worried about before. Like, you didn't worry about putting infrastructure as code there. That's, what, 10, 15 years new?

WILL: Yeah. They all want more. I mean, like, the past, you know, all the infrastructure, all of our technical infrastructure advancement, I think it's maybe thirds between better user experience, developer productivity, and increased performance, and stuff like that. But I'll be the first to tell you that, in my opinion, the past 20 years of Moore's law speed ups have all been consumed by shitty JavaScript.

[laughter]

KYLE: That's so bad, but it's true.

MIKE: You know, what's being consumed with now is kids asking ChatGPT to generate cat images.

WILL: God bless them. They're generating all kinds of images, all kinds.

MIKE: All kinds. That is true. And we're going to keep going, not on that topic [laughter].

WILL: Nope. Nope. You know, yeah, whatever comes out of the other end of your little creative writing essay about what you're imagining between you and God and OpenAI presumably.

MIKE: [laughs] I think we're coming to a good closing point here.

WILL: [laughs] I think we’ve shot way past a good closing point.

MIKE: Yeah [laughs]. We did. We went on this tangent where we started talking about the details of specific tech, but it comes down to these same trade-offs. It comes down to these same trade-offs. Hopefully, we've given you some things to think about [chuckles] as you're making these trade-offs. Where are you at? Where are you at in this cycle of your business? What trade-offs should you make, and what can you do? And Justin's question was well placed. What can you do to be prepared for the trade-offs you will inevitably need to be making in the future?

WILL: Absolutely. I think, in the end, I have to call it, like, an evolutionary process in that, whatever you're building, the initial step, I think if you're doing it right, if you're doing it the smart way, is whatever is the least resource impactful. If you are broke, if your budget is zero, you're going to slap something half-ass together, you know, that works and call it a day. If you've got a big enterprise, then get a hundred bucks a month subscription to something. Your boss will sign off on it. It's going to be fine.

And then, you take it as far as it'll go, and, at some point, the real art is going to be seeing that inflection point where you need to take a fork in the road. And either what you're doing is, in the gross majority of cases, something, you know, not essential to the business. Just buy something to take it off your plate. Or, you know, occasionally, you'll find something where it's like, this is a competitive advantage, and this is special sauce that only I could produce. Then you'll get there. And the art is identifying that fork in the road and not just getting pulled along, you know, whatever path you initially chose through inertia. That's really the art of it, in my view.

MIKE: That's perfect, I think where we'll leave things, and we'll see you next time on the Acima Development Podcast.

In this episode of the Acima Development Podcast, host Mike and the team—Will, Matt, Kyle, and first-time guest Vignesh—explore the topic of testing in production, using robotics and AI as a compelling entry point. Mike explains how reinforcement learning, while powerful, is limited in robotics due to the high cost and risk of physical failure. However, recent advancements in simulation environments have opened doors for safer learning before deploying robots into the real world. This analogy sets the stage for the broader discussion: the balance between risk and necessity in testing software systems directly in production environments.

The conversation shifts to real-world engineering practices, with Will sharing how his team manages mobile app rollouts using feature flags and staged deployments to control risk. Vignesh and Matt echo the sentiment, highlighting scenarios where pre-production environments can’t replicate real-world conditions, especially when working with third-party vendors, such as payment processors, where true testing isn’t possible until go-live. The team agrees that in these cases, observability, logging, and fast rollback mechanisms are critical. Production testing, when done responsibly, can create long-term resiliency in systems and processes, especially when it’s augmented with metrics, monitoring, and small, incremental changes.

By the end, the group reflects on the importance of ownership and accountability. They distinguish between testing and validation, noting that while testing should ideally be done before production, validation in production is inevitable and often essential. They also acknowledge the contextual nuance: startups may need to move fast, and students may operate in sandboxed environments, but in professional practice, taking testing seriously is a non-negotiable part of responsible development. The episode concludes with a reminder that thoughtful, well-instrumented production testing can be a powerful tool—if you approach it with caution, strategy, and care.

Transcript

 MIKE: Hello, and welcome to another episode of the Acima Development Podcast. I am Mike, and I am hosting again this week. With me, as usual, I have Will Archer. I've also got Matt, and Kyle, and Vignesh. Vignesh, is this your first time joining us?

VIGNESH: Yeah, this is the first time.

MIKE: Great. So, Vignesh comes over from the Rent-A-Center side, and we work with him a lot. He's a deep technology expert, been in tech for a long time, and he is a solutions architect. Should provide a lot of expertise and expect to be seeing him more in the future. So, great to have you with us, Vignesh.

VIGNESH: Thank you.

MIKE: Today, as usual, I'd like to introduce our topic by going kind of sideways and talking about something that will get us there. There's an interesting thing that we've seen in the world of AI. We've seen lots of text generation, right? If you haven't noticed, you've probably been living under a rock, but [laughs] there have been some really popular language models over the last couple of years that have made a huge impact. It's been a big thing.

And there's been other AI achievements, audio, you know, you can talk to your smart speaker, and it actually understands you. There's been progress in image recognition, image generation. But you know what there has been very limited progress in? Is improving robotics. There's been some, but it's been fairly limited, and there's a good reason for that. And it's really very simple.

If you want your robot to learn something, it's probably going to destroy itself while it does it. If you've got a robot that's strong enough like an industrial robot that can go and pick up a car, you probably don't want to say, “Hey, just industrial robot, go try and learn some stuff and see what happens,” because it's going to pick up things that are not cars that you don't want picked up, like, maybe itself [laughs]. Or maybe it's going to throw said car, or maybe it's going to use its internal hydraulics to destroy itself. In fact, this is a pretty common problem. And when you've got a million-dollar robot, that's a very, very expensive test.

There's areas of machine learning, and one of them is reinforcement learning. Reinforcement learning is when you try stuff out, see if it works, and then adjust your policy to be a little closer to what works. And, over time, you learn better what works. It's worked really well for games, but it's been very limited in robotics. And it really comes down to it's just crazy expensive. If you've got robots, you don't want to go break them. Robots are expensive and testing them in production, you know, is the only way to do it.

The only way you can test your robot is with a live robot. Now, that's actually changing a little bit in recent years, like, year, you know [laughs], they've built better simulation environments. If you can build a simulation environment that's really, really close to, you know, the real world and you can build a simulated robot, let it go out there; let it go ahead and destroy things; it’s fine. It's not going to break itself because it's just happening in software. And then, if you have it learn a policy that's pretty good, then you bring it to the real world and let it finish. Now that it knows not to do a lot of horrible things, it will do some reasonable things, and it's just getting a little bit better at them.

So, reinforcement learning is, you know, a big deal. It can be extremely effective. And they're just now getting in robots because if you couldn't test it ahead of time, it's just too expensive. It's just too expensive. They do have some lower-priced robots, you know, some people have managed to do with drones because they're fairly cheap. You can use a small one, and you can put it in a padded room. You can do some things to help keep it from being destroyed, but it's just really expensive.

There's a lot that we can learn about that, about testing. There are times when you have to test in production. You got your robot, and you want it to learn there. It's the only place you can. You cannot really do it in the real world. But, man, it is so expensive that almost nobody does. They don't release it because, without a test environment, you just can't learn.

Today, we're going to talk about testing in production, and [laughs] when we talked about this topic, you know, Kyle said, “Testing in production?” I said, “Never [laughs]?” [laughs] And we got a chuckle out of it. The truth is you wish it could be never, but sometimes you've got that robot you've got to teach, right? There's sometimes when it's your only option.

WILL: I do it all the time. All the time.

MIKE: Do tell, do, tell.

WILL: Well, I mean, so one of the main things that I'm doing right now for my major electronics retailers I work on their mobile app. And so, the thing about our mobile app is the release cycle is really long, right? So, I go out. I package the thing up. I send the app out to, you know, the approval board, right? Like, Apple's decently fast these days, but it's still maybe a day or possibly two to get approved, right? Google usually you can get it through the same day. And then, it rolls out to the customers, and it rolls out to the customers whenever their phones feel like downloading my update, right?

So, it goes out over the course of, I would say, like, a week or so, like, to everyone to 100%. And so, commonly, what we will do is we will release new features behind a feature flag, right? Which we can sort of manually turn on, you know, in test devices. And then, we can turn it on for a few people, and then a few more people, and then everybody when we feel confident. In fact, even our regular rollouts of a new version of the app we'll release it to, like, 1%, make sure everything's okay.

Because the other thing about it is we are releasing software on a platform that we do not own. Your phone, that's your phone, and you have a greater or a lesser level of control on what happens with that phone. You know, Apple's pretty locked down. Android, anything could happen. We don't have control over what's going on in your phone. You could be out of disc space. You could be out of memory. Anything can happen.

And so, there's a certain level of uncertainty around any release that we execute. That means we may need to claw it back. It’s easy to release to a production server when it's your server, but it's not my server. And I can only be so confident until it actually runs and everything is good, and it's like, oh, is it good on a Samsung? Is it good on an LG? Is it good on an Apple? You know, because they all rolled their own operating system, and they all have their own problems from day to day. Sometimes we need to claw it back. That's just how it is.

VIGNESH: I don't think we’re ever able to say, “No.” Everything done not in production, everything done [inaudible 07:21] environment go to production. That's impossible. Sometimes you have a lot of vendors. Like, if you're own grown solution, you may have all flexibility, but when you are integrating with others, you have to somewhere, you know, take a payment provider. You never have a real card testing, right?

You'll always have a dummy card, dummy things. They’re just a mock-up of data. There's not even running through the true system. They just do a lot of mock-ups on the side. This is the test card. You’re getting this number. I get test card. You return this response, right? So, then you only have option is before even rolling out to everybody as customers, you have to do your validation when you deploy. So, you need to have that production testing for certain cases.

MATT: Yeah. You hit on something really important, right? You need to control your risk, and that's extremely important. So, feature flagging, great way because you can react extremely quickly, and if something goes awry, you hurry and flip feature the flag off, and you mitigate your losses.

I just recently went through something, and, in fact, we were talking about robots and AI, AI-related, where we had to test in production because we didn't have the data we needed to properly test in a pre-flight or dev environment. So, you do as much as you can to mitigate your risk, and you have to keep a really close eye on so you can pivot quickly. And I think that's really the most important thing in production testing is you need to watch it extremely closely and be able to react quickly in case anything goes wrong.

MIKE: And you also touched on something there. In all the cases we mentioned, you don't have full control. And [chuckles] that's the reality sometimes, right? Sometimes you're working with a third-party platform you don't control, or you don't sort of control the data source. You have outside vendors. Credit card processing is a great example, Vignesh. It's one recently I’ve seen done where, yeah, you just can't. You want to tokenize a card to put in a mobile wallet? Good luck doing that outside of production because there is no sandbox credit card processing network [laughs].

MATT: And, Large Language Models, you could test a hundred times, and then it just takes that one request, right? And the model gets confused. It doesn't know how to respond, and you're going to get something totally unexpected. And that's when you have to make adjustments as you go. And I don't think there's really, you know, at this point in time, a way around it, unless you collect, you know, millions and millions of rows of data to throw against it and train on.

WILL: I’ll also say it's not all bad. Because one of the things about this sort of, like, a heavily feature-flagged environment, when you build that stuff out, you could ratchet stuff up and ratchet stuff down. Like, an emergent property of those kinds of things is you have fine-grained troubleshooting techniques where it's like, oh, with this payment provider, they're not having a good day today. Boom, they're off, right? No service interruptions, no problem. It's just like, okay, those guys aren't getting any of the traffic right now.

Similarly, if I'm having a problem, you know, there's a feature on my mobile app; it's, like, some backend, downstream services just having a day, and I'm just like, well, not today. You know, we're going to turn this thing off. You get everything else, but, you know, this aspect of the app is disabled until we can get it back up. This is one of those things where if you find yourself on shift where somebody else's, you know, experience or a backend subsystem is going wrong, there's only so much you could do. And having, you know, a button, you know, a panic button that you could press is a really valuable tool to have. And it's something you could get for free by testing in production [chuckles].

MIKE: Well, that's really interesting. So, the risk mitigation that you have to do to do something as risky [chuckles] as testing in production, when you're doing it right, when you're not cowboying it, but when you're actually saying, “Well, this is something I have to do, so I'm going to build as much structure as I can to make this as safe as I can,” is not single use. Having that system of feature flags actually builds resiliency into your system that can protect you operationally. So, when third-party service X goes down, well, you've already developed the ability to deal with that.

WILL: Exactly.

MATT: And we said, how often do we want to test in production? And everyone says, “Never.” You can also argue always.

WILL: Right.

VIGNESH: I don't think never is the right answer [laughs].

MATT: Well, right. And I would argue you always do. Yes, you want to go through a QA process. You want to do developer testing. You want to throw as many scenarios at something as you can prior to releasing, but you always want to go test and validate in production, post-release, always.

WILL: Yeah, that's the final...like, I think, informally, everybody does that, where you could have done every single possible peer review, code review, you know, linting, QA process, but I always check it. It’s like, okay, we rolled it out. I'm just like, okay, I'm just going to run one.

[laughter]

VIGNESH: Unless, you know, the point you mentioned, you know, I have a percentage of rollout, rollout for 1% or 10% of customers, monitoring a log. So, if something goes wrong, then I roll back. But you have to sit through that time, also, hey, that rollout process. But most of the system doesn't build that way. I know that those rolling out, like, percentage-based it’s very few systems right now and few companies does that one. Definitely, you need to validate yours in production, not just say, “Hey, it's going to work it out.” And a lot of times, you know, that's not the true statement.

MATT: Right. And if you don't, there's a really big cost because bugs go unnoticed and unreported because customers may be running into them and just bouncing, and that whole time you're losing money. Whereas if you would have went and validated in production, caught it early, you would have saved the company and yourselves a lot of time, and money, and effort.

MIKE: So, the monitoring instrumentation, visibility is how you make it work is what I'm hearing. Kyle, you might have some opinions on that [laughs].

KYLE: Yeah, I was just sitting here thinking, so it kind of seems like we can't, or we shouldn't be doing any of this testing in production if we don't have the observability behind it, if we don't have monitoring, if we don't have the logging because you want to do metrics. Because it sounds like, you know, feature flags and canaries is what we're talking about, sorry, canary deployments is kind of what we're talking about and doing a subset of data. And we need to know how each of these different systems are performing.

So, more than any other environment, it sounds like it's more important just to be able to have that kind of information for these testing scenarios, especially with the idea of load testing, too, because you don't get a more realistic scale of your services than load testing with real user data.

VIGNESH: Yeah. I mean, nowadays, due to microservice model, a lot of this new tech stack, which we will [inaudible 15:27], even though it's our environment, but we don't technically own a lot of things, you know, behind the scenes. Observability is the key. Nowadays, you know, even before starting working, the first thinking you should come in mind, say, how do I monitor this one? You know, what is my criteria failover scenarios? But I don't say 100% time as Matt said it. We don't need to test in production, but there is certain areas you definitely need to validate in production. Otherwise, you know, you may end up risking, especially, like, payments or some of the decision engine, something you have to run a dry run to make sure, hey, it's working [inaudible 16:07] connection.

Sometimes I saw, you know, deployment people think it works, and if one person deployed an hour before, especially when you are deploying cross-team application deployment, that's even critical your application connections is correct. If you think, I did it; my application is working; there's no logs, and you went off, then other one is doing, gone. One time I saw, like, six hours was an issue. So, that's where the five minutes taking just validating either through observability, I mean, yeah. One way of fixing is...I said two ways, right? One, you do real production testing, other is your observability is key point.

MATT: Yeah. I mean, it's critical. And when I say 100% of the time, I do mean it if we're talking about changes in business logic. If I say, go make that border a different shade of blue, probably okay to not go do production testing. But if you're changing business logic, I think it's really important, especially when there's the possibility of a lot of edge cases that just aren't going to get caught in quality assurance testing and dev environments.

VIGNESH: For example, this Monday, we are going to release, not this Monday, the following Monday [inaudible 17:30]. We said we have to test a program. We don't want to just say, “Hey, all the services deployed working. The customer is going to validate it.” No, we want to validate before even saying, you know, it goes to calendar, we have to: the new features, new integration, new business logic. It's worth to test it than just assume it will work.

MATT: Yeah, absolutely.

KYLE: There's a subset of testing, right? As much as we would like to test everything, there's RUM, right? And we can't get RUM without actually giving it to the customer, to the user. And that's, you know, as much as any developer would love that they use the product the way that we intended them to, they're always going to use them outside those bounds, be it a large percent or a small percent. It's not going to be used quite as we expected or hoped some of those times.

MATT: Yeah, that's really hard to predict, too. I mean, user behavior, you know, psychology, and you can't always predict everything somebody's going to try to do.

MIKE: So, it's interesting. We started, and we're still talking about testing in production, and it's almost 100% come down to validation and observability, mitigating risk by making sure you're watching it, doing slow rollouts. There's process around this risky thing you need to do that you can use to make it work. It's a fascinating idea, what do I say, and we all arrive to the same place. You make this work by being as careful as you can, and you only do it when you have to, but in some ways, you're always doing it.

When can you avoid? When can you avoid testing in production? Because there's a lot of times you think, well, I have to. So, give me an example. I'm going to be a little bit vague here because I’m not going to go deep into the business. But sometimes there's third-party providers that have sensitive data. You think about, like, credit reporting agencies, for example. They have a contractual obligation that you will not give them test data because they don't want to have their data polluted.

So, if you try to test with them in production, you're violating your contract, and that's not a good situation, right [chuckles]? So, there's, you know, there's a, well, you cannot, and in that kind of case, well, yeah, I'm not going to test in production because I can't. I'm not allowed to. My contract says, “No,” so all I can get is observability.

But there are other cases where like, well, yeah, I could probably come up with a test for this, but it'd be easier just to roll it out. And this happens especially at small companies or new projects, things that are pretty small. You're like, well, yeah, I don't have to set up my test environment. I can probably just roll it out there. What should that boundary be? Are there times when you really can get away with just putting it out there and seeing what happens?

MATT: Yeah, I think so. Something you have to keep in mind is how sensitive is the data that you're dealing with, right? Are you dealing with people's financials, health information, things like that, where it's critical that it remains secure? And then, there's other things that you can just throw out there and see how it goes.

I think, you know, we went through a phase here, I don't know, 10 years back. It started roughly with the whole gig economy thing. I think that's a perfect example. Everyone, not everyone, but a lot of people were out there trying new ideas and things, trying to be fastest to market with this gig economy and just throwing a bunch of stuff at the wall to see what sticks. And you had to move fast, or you get left behind, and then you find your big players in that market, right?, DoorDash, perfect example, Uber Eats. You know, there were a million food delivery services, and the survivors are the ones that move fast to market and spread quickly.

MIKE: Well, did they get there by having lax testing processes though?

MATT: No, I don't know that's the case, but I think they were a little more loose with the features they were rolling out and liberal.

MIKE: So, there's a difference there, right? Say, “Well, I'm going to test this out. I don't know how this is going to work from a business use case. And I don't know that we need to architect this really well because we just need to trial it.” But you probably want that feature to work and not damage your customer relationship by always just having broken code.

MATT: Sure.

MIKE: So, there's some nuances there I'd like to tug at a little bit. So, I agree, you're a startup. Getting out to market is your life, right? You do not survive as a business unless you're trying stuff out and doing, just as you described, getting out products that aren't complete that maybe have some weirdness to them, right? But it's another thing to say, well, I'm just going to throw out garbage. And I’m not even going to bother testing, just throw out whatever, because that might not actually be faster.

MATT: Oh yeah, I mean, you always pay the price, right? With your debt. You're creating your relationships. And if anyone's just pushing out garbage, they probably shouldn't be in the business honestly. You always want your stuff to work. And I don't know that in my entire career I've ever actually pushed anything that wasn't at least manually tested. You know, prior to the days of TDD, we did that all the time, right? But you test your happy paths, maybe an edge case or two, but you don't test everything.

I worked at a startup and was kind of the lone wolf for quite some time just due to resources and funding, where the CEO would just come in my office and say, “Hey. I want to do this thing. Can you have it ready by tomorrow?” And I'm thinking, well, that'll take three months, four months to build, but I guess [laughter]. And you prototype something, and sometimes that prototype is good enough to get out in the wild and just see how people behave to it, right? It's not perfect. You're going to run into issues, likely bugs, but it's good enough to test the market. And that's kind of where I was going with that.

MIKE: And you hit some important stuff there, that you care about validating, right, about making sure your system works, and any responsible engineer is going to care about that. On the flip side, there are going to be business needs, whether that be internal business needs or external ones, right? Internal ones say, “If we don't get this out tomorrow, we lose a client.” External ones say, “You cannot contractually test this,” or “We don't have a sandbox. Sorry. We have to test this in production,” you know, the credit card network, for example, that force you to test in production. And that's the reality of it [chuckles]. That's the reality of it. There's sometimes you have to. But you talked about debt. Every time you cut corners, you make things a little harder for next time.

MATT: Absolutely. Absolutely. And that's the price you pay to move fast.

KYLE: I wonder, too, how much has this faster movement affected testing in production? Like, when you've got a release cycle that is, you know, once a month or every two weeks or whatever, how is your testing methodology in production changing compared to if you can release to production 10 times a day? You know, are you making...I would hope at least that we're shipping out code that's easier tested and doesn't need as much testing type thing, and there are going to be much smaller and less effective changes in that case. But are we less cautious because we can iterate so fast?

MIKE: One thing...we've talked about this before. The larger your release is, the more complex it is. And it very quickly becomes impossible to truly test every path through the code, just mathematically [chuckles].

KYLE: Too big of a matrix.

MATT: Yeah. I mean, that's why, you know, moving into the world of CI/CD is so important. You are mitigating so much risk versus waterfall style where you're releasing, you know, 20 features in a single release. You find a problem. Those problems sometimes are extremely hard to track down, whereas you're releasing a small incremental change, really easy, less impact, and you can react way more quickly.

MIKE: You did kind of hint at something there, Kyle, about becoming complacent. And you go, wow, I've tested these are always working, you know, I'm really good at pushing out these small releases. I always test them. Maybe I forget to even think about validating it. And you knew it was important, but you just got so good at your process that you started missing parts. And you start believing that it's going to save you every time.

MATT: Yeah. And that's dangerous because you don't think about downstream effects and impact, right? You think, oh, well, this small thing I did is isolated here. But that change could impact things further downstream and cause real problems. And we've all seen that before.

MIKE: Yeah. So, the rules: while continuous CI/CD is huge for reducing risk, it's just reducing risk. It doesn't eliminate risk. And all the practices we've talked about, particularly the validation and having all of that instrumentation, whatever it is you do to make sure that it works, which may just be going up and testing it, is vital. That never goes away.

MATT: Yeah. I mean, we can get close, but we're never going to be 100% of anything. It's like solving for pi, right? The farther you go, the closer you get, but you're never actually going to solve all those problems. And you run into that a lot with machine learning and artificial intelligence. And sometimes the closer you try to get to that 100%, the further off it pushes you because of the impact downstream that you just aren't seeing at the time of change.

MIKE: So, we've talked about some good practices, about sometimes when you need to get there, about times to, you know, where you can't. So, we haven't talked that much about when you don't. And we've talked about, you know, yes, you're always testing in production. You've always got users who are using your stuff differently than you expected. So, I’m going to ask a simple question.

So, I mentioned it before, you know, when do you test in production? Well, you don't, yeah, never. Of course, it’s not true, as we've talked about thus far. But where does that sentiment come from? Why would we have the gut reaction, well, no, you don't test in production? Why is that the case? And what are we actually trying to say?

MATT: I think the reaction comes from data. You're muddling your data. When you test in production, you're not using real data. Your reporting can get thrown off, things like that and, you know, the risks you talked about before, contractual stuff, compliance issues, those types of things. So, maybe we think about things a little differently, and we talk about testing versus validation, you know, actual testing. Your tests should be written. You're not, you know, you're trying to get as much coverage as you can with written tests, with quality assurance, developer and peer testing before you send it to production. Once it's there, you want to validate. Sometimes you can't with reliance on third parties that you can't control. And that's when you're kind of forced to really test in production versus just do validation.

MIKE: I'm going to go back to the industrial robot thought, right? Let's say you've got a million-dollar robot. If you had a reasonable simulation environment, would you test in the real world without testing in the simulation environment first?

MATT: Absolutely not.

MIKE: It would be just burning money.

MATT: Yeah, and irresponsible.

MIKE: Irresponsible.

KYLE: But there's also a cost to getting that simulation environment.

MIKE: Exactly [chuckles]. That simulation environment is really expensive, and there's companies that have been spending a decade or more working on building that sort of thing. Companies like Nvidia who have a vested interest in having more companies using their product are trying to provide an open environment for people to test in because it benefits them. And so, they put millions, a billion dollars, you know, just vast sums of money into developing that because it is so expensive. Individual companies are going to have a really hard time doing that on their own.

MATT: Yeah, and it's like we talked about with the debt you're creating by moving quickly, right? The cost over time may be much greater than the cost you front load with in setting up that environment. It depends on what you're dealing with. You have to do the math and say, is it worth it?

MIKE: That's a very conscious decision. You say you do the math [chuckles]. You don't just kind of roll with your gut and say, “Ah, it'll be fine,” right?

MATT: Yeah.

MIKE: You look at the value of what you might damage, your relationship with your customers, financial information, you know, loss of information, and, you know, these extremely...well, I'll say particularly financial information, right? I mean, it's just not something you ever tinker with. The risk is too high. It's irresponsible to do anything with your customers that is not extremely well-vetted. Or even more, what about medical hardware?

MATT: Right. When you're dealing with people's lives.

MIKE: Yeah. You know, you test that in every possible way that you can before you even consider going into production. And where the risk is high, where the cost of the risk is high...it's not just financial cost but however you define that cost because, you know, people's lives that's high cost. The cost is extremely high then you're careful. You're careful.

MATT: Yeah. Use good judgment. I mean, everything isn't black and white, and not everything is objective.

MIKE: To flip it around, we said, well, there's times you don't, and we've also talked about low risk. And this is, I think, maybe where testing in production gets its reputation from. You've got students in school. They're writing the code for their project. And a lot of them aren't going to have a test environment. The risks are so low, right [chuckles]? Do I pass the class or not? You know, do I pass this assignment or not? And the cost of building a test environment is high enough if it's a small project, right?

Rationally, they look at the risk-reward and say, you know what? My test environment is the TA [chuckles]. I will be testing against the TA, and if they can run the code, then that is passed. And that's a valid test environment in that scenario. I could argue against it, and say, well, students should be learning to write tests. Well, they should, but if the system doesn't incentivize it, then they're going to make the rational choice [chuckles], which is don't do the extra work because I have a tester. I have a test environment built. In fact, they probably built one in the class that's going to test this for me.

MATT: Yeah. And, in that way of thinking, you're putting the reliance on somebody else, you know? Just a little bit of advice for any listeners: have some ownership; have some accountability and try and do as much of this stuff yourself, and don't always rely on somebody else to cover you.

MIKE: It's a sandbox, right? School, or, you know, bootcamp or, you know, your own test work, you know, your own learning. It's all sort of sandbox. But in the real world, testing matters [laughs] and not ever relying on somebody else is one of the things...now, let me state that a little differently. We rely on each other heavily, and it's the only way you get software made. But not putting the expectation on somebody else where you could be reasonably be expected to do it is the way that you end up building good software and moving ahead in your career because you care.

MATT: Yes, and good relationships with those around you. In this case, don't ever think it's not my job because it is.

MIKE: And that may be a good way to tie this one up [chuckles]. It is your job. All of us, you know, we've talked a lot about the exceptions, but in the end, in almost all cases, it's your job to validate before it goes into production and after it's gone into production. You've got to do both. And owning that, knowing that you, you know, this is something that you built, and it's your, what do I say, stewardship, you know, it's something that you take personally, is what's going to lead to the craftspersonship, where you truly become an expert who can build great things.

MATT: It's my name on my commits.

MIKE: Yeah. Well, thank you for listening. Thank you for listening. We've covered some good stuff and not one of our longest sessions, but really pointed the critical importance of visibility, monitoring what you're doing, testing it as early as possible in the process, which may be in production in some cases but doing everything you can to mitigate that risk so that you treat your customers and their data with respect.

And until next time on the Acima Development Podcast.