Acima Development

Episode 90: SQL as a Superpower


Listen Later

Mike kicks off with stories from his career to argue that SQL is a “never-goes-away” superpower. He describes early jobs where everything was handcrafted queries and good database design was foundational, then later roles where rapid growth made inefficient queries and missing indexes painful fast. Even with modern ORMs making raw SQL feel like a “code smell” in app code, he still relies on SQL constantly for investigating patterns, diagnosing anomalies, and answering urgent business questions in real time. His core point: avoiding SQL is like avoiding algebra or relying entirely on GPS—you can get by, but you’ll be weaker when you need real problem-solving power.

Will pushes back with a reality check from big enterprise environments: many engineers simply aren’t allowed to touch production databases for security and scale reasons. He explains how, in those worlds, “SQL skills” get replaced by working through service boundaries—mocking/spoofing microservice requests and relying on managed interfaces rather than direct queries. Mike agrees scale changes access, but argues the underlying concepts still matter: relational thinking, knowing what’s expensive, understanding how data is shaped and retrieved, and especially understanding concurrency and locking at the database layer. They trade war stories about bad concurrency patterns (like incrementing an integer in a table inside nested transactions) causing real production pain, and riff on why older systems leaned on sequential IDs vs UUIDs due to historical CPU and memory constraints.

The conversation broadens into “fundamentals change how you think.” Dave argues that specific jobs (like DBA roles) may evolve or disappear, but the principles behind them are evergreen—much like learning Lisp/Clojure or watching SICP to internalize functional, transformation-based thinking. Mike ties this directly to SQL as a declarative language: you describe what you want, not how to do it, and that mindset carries into MapReduce, streams, comprehensions, and even modern AI prompting. Eddy and Thomas add practical perspectives: ORMs can hide SQL until you need verification, debugging, or analysis—then SQL becomes essential (especially in support/data roles). They end by stressing curiosity and communication as the real career accelerators, capped by Dave’s interview story about spotting a SQLite aggregation quirk: the takeaway isn’t “memorize tricks,” it’s “stay curious, keep learning, and you’ll keep moving up.”

Transcript:

MIKE: Hello, and welcome to another episode of the Acima Development Podcast. I'm Mike, and I am hosting again today. With me, we've got Dave, Eddy, Will Archer, and Thomas. I think we're all return [chuckles] panel participants here. I'm going to jump into, actually, kind of a series of stories here, talk a little bit about my career.

My first full-time dev job, all of our database queries were handcrafted. We did use parameterized queries; that was supported. But this is in the time before you had Object Relational Mappers, you know, ORMs, they call them now, before they were widespread. I'm sure they existed. They probably existed for a long time. It was before anybody really used them.

The leader of my team cared a lot about data, and we focused a lot on good database design as the foundation of our work. We always kind of started with the database and then modeled way up. It's a common way now, isn't always common. You can tell when people don't do it that way. He even discovered a bug in the PostgreSQL adapter for Java and submitted a patch to the project, so, you know, it was a thing.

Those were both kind of new tech at the time, so [laughs] times have changed. At the time, I hadn't really done much with...okay, now here's a pause. You can call it SQL or sequel. I don't care. And I'll probably call it both in this conversation. It's a debate with no clear answer, and it really doesn't matter, so...[laughs]

DAVE: It's a debate with no clear need, right?

MIKE: Yeah.

DAVE: Because if I say, "Sequel" and you say "SQL," I know what you meant.

MIKE: Yep. So, I hadn't really done that in school, so it was this interesting new tool to learn. It's this language that's oriented around relational algebra instead of implementation details like loop counters or sorting algorithms, right? And different's fun, right? It's the cool thing. Wow, this is a different way to think. It was more than that, though. I got to...I enjoy...I started to, and I still enjoy thinking about problems in terms of a series of transformations on data. We could probably talk about that. I think it's a big deal.

Sometime later, that company had been acquired. I ended up spending months just writing queries, like, all day, every day to drive reports, to make them efficient, to run quickly, so many queries. Probably wasn't the best way to do things, but it's what we did. I wrote a ton of queries back then. You know, at a later job, I've probably mentioned this before, our traffic grew 100x in a year, startup, right? And we were working with publishers.

So [chuckles], we grew really fast, but it meant that anything inefficient became a bottleneck really quickly, like, weeks. Within weeks, a thing that worked great doesn't work anymore. And so, I added indexes to tables I don't know how many times. I added external indexes for full-text indexing, like twice. We did it once, and then that tool failed, so we did another one. Designed a new database schema for arbitrary web content, you know, wrote the queries around it.

We even had a little micro language for publishers to use. I didn't originate that, but it was something we worked with to query their data into their pages. It was queries all the way down. It was just queries is all we did. Later, we built a data warehouse that powered, you know, analytics for our users, and, again, SQL [chuckles].

In recent years, those Object Relational Mappers, or ORMs, they've gotten really good for web development, right? I mean, it's the standard. And I'd say that using raw SQL in production code there's usually a smell nowadays. And I've been a lot less hands-on in the code for several years now, but I query the data warehouse all the time.

Just in the last few weeks, a couple examples, I've pulled stats on historic traffic patterns and all sorts of configurations. I don't know how many times I've run the query, tweaked, to let us know what to expect for seasonal load as we went into holiday season, lots of shopping. I think it was last week with a group of people in a conference call. They're all watching me live coding, one of those make you sweat experiences.

But, you know, I wrote this fairly complex query to show the average daily time between merchandise, like a customer choosing to get the merchandise and when it got shipped, and to show that a big vendor, that I'm not going to mention, was temporarily delayed. And so, this anomaly we saw in our system, because they had a big impact on us, was external and not a problem in our stuff. You know, live coding is always a little dicey, but it worked. And, fortunately, I had done this kind of thing before.

This is all to say, knowing Sequel, SQL, call it either way, is as important to me today as it was at the start of my career. You know, way back then, like, wow, this is an important new tool I need to learn. And I'm still using it as much today as I was back then, and it just hasn't changed. It's still important. I haven't written Perl in decades, I don't think [laughs]. Anybody remembers Java applets? [laughs]

DAVE: Yes. Wow. Yes.

MIKE: I could talk for a long time, right? We could make a whole episode on dead or unpopular tech that was once a thing. But SQL, it hasn't gone away. Getting data out of a database is just this critical skill that never goes away. So, today we're going to talk about Sequel, or SQL, as kind of a superpower. You can get away with not knowing it now for a long time because you've got good ORMs. You've got visual query tools. You've got, like, the Business Intelligence Team they provide queries for you.

But based on my own experience, I think that avoiding learning it it's like never learning algebra or never learning how to plan a route without GPS. Like, you can get by without it for quite a while, but you'll have so much power to solve problems you couldn't otherwise solve if you learn the amazing tool.

WILL: You know, like, I love that, and I have, like, a similar...I have a similar arc. But as I was listening to it, I was thinking about, like, well, when's the last time? So, I mean, I'll just say, right, like, I think you have a unique and privileged position in that, like, you are allowed to run SQL. Most folks aren't allowed. You don't get to run SQL.

And I thought about, like, what I had sort of replaced those SQL query skills with. And to be perfectly honest with you, like, you know what I mean, because I'm working for, like, big boys, like, you know, like, I went, you know, I worked with you guys at Acima, and Acima's not small. And then I went, you know, an order of magnitude bigger someplace else. And I went another order of magnitude bigger at someplace else. And, like, you don't run SQL.

Like, I'm pretty, you know, I'm pretty up there. I'm a software architect II, you know, which is pretty, I mean, it sounds cool, right? There's a cool sound. It's got a cool sound to it, you know. They trust me with some stuff, but, like, I don't get in that database, never, never, ever. And, like, you know, and so, like, you know, you could do that, but I'm not allowed to do that.

But I have all these tools, and I love SQL. And it would make my life a lot easier if I could do it some of the time. But I've kind of replaced SQL with microservice request spoofing, you know what I mean? Microservice request spoofing kung fu, in that, like, I can invent an entire data layer out of whole cloth and just be like, you're talking to them. And my application's like, are you sure? And I'm like, shhh, trust me, trust me. It's there.

DAVE: Just talk to the interface.

WILL: It's not some reverse proxy, like, three-card money thing. This is just, shhh, you know. God, man, I mean, I wonder whether, I mean, it might just be my lived experience in that, like, like I said, I'm working for big boys, you know what I mean, that have, you know, I hate to call it...I don't want to call it, like, stumbling blocks, but, like, you're pretty robustly firewalled, you know. Because the SQL people can't see the things that I see either, you know.

Gosh, I don't know. I don't know. I mean, is the game evolving where, for big enterprise development, SQL isn't the thing anymore, actually? It is not, actually, in fact, the thing. And the data that SQL would be providing to you is managed microservices that probably, eventually, I can only assume [laughter], have a database attached to them somewhere. But, you know, I'm sorry, like, as you're talking it through --

MIKE: No, well, you got a good point.

WILL: I'm like, they don't let me write SQL. I haven't been allowed to write any SQL since back in my Acima days. And part of that's just maybe the work I'm doing but anyway.

MIKE: Well, there's probably a scale. I mean, we've talked about a few things that scale has an impact on. If you're not a massive, mega corporation, you might have a really hard time working with big contract shops, for example, and I think we've talked about that.

WILL: [laughs] Moving on. Moving on. Keep it moving.

MIKE: I'm not going to go deep into that one, moving on [chuckles]. But there are some things where scale makes a real difference. And so, yeah, if you're at a big place, maybe you are using some other tools because there's layers in between you and the database. But a lot of the principles are probably the same. And understanding that idea of…understanding the concepts of maybe relational algebra, what it means to pull data, what might be a bad idea, probably still applies here, even if you don't get down to the language, you know, question mark [laughs].

WILL: Well, I mean, I think, I don't know, like, another thought that I had, something that I was going through my head when I heard the topic, like, you know, at breakfast time, was I was thinking about, so, like, I came up, like, my career has been, like, a little bit weirder, right? And then I came up from the bottom, like, not moving bits, moving atoms. Like, do you want to know the voltage level that a CMOS transistor considers a one versus a zero? I actually know that [laughs], you know, and all kinds of, like, crazy stuff, down from the atoms all the way up, right? And I learned a lot of things that you don't need to know, and there's no value in you knowing.

But one of the things that I learned that was very, very difficult for me to learn, but it was absolutely worth the price of the mission, is concurrency primitives, right, and how, you know, multiple processes can access shared data in a, you know, a non-destructive kind of a way. And modern web development, you know, when people do async await, right, which is good, right? But, like, all of the hardcore, nitty-gritty concurrency all happens at the database layer, all of it. Like, the database layer is where that stuff happens. But it still happens, still happens, and it's still...it's very, very real. You know, it's as real as it comes.

MIKE: I've got a story about that. I can't go into too much detail, but scaling up on a Black Friday, there was locking. I was in a situation where there was locking around a value. And it wasn't using the native database concurrency or tooling, not [inaudible 11:53], the native database tooling to drive a sequence. It was just an integer in a table and in a transaction [laughs]...it started a transaction, incremented the value. For those of you who are listening, Will is just wincing [laughs]. It turns out that this transaction also ended up nested in other transactions, right? So, there was significant amount of work going around incrementing this heavy traffic.

WILL: Oh my God.

MIKE: It worked for years. One year, wait, it's not going to work anymore, because you can only fit so many of those in a minute, and you start to keep up.

EDDY: I called this out, I think, a couple of months before it became an issue. I was like, wait, how are we creating this value? Dug deep, and I'm like, wait, this is really strange. Anyways, like, the whole nested transaction thing on creating the next iterative number was a little obscure. And I basically said, "This is going to come back and bite us," and sure enough, which kind of brings up a question, why not use UUIDs, right, versus just IDs, right? I think it's safer. I really do. I do.

DAVE: Is it?

EDDY: Yeah, because then if you --

WILL: It's safe-er.

EDDY: It's safe er, and I think that's still considered a primary key, right?

MIKE: It is.

EDDY: So...

DAVE: As long as it's unique.

MIKE: And this particular situation it was generating a user-friendly ID that some customer could easily read off.

EDDY: That's right.

MIKE: But it didn't want to have, you know, collisions. There were so many different ways it could have been done to let the database do the work rather than trying to do it manually. And there is a real business impact.

EDDY: So, again, I kind of am curious, though, because I don't understand, like, the full scope of why not use UIDs, you know, from the database level. Because, at that point, if they're already indexed, right, you're not losing any speed whatsoever when you're trying to read. Why not just use a UID out of the box? Even if you happen to leak the entry, like, it doesn't matter, right?

WILL: Because it's old and, like, and we didn't do it the smart way. In the bad old days, in the bad old days [laughter], it was just like, well, 1, 2, 3, 4, like, this is...okay, we got it going. This is sequential. It all makes sense, you know what I mean? Like, the UUID thing, like, that was, like, I don't know, that's, what, just 20 years old, not if, not even, you know. Like, it's pretty...I don't want to say new, right, because if it's been 10 years, at least, since people were like, yeah, okay, the 1, 2, 3, 4 thing was kind of dumb, you know, like --

DAVE: I don't think we hit that point, but the C definition of an integer, when you look up in the C manual, like, what size is an integer? Like, we're used to it's 8-bit, 16, 32-bit. There's a really interesting formal definition of the size of integer in the C language, and that is the data type that is the most efficient for the processor. So, if you're on a 14-bit processor, which I have been, it's a 14-bit word. That's the size of int on that machine.

30 years ago, when you're running at 5, 10 megahertz, taking time to calculate a GUID, especially when we didn't have a real good source of randomness back then that we'd figured out, it was catastrophic, right? We couldn't spend 20 clock cycles, you know, generating an ID. Let's just grab one, increment, and go. So, I mean, for efficiency, they were key.

WILL: Yeah, I bet you a million dollars. I bet you a million dollars, but why, right? But why is the 64-bit operating system shift when all this stuff went to 64-bit words, right? So, like, this ID needs to be...it needs to be a word for reasons around CPU efficiency because, like, you know, we don't care about the CPUs. CPUs are free forever. No, no, no, no, no, no, not at the database layer. They care about the CPU, and they care about that primary key fitting in one database word, right?

So, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, okay, that? I can do that in a 32-bit word, no problem, and I could do pretty good, right? And I could probably go down to 16-bit, right, because we've got 16-bit databases. I don't think we have eight-bit databases. I think that's probably too small to work. But, like, that 64-bit word architecture swap over, where, like, it just, like, you know what I mean, whereas, like, 64 bits is, like, the size of everything. That's it.

Because here's the thing about, like, and we can get, like, really low-level, nitty-gritty, which is what databases do. But you have these UUIDs, right? And so, the UUID is, you know, it's structured. I don't know exactly how it goes, but I know it factors in, like, the CPU clock time at, like, a microsecond level, right, like, to create that word. And so, it's got a lot of wasted space.

DAVE: And the serial number of your network card is in there as well.

WILL: Yeah, and the serial number...yeah, and just, like, it's got, like, stuff, right? And, you know, you hash all that together, you get a UUID, right? And, like, and it's more complicated than that, which is why I don't want to, like, talk too much about it because people put some really smart...they put a lot of brain power in terms of, like, creating these universally unique identifiers. But the consequence of that is, they have a lot of wasted space, right? They got a lot of wasted space. Like, it's, what, I don't know, like, a 32 characters or so, right? Like, it's a sparsely populated, like, number space.

And so, when memory and, like, word size were premium, you couldn't afford to be messing around like that. You know, like, if I've got 32 bits, or God help me, 16, I'm going to need all of those [laughs], you know what I mean? And so, anyway, I mean, like, it is the battle days, I mean, in that, like, yeah, things were bad, and a lot of things were harder, but it wasn't that people were dumber. They just had less to work with and also less access to information. That's a legitimate thing.

But, like, also, like, we were squeezing those computers until they begged. If you ever want to know, like, the pinnacle of, like, bad old days, smart but dumb, but smart programming, is people have breakdowns of old video game machines and, like, all the black magic, all of the voodoo that people did to just squeeze an eight-bit Nintendo's CPU...I think it had a Motorola in there or something.

DAVE: 68.

WILL: But, like, just squeeze it until it screamed. Like, they took those old, you know what I mean, the old video game programming breakdowns, if you're a dork, are just incredible. Anyway, so, you know what I mean, like, you know.

DAVE: Absolutely. And you can still see it.

WILL: [inaudible 19:28] wasn't an idiot, you know. He did it for a reason [laughs].

DAVE: You can actually see that. Like, a lot of the programmers that were really big into, like, bit shaving, you know, in the '90s, they're like, oh, well, these 64-bit processors with 60,000 cores and a bajillion megahertz, da da da. And then somebody says, "I can't get my software to fit in my Raspberry Pi or my Arduino, or my Teensy, my Teensy…" God, I did so much code for the Teensy.

And, you know, it's like, I had 4K of RAM in the thing, no, sorry, 4K of ROM. And I only had, like, 256 bytes, as I recall. It was one of the very, very first ones, teeny, teeny, tiny. And all of a sudden, all those weird hardware hacks where you're like, oh, I'm going to do a division followed by a modulus, but I've very carefully chosen my modulus to be 65536, which means all I have to do is do the multiplication in a 16-bit register, and I get the modulus for free, like, that kind of nonsense that you play.

And nowadays, it's like, well, the Teensy got replaced by the Arduino, and the Arduino got replaced by the Raspberry Pi. So, that's now a full computer, and we can be sloppy and lazy. But now we're going to get the teeny, tiny chips that are going to fit, you know, inside a ballpoint pen or inside, you know, a spec. And there's always room at the bottom.

WILL: It's true. It's true, yeah.

MIKE: Well, and this is a long arc. We're talking about learning the fundamentals, right? That's why we launched into this, you know, why learn SQL? Well, it turns out those fundamentals matter, whether it be querying the database, or understanding concurrency primitives, or even having some basic understanding that you might have to cram stuff into tiny, little spaces [chuckles], and that's hard. It still helps. It still helps to know these things.

I'm sure there's some knowledge that will just be arcane, you know, nobody's ever going to use it. But I know a lot of those things they carry over. Even if you don't use them directly, they change the way you think, in a way that, yes, does carry over quite well.

DAVE: Yep. You start your career being able to figure out anything from first principles, but bound by the problem of you have to figure everything out from first principles. One of the best ways to find a senior programmer is find out how many ways they've forgotten to do a task. It's like, if I give you this weird problem, and you immediately are fluent in seven different approaches, and you instantly know that these three aren't going to work because you're on the wrong reservation to do that, that is so, so useful.

MIKE: Dave.

DAVE: Mike --

MIKE: I'm going to ask you specifically, because I've talked about my career arc with querying, with SQL, and Will's talked about his. So, he's been kind of shoved out, right, and squeezed out of that space.

DAVE: In the enterprise space.

MIKE: Yeah, security reasons. No, you can't touch this. But I know that you kind of have gone in the opposite direction. You even worked with the data team for a couple of years.

DAVE: Yeah. Yeah. So, for me, like, I kind of stayed in full stack, which meant small company land. I was a small business consultant. That's just what I was. I was getting rid of people's pen and paper and replacing them with digital systems. And so, even 10 years ago, 2015-ish, before I went to work for, or 2013, '14-ish, before I went to work for CoverMyMeds, I was hopping small shops as a freelancer and really, really enjoying it.

So, in our back chat…I want to call out Thomas a little bit here. Thomas raised the question about, like, should I even bother learning SQL? Because, I mean, like, AI is going to already know it. And you're not wrong. You can straight up...right now, if you've got GitHub Copilot, you can open up a command line and say, "GitHub Copilot, suggest query to...give me a query to do the following things," and you can pull it off.

And my take on that is that the end product of these skills is always churning. Like, the DBA job you had 10 years ago doesn't exist anymore. The DBA job you would have today won't exist in 2035. I strongly feel that way. And it's not just AI, just the future is coming for your job continually. But the skills that go into it, those are evergreen. Those you will take everywhere. And, Mike, you and I have said this before, that, like, everyone should learn Lisp, or I've said it, and you've nodded vigorously.

MIKE: [laughs]

DAVE: But everyone should learn Lisp, not because they need to know how to program in Lisp, but because it will make them a better programmer in whatever language that they're using. And --

WILL: Clojure.

DAVE: Clojure, mm-hmm.

WILL: Clojure. Clojure. Clojure is the bomb. Oh, it's so good. And it's on the JVM, so you can actually work in it.

DAVE: Big time. I wholeheartedly agree. If you go look at the SICP lectures, Structure and Interpretation of Computer Programs, these are free. They were taught at MIT in 1984, and I didn't find them until 2010. And I was livid because I did not have a single coworker who knew anything that was in those lectures from 1984. And it's not because the information had expired; it's because we went off and we pounded rocks until we forgot that technology even existed. Like, we literally lost the golden age of computer science.

Streaming, or transfer functions, the idea that you can take a set of something, do some transformation on a single element, and then turn that into a new set of things. And whether some things get filtered out of one set, and you're going to run into...Google the word surjection. You will go into a whole, like, silo of mathematical jargon for, like, selecting from a set...and they won't say selecting from a set. They'll say transferring the domain of a function into its co-domain, and that kind of fun stuff.

But you get down into that level, you start seeing streaming as a way of taking something, you know, like a list, you take one item, one item, one item, and you don't need to know where the end of the list is. You're just continually taking the next one. If you can do streaming and map reduce, you now can handle any database. And you can also handle any data processing task, where you've got more data than your CPU can hold in its hands. I think those are the fundamental principles from it.

And, by the way, if you go Google surjection, and you're like, what does this even matter? That's a left outer join. Go look at it again, right?

MIKE: And that is, you know, I've written some notes [chuckles] to prep for hosting today, and it nailed one of the points that I wanted to get to.

WILL: Oh, good, good. Which one [laughs]?

MIKE: Well, so, you're talking about a certain approach to problems, which is...and we talked about Lisp, Clojure, you know, we're talking about a functional approach, functional languages. A lot of reasons, a lot of companies, one reason, not the only reason, write sensitive code in functional languages...like, I think that Twitter they [inaudible 26:25] with Scala to do all their heavy stuff. And you'll find similar patterns all over the place.

If you let the language take care of the implementation, well, the engineer focuses on, like, the boundaries and definitions, you think deeper about the results, instead of, like, oh, have I incremented my loop counter? And you get fewer bugs. Because you're standing on the shoulders of whoever wrote the compiler, you know, whoever wrote the language. You're doing less work. You're thinking about the actual problem. And SQL is an extreme take on that approach. You define the data sources and the transformations. You don't get to choose the implementation. You don't get a say in that. And it's called a declarative language, as opposed to an imperative language.

You say what you want instead of telling the computer what to do, and that's a fundamental shift. And I think that it's a big deal, not just in SQL, but when you've thought about that declarative approach, I think it changes your thinking. And I wanted to ask how it's changed you all's thinking. And, specifically, MapReduce, is that kind of approach. You might not know the implementation. Somebody's implementing it, right? Or you think about comprehensions in Python [inaudible 27:50].

Like, you're not manually iterating through stuff. You're letting the language do it. A lot of functional languages have these pipelines. You know, you don't know anything about the implementation of that. You're doing a series of transformations on data. And Ruby, you know, you might have a series of maps where you're transforming data, and you're piping into the next one. It's a way of thinking that behaves differently than taking control of yourself. And that declarative approach is, well, I think there's a reason that SQL is still around. I'm sure other people were doing it in different ways, however many years ago, right? But this one is stuck because it works.

WILL: Can I take this on a little bit of a wild tangent?

MIKE: Please.

WILL: Sorry, like, two, like, neurons, like, maybe crossed in my brain, not exclusively because I've been having a double IPA, because I'm not at work today. It's Boxing Day [laughter], and we don't work on punching day. But, I wonder, oh God, this is so random. David will appreciate this, if no one else.

But I wonder if, like, when we start, when you were talking about declarative language, immediately my mind went to, like, you know, sort of our AI prompt code generation, where we're declaring things. And, like, I had, like, a weird leap going to, like, no, you know what it was? It was Clojure. And I started to think about, like, the Clojure, like, statically, strongly typed languages: Clojure, Scala, you know, God help you, Haskell.

But I was thinking about, like, the marriage between these very strictly typed languages and then prompt generation, and whether one hand couldn't wash the other, and whether we might have a resurgent of these, like, high-level, difficult to reason about, very strict and structured languages combined with LLMs, which, let's be honest, they have a little bit of a telling stories problem, right [laughter]? You can't tell stories to Haskell. Haskell don't play that [chuckles].

But, like, it's a tangent. I'm just thinking about, like, these very, like, rigorous, strict mathematical but high-level languages that are strict, you know, and whether there might be a multiplicative effect of that between these declarative prompt generation things, you know what I mean? In terms of, like, structured reasoning, which LLMs are not great at.

MIKE: Right. Well, using the best features of both tools and combining them.

WILL: Exactly.

MIKE: That's a fascinating idea. Well, and even the idea, just the first leap you made of using LLMs being effectively declarative implementation, rather than imperative. That's an interesting thought. But that idea, I mean, I think it holds, right? The idea is, you're saying, this is what I want; give it to me. And then you're letting all of the hundreds of billions of dollars being poured into AI [chuckles] work on those implementations for you. Why shouldn't you?

WILL: Right. And, well, I mean, and you get a tight, you know, strictly defined feedback cycle to feed that LLM right versus wrong, and, you know, from a language level, right, where, like, there's a level of rigor that does not exist in, say, Ruby, you know, where…looks good to me. If that function didn't actually exist, we'll just crash [laughter], you know. Anyway, sorry, the tangent is taking this thing off the rails.

MIKE: Well, we've gone on a few of those already. But, you know, in the conversation before the...it was, even before the pre-call, there was some chat conversation where we were celebrating tangents. So, you know, we're in a good spot [laughs].

But that idea of declarative approach of being valuable is what I was trying to explore a little bit. And, you know, Dave, you said, well, learning Lisp it changes the way you think. There's value there. And I saw nods all around. And I learned Haskell, I don't know, 15, 12, 15 years ago, something like that. And I've never used it, like, you know, it's never professionally helped me at all, but it definitely changed the way I thought. And it's just thinking that way is different, and it's the same way, again, as SQL does it. That approach is going to give you a lot of benefits and save you some grief.

Now, you have to think about what your transformations are, but you have to anyway. And if you think that you're not thinking about it, you're pretending, you're just making a big mess that's [inaudible 33:07]

WILL: You're playing Russian roulette.

MIKE: Yeah [laughs], absolutely.

WILL: You just hope that spin the barrel, and it goes click.

MIKE: Sometimes it won't.

WILL: Sometimes it won't, and that'll be the last problem you ever had [laughs].

MIKE: So, write your transformations, and declare them, and let the compiler do the work, or your interpreter, you know, let the language do the work. And then you can sleep at night, you know [chuckles], worry about other things. You know, go and have a life after work.

WILL: No, no. The thing that keeps me fed is, like, no matter how much I give them, they just want more [laughter].

MIKE: They do; they do.

WILL: And then they start saying things like, "Well, you know, that site is great. Listen, the site is great. It's tip-top, looks good, pixel perfect. Everything runs. Uptime is fantastic. We love it. But, man, I just wish it was a little bit faster [laughter]." And that's when I [inaudible 34:08] out the SQL [laughs].

MIKE: Oh yes.

WILL: And everything comes back again, you know. Like, because if you want to make it cook, if you want to make it cook, like, you write your specifications at a low level, you know, where you can help the database help me help you, right? And we find ourselves right back where we started, where we're making deals with the devil with just a little bit more load time because time is money. Don't get it twisted [laughs].

MIKE: We've got Eddy here, who's, what do I say, early career? And Thomas who is --

DAVE: Early expert?

MIKE: Yeah, like, Eddy jumped right in the deep end. So [laughs], when I say early, like, he's not, like, you know, just getting started. He's one of the --

WILL: That's like dog years with Eddy [laughter].

MIKE: You know, he's an exceptionally capable engineer. But, you know, he doesn't have some of the gray hairs that some of us here may have [laughs].

WILL: Oh, you got your first one, buddy [laughter]? Oh.

DAVE: Coming in, nice.

WILL: Yeah. Yeah.

MIKE: And Thomas, he's working in application support, where they are liaisons to the engineering team and translate between that and the teams who are out on the call floor and have to figure stuff out. Now, they run queries all the time because they're investigating the data. So, he's heavy on the look at data side. And so, I'm really curious, from you all's perspective, you know, we've been talking about longer career arcs. But what are your thoughts about learning SQL, learning query languages, and how, you know, what impact that's had? Please do tell.

EDDY: Well, I don't know. I'm just speaking for myself, but I used to be under the impression that I could probably get away with not knowing SQL, because Rails, quote unquote, "Did all the magic for me," just under the hood, right? That's the benefit of ORMs, right? It's really good at, like, abstracting that and just giving you a very specific DSL syntax. So, I was doing that for a while, until I realized that I needed to run a few queries in production to verify that some of the code that I deployed worked. Quickly found out that I don't have Rails or console access for production. Who would've thought?

MIKE: [laughs]

EDDY: So, how do I verify that certain things are working the way they're intended to work if my hands are cuffed, right? So, there's only so far you can get, you know, with hunting, you know, until you fully have to accept the fact that SQL is just another layer of knowledge, you know, to put under your arsenal for being considered a full-stack developer, right? And you don't necessarily need to do SQL, but, like, just doing a quick Google search, right, I found, like, five-plus articles, you know, on Google that basically just say that SQL is a de facto winner, you know. It solidified itself as the top, you know, query language.

So, I think it is really essential. There are other alternatives that I know of that are trying to replace SQL, like NoSQL, you know, things like that that are alternatives. But it never really made an impact. And, for some reason, we keep going back to SQL, right? Like, there's a reason to why it's going strong, 50-plus years. I think it's relevant. I think it's so important, you know, in engineering, that we have a devoted department just for it. Like, if that doesn't tell you just how important it is for you to understand, you know, I don't know if anything else will. So, TL;DR, I think SQL is really important.

MIKE: And, Thomas, I'm curious about your thoughts as well.

THOMAS: Yeah, I agree, actually, especially for this role, you know. SQL, learning SQL has helped me significantly, just being able to have that access to get into, kind of, like, the data, you know, analyze certain columns and everything at certain tables, and being able to apply where issues are arising from, even just kind of looking at what information is being updated in the database, right? Seeing if that information that we're looking at is old information and how we can force an update has been extremely influential.

I kind of quoted in the chat, too, I feel like SQL, for me, has been kind of a learn one to know all, whereas I can apply the language to the other languages, you know. A lot of the other query languages are not exactly the same. But it gave me a really good understanding of, kind of, like, what you can do with the language, how you can kind of build tables, how you can also pull from those tables and everything.

So, SQL has been huge, learning SQL. And, I mean, for my position, we use it a ton, you know, just kind of running in Snowflake and all that. It's the bread and butter, I feel like, for my position. So, I've really taken to SQL. I actually really like it. And I was excited for this podcast, because that's been kind of the big thing for me right now, is getting into SQL and all of that.

MIKE: Nice.

THOMAS: And I'm close to finishing up on one of my courses. And it's awesome.

DAVE: That's awesome.

THOMAS: It's night and day from, you know, any other language I've learned so far.

DAVE: That is fantastic.

EDDY: Well, and, like, you think you understand SQL, right, and, like, data structures, and migrations, and, you know, things like that, until you have an engineer whose whole, like, title is to be the architect of how your data is structured, and, suddenly, you're thinking 180. Holy crap, I haven't even begun to scratch the surface, you know, on what SQL begins to offer, right?

Suddenly, you have to think, like, oh, man, now I have the main primary keys. Hmm, crap. Now I need a foreign key, you know, on these joins. And, oh my gosh, like, there's so much abstraction to SQL than just, you know, table, meet this table [laughs]. There's so much more complexity behind data structures that way; it's crazy.

THOMAS: Yeah. Yeah. I want to add to that, too. It was funny, actually, because I was building a query to kind of look for something in the database. And I was thinking, I'm like, okay, I was comparing two columns. And I was like, I think I can get what I need to with these two columns. And then I was kind of researching, looking into the columns and everything. And then I was like, you know what, maybe this isn't enough.

So, actually, had Sam, he was helping me, and he developed this huge query. And I was like, oh yeah [laughs], I'm still a little bit behind. There's a lot I need to learn, you know. There's a lot to catch up on. But I was like, it was so fun being able to look into that stuff because, to me, it's like building a Lego, you know. The more you add onto it, the bigger you build it, the more a picture comes out of it. And the puzzle that is involved in it, I think, is super satisfying. But I totally agree with that, the scratch on the surface, because, man, when I saw, you know, these queries that are put together, I'm like, I'm barely dipping my toes in the water here [laughs].

DAVE: I had that when I went over to the data team. I thought…I've done some 20 table joins. And, you know, I've done five or six table joins with sub-queries, and sometimes two of them. And, man, I thought I was hot snot. And I got over to the data team, and they're like, 30 tables is a Monday for us. And how many common table expressions do you have in there, and how many left joins? How many window functions are you using? And I'm just like, oh, dear, I get to go back to the books and learn how to do SQL all over again.

WILL: I mean, that is really, like, the final frontier, right? I mean, in terms of, like, you know, selects and joins and, like, you know, queries and stuff like that, I mean, like [inaudible 42:11] got that. They got it. It's fine; it's no problem. But, like, you know, where things start to get really interesting from a performance perspective is the database is so much smarter than we give it credit for. And, like, do you need your own sort of synthesized table, you know, joining all these things at the database layer? Because the database has it all right at its fingertips. It doesn't need you. It doesn't need you, right?

Like, if you just tell it, right, where it's like, hey, can you just make me this view with this and this and this? Like, the database is just, like, yeah, that's really easy. But if you go back and forth 20 different times to do it yourself like a dumb-dumb, you know, like, your caveman table is both going to be, like, crappy, and limited, and slow, whereas the database is just like, it's all right here, man. Like, I have everything. Just tell me what, like, if you can figure out how to make the words come out of your mouth to tell me what you really need, like, it's all here, you know what I mean?

From a programming perspective, like, I mean, anybody can learn the foundational, you know, nuts and bolts pieces of an ORM, right? Like, some of that's why we made ORMs because it's a waste of time, like, anybody could do this. I don't need to think about a select, really, not really, you know. Simple select, it's pretty easy. It's pretty light work, you know. This is the kind of thing you could code up in a weekend if you thought at all about, like, you now, about this stuff. But a database is so much cooler.

Like, it's so funny. I mean, we were talking in the pre-call about, like, sort of things you've changed your mind on over the course of your career. And I used to think databases were the stupidest, most boring part of any application. Like, I thought, like, my buddy in grad school was going in, and she was specializing in sort of, like, databases and stuff like that. I'm like, oh my God, can you imagine spending your time [laughter], [vocalization]. And how wrong I was. Mariah, I owe you several apologies, for that one, at least [laughter]. I just didn't get it.

MIKE: Well, Dave was mentioning common table expressions. I learned those later in my career, unfortunately. I mean, I guess they haven't always been with us, but they are a mechanism for providing structure to your queries so you can make them modular. Like, you know, a class is the unit of structure in an object-oriented language. Well, you got these common table expressions in your SQL. That's a big deal to be able to structure your queries in ways that perform cohesive units that you can think about separately.

And, for me, I was like, oh, wow, this is a game changer. I don't have to think about sub-queries. I can structure my data in these modules that can move kind of linearly forward. Oh, wait, this is like those transformations of data that we've been talking about all along. And I can structure it that way in chunks that my mind can wrap around. That's fantastic.

I was using some of those [inaudible 45:33] [laughs] I mentioned earlier, where I had to do some gnarly averages of time a week or two ago. They're fantastic and probably my favorite feature of the language now. I'll probably learn another one sometime. Window functions are cool, too. A lot of people had the same experience with common table expressions when they have to write big, gnarly queries.

DAVE: Hands down, hands down. The CTE is where you get the full forward extrapolation of take something, transform it into something else. If that is a CTE, it's just an expression. Well, now you can transform into that then you can take that and transform it into another thing. And they can be things that cannot be computed at the same time. The first one can be, look at all these rows and find all the pairs of these data, and give me the second one in the pair where their relationship to each other in their window, I'm talking about a window function, right, in their window is important. And you can't see into that from the other transformation.

So, you do the first transformation, and you end up with, you know, a CTE of renters who paid after the deadline, right? You know, payments made late, you know, that kind of thing. And then from there, you go into your next CTE, or your next expression. And yeah, you get, on the data team, it was not uncommon to have 15 CTEs scrolling off the top of the page just to get everything ready for one big query that is, show me all the customers that need to be called today. And all the CTEs are all the business rules that went into who gets called and why and when.

MIKE: I think that modularity is, well, it's not unique to SQL, but modularity is how you make your code comprehensible. If you don't break things into pieces and you just throw it all in one big pile, well, there's all kinds of derogatory names you have for that sort of code. If you [chuckles] modularize it, then it becomes comprehensible, because we're writing code for humans, and then the computer will figure it out. And as has been mentioned, it's really good at figuring out if you say what you want, you know, that's all built. So, say what you want clearly, and it'll figure it out.

DAVE: We've been talking a little bit about, like, do I need to learn this, and do I need to improve? And I have a brief story to share on this, but it might be solid for closings. Is now a good time to dive in? I want to talk about SQLite aggregations.

WILL: Yeah, we've been on for a while. Play us off, Dave [laughs].

DAVE: So, SQLite used to have this bug in it that it would do aggregations incorrectly. And aggregation is when you are saying select count from the table or select the average. When you are selecting all of the items, individual items, you cannot also select the aggregate. Well, you can, but it gets kind of hinky and weird.

And I was on a call in a job interview, like, just like a phone screening. And they wanted to see how good I was with HTML, how good I was with Ruby and JavaScript, and how good I was with SQL. And that was the third one. And we were running out of time. They were moving buildings, and they were literally throwing my interviewers out of their building. And so, I'm like, type faster, type faster, because we're over time. You got to do this.

Now, Thomas, to what you said earlier about, like, should I be learning this if I'm in application support, or if I'm in QA, or if I'm in engineering? And the answer is yes, yes, yes, yes, yes, yes, yes, not because you need to know this thing, but because if you are the kind of person who goes and learns new things, 10 years from now, you're going to be in whatever career you want to be in, because you are continually creating value in yourself. Eventually, you're going to be the Sam on your team. You're going to be the one that people, you know, go to Thomas. He knows how to get this out of the database. And then you move up and you move up and you move up.

And I have trait curiosity, not state curiosity. I have a ticket that says, "There's no cure for curiosity," and it's kind of a life motto for me. So, I'm always reading and learning and typing and exploring it and actually testing. And I found this bug in SQLite myself. I'm in the middle of the interview, and the last question that they give me is, we want you...here's a table of inventory items. Here's a table of sales, and the sale price can vary over time, sorry, items in departments. That's what it is, items in departments. And we want to know what the most expensive item in each department is. So, there's a sub-query, right? You've got, for each department, what is the most expensive item, and how do we pull that out?

And you're supposed to, you know, do your window function or some other way, or a sub-query, or pull it into...you know, a lot of programmers, if you're just beginning, you just get all the prices, and then you just walk through them in code. People will do that. That's not a great way to get the answer, but it's better than not getting the answer. And I just wrote select item department price from the table and max price on it, you know, give me the item with the max price on it, go. And when you combine an aggregation, you only get back one row. Like, you basically say, like, so you'll get back one item and one department and one price.

And if you're in a normal database, that is illogical. Like, Postgres will refuse to do the query. It's not logical. I'm not going to tell you the price of this Game Boy is $2,500 because there's a $2,500 couch in this department. That's the largest price. But the last item that we selected was a Game Boy, and you can't control how the database will order and sort. So, I just wrote the query, select item, department, max price, and threw in an order by and ran it and kicked it off. And they're like, but that...and then they look at the output, and the output was correct.

And the reason it was correct is because SQLite does...I don't know if the source code is like this, but it's equivalent in Ruby to taking a string of hashes and just merging them continuously into the latest one. So, basically, oh, this one's got a larger price. We're just going to update this. And every row that goes by an aggregator updates it, updates it, updates it.

The last row you get to, it's going to say, yes, this is my name; this is my department. We're going to update that. If it happens to be the most expensive item in the department, it will be set correctly. And SQLite won't refuse to give you the right answer for the wrong reason [chuckles], which means you get the right answer for the right reason or the right way for the wrong reason.

And that got me the job at CoverMyMeds because they were like [chuckles], "You can't do that." And I just hit Enter. And they're like, "You can do that? I'm like, yeah, it's a bug in SQLite." And I didn't get hired because I knew SQL, and I didn't get hired because I knew there was a bug in SQLite. I got hired because they said, this is a guy that goes home at the end of work and goes hunting for bugs in the database engine. Let's hire this guy. We want this guy working on our software.

So, is AI coming for your job? AI's coming for everybody's job. You don't got to outrun the AI. You just got to outrun the next programmer next to you. Stay curious.

THOMAS: Yeah, thank you for that.

DAVE: That's my advice. Stay curious.

WILL: AI is not coming for anybody. Like, it's the same thing it's ever been.

DAVE: Tomorrow is coming for everyone's job.

WILL: Every eight years, every eight years, the computing industry turns over. And the next wave is coming, and you're on it, or you're under it. And all you young, fresh-faced fellows, like, you think, like, it'll never happen to me. It's happened to me, like, three times. I think --

DAVE: When we were graduating college, we never would have believed you if you told me that the job of technology evangelist would exist. And now we go to people, like Thomas's age, coming straight out of college, and we tell them about that job, and they go, that job never existed. There's no way, right? We are literally on the opposite edge of the event horizon from that moment in time, just inside the computer industry.

WILL: If you asked me when I graduated college, "What was your favorite class through engineering school, where you learned the most?" Like, I would have had a weirdo, like, freakazoid answer. And I would have been, like, "Technical writing, actually."

DAVE: Mm-hmm, [inaudible 53:57]

WILL: And I had a very good technical writing instructor, and I learned a lot about technical writing. And, like, I'm real, real good at it now. And, like, I mean, there's actually never been a day in my career where I wasn't really, really good at technical writing. Like, I'm really good at it because I learned how to do it right. And it's a foundational skill for any engineer ever, at any level, ever.

But, like, I'm not scared of prompts. I've been writing prompts since 1995. Like, now a machine can read it, but, like, it's the same tool set. And, you know, that's it, like, where it's just, like, oh, man, what makes a really good engineer? Could I say English language skills, you know what I mean? And it's just, like, it's not even news [laughs]. It's not even news.

DAVE: Soft skills. Soft skills will make your career.

WILL: [inaudible 54:48] for somebody else. I didn't say soft skills. I'm not writing poetry [laughs].

DAVE: Well, I'm not either. That's not a soft skill.

WILL: [laughs] This ain't [inaudible 54:55]

DAVE: Soft skill is the ability to laugh at a podcast, right, is the ability to invite someone to participate when the conversation starts to get tense, right? That's all. Soft skills is just how do you talk to another human being?

WILL: Yeah, yeah.

MIKE: So, communication. Our field is more about communication than it is about writing code, although writing code is a form of communication.

WILL: It absolutely is. It absolutely is.

MIKE: That was a great ending because we came in and said, these skills will help you someday, not just SQL, although it is included in the set. Learn it. It's worth your time. Until next time on the Acima Development Podcast.

...more
View all episodesView all episodes
Download on the App Store

Acima DevelopmentBy Mike Challis