Raw Data with Rob Collie

There are two ways to build backwards. One starts with the answer you actually want and works its way toward the plumbing. The other starts with the plumbing and hopes the answer is in there somewhere. Consulting has historically been awfully fond of option two.

AI makes that harder to defend. When you can get something working in front of the business early, there's no particularly good reason to disappear into the infrastructure for six months first. Give people something they can see. Something they can use. Something they can tell you is wrong. Then fix it while fixing it is still easy. Better yet, let what you learn change what you build next.

Rob and Justin start with AI and eventually wander into the hazards of building in the dark, the value of building in plain sight, and a particularly useful new term: "Byzantine Consulting." It's a conversation about working backward from the answer instead of forward from the infrastructure. Because if nobody can react to what you've built yet, there's a decent chance you haven't built the important part.

Also in this Episode:

Rob's Substack

Rob has a lot of Claude folders now.

One knows the book. Another knows the company planning. Others have picked up meeting notes, briefing docs, LinkedIn ideas, strategy work, and whatever else was useful enough to save at the time. None of this felt like a problem while it was happening. Quite the opposite. The whole setup was working well enough that Rob kept giving it more to know.

Then one of those markdown files started telling him what he was and wasn't supposed to do.

Rob and Justin follow that thread into a much bigger problem than one weird sentence in one weird folder. Once AI starts carrying knowledge from one place to another, remembering things, connecting things, and helping everybody work faster, a few very old problems start wandering back into the room wearing new clothes. The BI crowd may recognize them. Rob certainly did. And one of his solutions will be immediately familiar to anyone who has ever opened a Power BI model, looked at the relationships it helpfully created for them, and started deleting.

For a minute there, it looked like the AI wars would come down to who built the smartest model. Rob's not buying that anymore.

Take Claude. Rob's increasingly convinced that what makes Anthropic so sticky isn't Claude itself. It's Claude Code. It's Cowork. It's the software wrapped around the model that makes the whole thing so ridiculously useful. Great news for Anthropic, except for one tiny problem: software can be copied. And when the models themselves are interchangeable enough to live in a dropdown menu, you have to start wondering what any of these companies really have that somebody else can't recreate.

That question leads straight to Microsoft, which may be holding a much better hand than it gets credit for. Everyone else is trying to worm their way into your email, your files, your chats, and the rest of your working life. Microsoft is already sitting inside the castle. From there, Rob and Justin talk through the increasingly strange economics of all this, whether actual humans using your product become the moat that matters, and finally, the proposed fix for our data center problem that involves launching the data centers into space. Give it a listen for Rob's take on that one, starting with the minor inconvenience of physics.

For years, businesses learned that having data and being able to use it are two very different things. We cleaned it, structured it, modeled it, argued about which version was the truth, and built entire careers around making messy information useful. Now AI is creating the same problem all over again. Except this time, it's knowledge.

In this episode, Rob explains why everything we learned from the data era is suddenly relevant again. Companies are sitting on enormous amounts of knowledge scattered across documents, conversations, systems, processes, and people's heads. AI can consume more of it than ever before, but that doesn't magically make it organized, trustworthy, or useful. Which means a lot of those supposedly "old" data skills are about to look awfully cutting edge again.

Rob also introduces The Goldilocks Altitude, his new Substack for ideas that live somewhere between the 30,000 foot AI think piece and the technical weeds. This episode features one of the first: "Everything We Did for Data, We Now Need to Do for Knowledge." Listen now, then head over to The Goldilocks Altitude. If knowledge is having its data moment, there's going to be plenty to talk about.

Also in this episode:

The Goldilocks Altitude

There's a new word for what your team is doing with the AI you bought them. You're not going to love it.

"Botsitting." Babysitting, but for bots. Journalists are suddenly all over it, and not one of the AI people Rob asked had ever heard of it. New name, familiar burn. Turns out mandating AI usage is a great way to pay good money for software that makes everybody slower.

Then the flip side. Rob's spent years telling anyone who'd sit still that conversational data changes how we work. A recent experiment made him realize he'd been underselling his own argument.

So one half of AI is wearing people out, and the other half is hooking people who never cared about data in their lives. We're still early, folks. And apparently, we're going to need some new vocabulary.

Listen to the latest episode of Raw Data. Turns out AI gets a lot more interesting once you see what people actually do with it.

For most of tech history, the new thing belonged to the new people.

Then AI showed up and ruined a perfectly good pattern.

Because the smartest model on the planet can know darn near everything and still have no idea how your company works. It doesn't know which rules matter, which ones everyone ignores, why that ugly spreadsheet still exists, or that the "temporary" workaround from 2019 is now apparently infrastructure.

Jon Perl, Tim Rodman, and Trent McKinster join Rob for a conversation about Fair Game that pretty quickly becomes a conversation about who has the upper hand here. And it might just be the crafters. The people who spent years poking at problems, pulling things apart, building better ways to do the work, and collecting the kind of business context you can't download with a model.

From there, things get wonderfully nerdy. Custom AI. Semantic models getting their long overdue victory lap. Whether SaaS is about to get picked apart one annoying subscription at a time. And the possibility that we've been thinking about the AI skills gap completely backwards.

Maybe experience isn't the thing AI replaces.

Maybe it's the thing AI has been waiting for.

There have always been people who can't help themselves. Give them a clunky process, a messy spreadsheet, or a problem nobody owns, and they're already halfway to building something better. Rob calls them crafters. They were valuable before AI. They're becoming indispensable because AI rewards exactly the way they've always worked.

While everyone else is trying to "become AI native," crafters are doing something much less glamorous. They're making one workflow better. Then another. Pedro Rossello from Aderant is one of those people. Along with Rob and P3's own David "Oz" Osorio, he makes the case that the biggest AI wins don't come from chasing the newest thing. They come from solving real business problems with better tools and better judgment.

The funny thing is, those wins don't always look like AI success stories. They look like fewer headaches, smarter decisions, and work that simply gets done better than it did last week. That's the crafter's edge. It isn't flashy. It isn't loud. But it's probably what separates the companies still talking about AI from the ones already putting it to work.

AI can write code faster than most of us can describe the problem. That's the good news.

The catch is that building software has never just been about writing code. It's about making decisions today that won't come back to haunt you six months from now. In this episode, Rob and Justin unpack what happened after Justin and Kellan built a real application with AI. The code came together quickly. The architecture, the maintenance, and the "why on earth did we do it this way?" moments took a little longer.

Along the way they discovered something that applies far beyond software development. AI can help you build almost anything, but it still depends on people to define the patterns, document the rules, and recognize when today's shortcut quietly becomes tomorrow's technical debt. Sometimes the smallest notes left behind become the biggest lessons later.

If you're building with AI, or simply trying to understand where human judgment still matters, this episode is worth the time. It turns out the future has a funny way of leaving breadcrumbs.

AI headlines have already moved on to the deep end. Most businesses haven't.

Every week brings another headline about what's next for AI. Build your own model. Train your own LLM. Customize everything. It's exciting, unless you're one of the thousands of companies still trying to answer a much simpler question: where does AI actually fit into the work we do every day?

Here's the thing. AI has become everyone's priority and almost nobody's job. Leadership knows it matters but the real work still lives inside thousands of everyday workflows, where tribal knowledge, context, and experience drive the decisions. That's the gap, and it's a very different problem than the one the headlines are chasing.

That's the conversation Rob and Justin have this week. Sparked by Satya Nadella's comment that every company should eventually have its own LLM, they make the case that the industry is getting ahead of itself. As Rob puts it, "Everyone's sitting poolside and Satya's talking about the deep end." Most organizations don't need a custom model. They need AI that understands their business, their data, and their workflows. That's where the biggest wins are happening today, and it's exactly where companies should be focused before they start worrying about building their own LLM.

If AI has started to feel like an arms race you somehow missed, this episode is a welcome reminder that the biggest opportunities are still waiting in the shallow end.

For about two years, we've all been reaching for the biggest hammer on the wall because someone else was paying for the nails. If you were on a subscription, you grabbed the biggest, baddest model on the menu and used the crap out of it. Two hundred dollars a month for work that would have cost thousands on the meter. It rounded to free.

Then a new model showed up for roughly fifteen minutes.

It wasn't covered by anyone's subscription. It was priced by the token. And Rob immediately saw something much bigger than a product launch. The migration everyone assumed would be painful, moving millions of people away from all you can eat subscriptions, suddenly had a simple answer. Just make the newest, smartest model a premium experience. Checkmate. The buffet doesn't disappear. You just have to decide whether the lobster is worth paying for.

Justin made the exact mistake he told himself he wouldn't make. He tried it anyway. He handed the model a sprawling request to audit an entire codebase and walked away. It came back with nearly twenty legitimate findings, from accessibility improvements to a legal disclosure that referred to the company as a corporation instead of an LLC. More importantly, it handled a level of independent work he wouldn't have trusted another model to do. His reaction afterward said everything: "I wish I hadn't tried it." Because once you've seen what the next generation can do, you can't unsee it. But if using it costs six or seven thousand dollars a month for one developer, "always use the best model" stops being a habit and starts becoming a business decision.

Whether you're building with AI every day or just trying to make sense of where it's all headed, this conversation is a good reminder that the technology isn't the only thing changing. The business model is too. Give it a listen and see where Rob and Justin think it all leads.