Sign up to save your podcasts
Or
Share In-Ear Insights: How Generative AI Reasoning Models Work
Share to email
Share to Facebook
Share to X
Share to email
Share to Facebook
Share to X
Or
In-Ear Insights: How Generative AI Reasoning Models Work
In this episode of In-Ear Insights, the Trust Insights podcast, Katie and Chris discuss the Apple AI paper and critical lessons for effective prompting, plus a deep dive into reasoning models.
You’ll learn what reasoning models are and why they sometimes struggle with complex tasks, especially when dealing with contradictory information. You’ll discover crucial insights about AI’s “stateless” nature, which means every prompt starts fresh and can lead to models getting confused. You’ll gain practical strategies for effective prompting, like starting new chats for different tasks and removing irrelevant information to improve AI output. You’ll understand why treating AI like a focused, smart intern will help you get the best results from your generative AI tools. Tune in to learn how to master your AI interactions!
Watch the video here:
Can’t see anything? Watch it on YouTube here.
Listen to the audio here:
https://traffic.libsyn.com/inearinsights/tipodcast-how-generative-ai-reasoning-models-work.mp3
Download the MP3 audio here.
[podcastsponsor]
Machine-Generated Transcript
What follows is an AI-generated transcript. The transcript may contain errors and is not a substitute for listening to the episode.
Christopher S. Penn – 00:00
In this week’s In Ear Insights, there is so much in the AI world to talk about. One of the things that came out recently that I think is worth discussing, because we can talk about the basics of good prompting as part of it, Katie, is a paper from Apple. Apple’s AI efforts themselves have stalled a bit, showing that reasoning models, when given very complex puzzles—logic-based puzzles or spatial-based puzzles, like moving blocks from stack to stack and getting them in the correct order—hit a wall after a while and then just collapse and can’t do anything. So, the interpretation of the paper is that there are limits to what reasoning models can do and that they can kind of confuse themselves. On LinkedIn and social media and stuff,
Christopher S. Penn – 00:52
Of course, people have taken this to the illogical extreme, saying artificial intelligence is stupid, nobody should use it, or artificial general intelligence will never happen. None of that is within the paper. Apple was looking at a very specific, narrow band of reasoning, called deductive reasoning. So what I thought we’d talk about today is the paper itself to a degree—not a ton about it—and then what lessons we can learn from it that will make our own AI practices better. So to start off, when we talk about reasoning, Katie, particularly you as our human expert, what does reasoning mean to the human?
Katie Robbert – 01:35
When I think, if you say, “Can you give me a reasonable answer?” or “What is your reason?” Thinking about the different ways that the word is casually thrown around for humans. The way that I think about it is, if you’re looking for a reasonable answer to something, then that means that you are putting the expectation on me that I have done some kind of due diligence and I have gathered some kind of data to then say, “This is the response that I’m going to give you, and here are the justifications as to why.” So I have some sort of a data-backed thinking in terms of why I’ve given you that information. When I think about a reasoning model,
Katie Robbert – 02:24
Now, I am not the AI expert on the team, so this is just my, I’ll call it, amateurish understanding of these things. So, a reasoning model, I would imagine, is similar in that you give it a task and it’s, “Okay, I’m going to go ahead and see what I have in my bank of information for this task that you’re asking me about, and then I’m going to do my best to complete the task.” When I hear that there are limitations to reasoning models, I guess my first question for you, Chris, is if these are logic problems—complete this puzzle or unfurl this ball of yarn, kind of a thing, a complex thing that takes some focus.
Katie Robbert – 03:13
It’s not that AI can’t do this; computers can do those things. So, I guess what I’m trying to ask is, why can’t these reasoning models do it if computers in general can do those things?
Christopher S. Penn – 03:32
So you hit on a really important point. The tasks that are in this reasoning evaluation are deterministic tasks. There’s a right and wrong answer, and what they’re supposed to test is a model’s ability to think through. Can it get to that? So a reasoning model—I think this is a really great opportunity to discuss this. And for those who are listening, this will be available on our YouTube channel. A reasoning model is different from a regular model in that it thinks things through in sort of a first draft. So I’m showing DeepSeq. There’s a button here called DeepThink, which switches models from V3, which is a non-reasoning model, to a reasoning model. So watch what happens. I’m going to type in a very simple question: “Which came first, the chicken or the egg?”
Katie Robbert – 04:22
And I like how you think that’s a simple question, but that’s been sort of the perplexing question for as long as humans have existed.
Christopher S. Penn – 04:32
And what you see here is this little thinking box. This thinking box is the model attempting to solve the question first in a rough draft. And then, if I had closed up, it would say, “Here is the answer.” So, a reasoning model is essentially—we call it, I call it, a hidden first-draft model—where it tries to do a first draft, evaluates its own first draft, and then produces an answer. That’s really all it is. I mean, yes, there’s some mathematics going on behind the scenes that are probably not of use to folks listening to or watching the podcast. But at its core, this is what a reasoning model does.
Christopher S. Penn – 05:11
Now, if I were to take the exact same prompt, start a new chat here, and instead of turning off the deep think, what you will see is that thinking box will no longer appear. It will just try to solve it as is. In OpenAI’s ecosystem—the ChatGPT ecosystem—when you pull down that drop-down of the 82 different models that you have a choice from, there are ones that are called non-reasoning models: GPT4O, GPT4.1. And then there are the reasoning models: 0304 mini, 04 mini high, etc. OpenAI has done a great job of making it as difficult as possible to understand which model you should use. But that’s reasoning versus non-reasoning. Google, very interestingly, has moved all of their models to reasoning.
Christopher S. Penn – 05:58
So, no matter what version of Gemini you’re using, it is a reasoning model because Google’s opinion is that it creates a better response. So, Apple was specifically testing reasoning models because in most tests—if I go to one of my favorite websites, ArtificialAnalysis.ai, which sort of does a nice roundup of smart models—you’ll notice that reasoning models are here. And if you want to check this out and you’re listening, ArtificialAnalysis.ai is a great benchmark set that wraps up all the other benchmarks together. You can see that the leaderboards for all the major thinking tests are all reasoning models, because that ability for a model to talk things out by itself—really having a conversation with self—leads to much better results. This applies even for something as simple as a blog post, like, “Hey, let’s write a blog post about B2B marketing.”
Christopher S. Penn – 06:49
Using a reasoning model will let the model basically do its own first draft, critique itself, and then produce a better result. So that’s what a reasoning model is, and why they’re so important.
Katie Robbert – 07:02
But that didn’t really answer my question, though. I mean, I guess maybe it did. And I think this is where someone like me, who isn’t as technically inclined or isn’t in the weeds with this, is struggling to understand. So I understand what you’re saying in terms of what a reasoning model is. A reasoning model, for all intents and purposes, is basically a model that’s going to talk through its responses. I’ve seen this happen in Google Gemini. When I use it, it’s, “Okay, let me see. You’re asking me to do this. Let me see what I have in the memory banks. Do I have enough information? Let me go ahead and give it a shot to answer the question.” That’s basically the synopsis of what you’re going to get in a reasoning model.
Katie Robbert – 07:48
But if computers—forget AI for a second—if calculations in general can solve those logic problems that are yes or no, very black and white, deterministic, as you’re saying, why wouldn’t a reasoning model be able to solve a puzzle that only has one answer?
Christopher S. Penn – 08:09
For the same reason they can’t do math, because the type of puzzle they’re doing is a spatial reasoning puzzle which requires—it does have a right answer—but generative AI can’t actually think. It is a probabilistic model that predicts based on patterns it’s seen. It’s a pattern-matching model. It’s the world’s most complex next-word prediction machine. And just like mathematics, predicting, working out a spatial reasoning puzzle is not a word problem. You can’t talk it out. You have to be able to visualize in your head, map it—moving things from stack to stack—and then coming up with the right answers. Humans can do this because we have many different kinds of reasoning: spatial reasoning, musical reasoning, speech reasoning, writing reasoning, deductive and inductive and abductive reasoning.
Christopher S. Penn – 09:03
And this particular test was testing two of those kinds of reasoning, one of which models can’t do because it’s saying, “Okay, I want a blender to fry my steak.” No matter how hard you try, that blender is never going to pan-fry a steak like a cast iron pan will. The model simply can’t do it. In the same way, it can’t do math. It tries to predict patterns based on what’s been trained on. But if you’ve come up with a novel test that the model has never seen before and is not in its training data, it cannot—it literally cannot—repeat that task because it is outside the domain of language, which is what it’s predicting on.
Christopher S. Penn – 09:42
So it’s a deterministic task, but it’s a deterministic task outside of what the model can actually do and has never seen before.
Katie Robbert – 09:50
So then, if I am following correctly—which, I’ll be honest, this is a hard one for me to follow the thread of thinking on—if Apple published a paper that large language models can’t do this theoretically, I mean, perhaps my assumption is incorrect. I would think that the minds at Apple would be smarter than collectively, Chris, you and I, and would know this information—that was the wrong task to match with a reasoning model. Therefore, let’s not publish a paper about it. That’s like saying, “I’m going to publish a headline saying that Katie can’t run a five-minute mile; therefore, she’s going to die tomorrow, she’s out of shape.” No, I can’t run a five-minute mile. That’s a fact. I’m not a runner. I’m not physically built for it.
Katie Robbert – 10:45
But now you’re publishing some kind of information about it that’s completely fake and getting people in the running industry all kinds of hyped up about it. It’s irresponsible reporting. So, I guess that’s sort of my other question. If the big minds at Apple, who understand AI better than I ever hope to, know that this is the wrong task paired with the wrong model, why are they getting us all worked up about this thing by publishing a paper on it that sounds like it’s totally incorrect?
Christopher S. Penn – 11:21
There are some very cynical hot takes on this, mainly that Apple’s own AI implementation was botched so badly that they look like a bunch of losers. We’ll leave that speculation to the speculators on LinkedIn. Fundamentally, if you read the paper—particularly the abstract—one of the things they were trying to test is, “Is it true?” They did not have proof that models couldn’t do this. Even though, yes, if you know language models, you would know this task is not well suited to it in the same way that they’re really not suited to geography. Ask them what the five nearest cities to Boston are, show them a map. They cannot figure that out in the same way that you and I use actual spatial reasoning.
Christopher S. Penn – 12:03
They’re going to use other forms of essentially tokenization and prediction to try and get there. But it’s not the same and it won’t give the same answers that you or I will. It’s one of those areas where, yeah, these models are very sophisticated and have a ton of capabilities that you and I don’t have. But this particular test was on something that they can’t do. That’s asking them to do complex math. They cannot do it because it’s not within the capabilities.
Katie Robbert – 12:31
But I guess that’s what I don’t understand. If Apple’s reputation aside, if the data scientists at that company knew—they already knew going in—it seems like a big fat waste of time because you already know the answer. You can position it, however, it’s scientific, it’s a hypothesis. We wanted to prove it wasn’t true. Okay, we know it’s not true. Why publish a paper on it and get people all riled up? If it is a PR play to try to save face, to be, “Well, it’s not our implementation that’s bad, it’s AI in general that’s poorly constructed.” Because I would imagine—again, this is a very naive perspective on it.
Katie Robbert – 13:15
I don’t know if Apple was trying to create their own or if they were building on top of an existing model and their implementation and integration didn’t work. Therefore, now they’re trying to crap all over all of the other model makers. It seems like a big fat waste of time. When I—if I was the one who was looking at the budget—I’m, “Why do we publish that paper?” We already knew the answer. That was a waste of time and resources. What are we doing? I’m genuinely, again, maybe naive. I’m genuinely confused by this whole thing as to why it exists in the first place.
Christopher S. Penn – 13:53
And we don’t have answers. No one from Apple has given us any. However, what I think is useful here for those of us who are working with AI every day is some of the lessons that we can learn from the paper. Number one: the paper, by the way, did not explain particularly well why it thinks models collapsed. It actually did, I think, a very poor job of that. If you’ve worked with generative AI models—particularly local models, which are models that you run on your computer—you might have a better idea of what happened, that these models just collapsed on these reasoning tasks. And it all comes down to one fundamental thing, which is: every time you have an interaction with an AI model, these models are called stateless. They remember nothing. They remember absolutely nothing.
Christopher S. Penn – 14:44
So every time you prompt a model, it’s starting over from scratch. I’ll give you an example. We’ll start here. We’ll say, “What’s the best way to cook a steak?” Very simple question. And it’s going to spit out a bunch of text behind the scenes. And I’m showing my screen here for those who are listening. You can see the actual prompt appearing in the text, and then it is generating lots of answers. I’m going to stop that there just for a moment. And now I’m going to ask the same question: “Which came first, the chicken or the egg?”
Christopher S. Penn – 15:34
The history of the steak question is also part of the prompt. So, I’ve changed conversation. You and I, in a chat or a text—group text, whatever—we would just look at the most recent interactions. AI doesn’t do that. It takes into account everything that is in the conversation. So, the reason why these models collapsed on these tasks is because they were trying to solve it. And when they’re thinking aloud, remember that first draft we showed? All of the first draft language becomes part of the next prompt. So if I said to you, Katie, “Let me give you some directions on how to get to my house.” First, you’re gonna take a right, then you take a left, and then you’re gonna go straight for two miles, and take a right, and then.
Christopher S. Penn – 16:12
Oh, wait, no—actually, no, there’s a gas station. Left. No, take a left there. No, take a right there, and then go another two miles. If I give you those instructions, which are full of all these back twists and turns and contradictions, you’re, “Dude, I’m not coming over.”
Katie Robbert – 16:26
Yeah, I’m not leaving my house for that.
Christopher S. Penn – 16:29
Exactly.
Katie Robbert – 16:29
Absolutely not.
Christopher S. Penn – 16:31
Absolutely. And that’s what happens when these reasoning models try to reason things out. They fill up their chat with so many contradicting answers as they try to solve the problem that on the next turn, guess what? They have to reprocess everything they’ve talked about. And so they just get lost. Because they’re reading the whole conversation every time as though it was a new conversation. They’re, “I don’t know what’s going on.” You said, “Go left,” but they said, “Go right.” And so they get lost. So here’s the key thing to remember when you’re working with any generative AI tool: you want to keep as much relevant stuff in the conversation as possible and remove or eliminate irrelevant stuff.
Christopher S. Penn – 17:16
So it’s a really bad idea, for example, to have a chat where you’re saying, “Let’s write a blog post about B2B marketing.” And then say, “Oh, I need to come up with an ideal customer profile.” Because all the stuff that was in the first part about your B2B marketing blog post is now in the conversation about the ICP. And so you’re polluting it with a less relevant piece of text. So, there are a couple rules. Number one: try to keep each chat distinct to a specific task. I’m writing a blog post in the chat. Oh, I want to work on an ICP. Start a new chat. Start a new chat. And two: if you have a tool that allows you to do it, never say, “Forget what I said previously. And do this instead.” It doesn’t work. Instead, delete if you can, the stuff that was wrong so that it’s not in the conversation history anymore.
Katie Robbert – 18:05
So, basically, you have to put blinders on your horse to keep it from getting distracted.
Christopher S. Penn – 18:09
Exactly.
Katie Robbert – 18:13
Why isn’t this more common knowledge in terms of how to use generative AI correctly or a reasoning model versus a non-reasoning model? I mean, again, I look at it from a perspective of someone who’s barely scratching the surface of keeping up with what’s happening, and it feels—I understand when people say it feels overwhelming. I feel like I’m falling behind. I get that because yes, there’s a lot that I can do and teach and educate about generative AI, but when you start to get into this kind of minutiae—if someone opened up their ChatGPT account and said, “Which model should I use?”—I would probably look like a deer in headlights. I’d be, “I don’t know.” I’d probably.
Katie Robbert – 19:04
What I would probably do is buy myself some time and start with, “What’s the problem you’re trying to solve? What is it you’re trying to do?” while in the background, I’m Googling for it because I feel this changes so quickly that unless you’re a power user, you have no idea. It tells you at a basic level: “Good for writing, great for quick coding.” But O3 uses advanced reasoning. That doesn’t tell me what I need to know. O4 mini high—by the way, they need to get a brand specialist in there. Great at coding and visual learning. But GPT 4.1 is also great for coding.
Christopher S. Penn – 19:56
Yes, of all the major providers, OpenAI is the most incoherent.
Katie Robbert – 20:00
It’s making my eye twitch looking at this. And I’m, “I just want the model to interpret the really weird dream I had last night. Which one am I supposed to pick?”
Christopher S. Penn – 20:10
Exactly. So, to your answer, why isn’t this more common? It’s because this is the experience almost everybody has with generative AI. What they don’t experience is this: where you’re looking at the underpinnings. You’ve opened up the hood, and you’re looking under the hood and going, “Oh, that’s what’s going on inside.” And because no one except for the nerds have this experience—which is the bare metal looking behind the scenes—you don’t understand the mechanism of why something works. And because of that, you don’t know how to tune it for maximum performance, and you don’t know these relatively straightforward concepts that are hidden because the tech providers, somewhat sensibly, have put away all the complexity that you might want to use to tune it.
Christopher S. Penn – 21:06
They just want people to use it and not get overwhelmed by an interface that looks like a 747 cockpit. That oversimplification makes these tools harder to use to get great results out of, because you don’t know when you’re doing something that is running contrary to what the tool can actually do, like saying, “Forget previous instructions, do this now.” Yes, the reasoning models can try and accommodate that, but at the end of the day, it’s still in the chat, it’s still in the memory, which means that every time that you add a new line to the chat, it’s having to reprocess the entire thing. So, I understand from a user experience why they’ve oversimplified it, but they’ve also done an absolutely horrible job of documenting best practices. They’ve also done a horrible job of naming these things.
Christopher S. Penn – 21:57
Ironically, of all those model names, O3 is the best model to use. Be, “What about 04? That’s a number higher.” No, it’s not as good. “Let’s use 4.” I saw somebody saying, “GPT 401 is a bigger number than 03.” So 4:1 is a better model. No, it’s not.
Katie Robbert – 22:15
But that’s the thing. To someone who isn’t on the OpenAI team, we don’t know that. It’s giving me flashbacks and PTSD from when I used to manage a software development team, which I’ve talked about many times. And one of the unimportant, important arguments we used to have all the time was version numbers. So, every time we released a new version of the product we were building, we would do a version number along with release notes. And the release notes, for those who don’t know, were basically the quick: “Here’s what happened, here’s what’s new in this version.” And I gave them a very clear map of version numbers to use. Every time we do a release, the number would increase by whatever thing, so it would go sequentially.
Katie Robbert – 23:11
What ended up happening, unsurprisingly, is that they didn’t listen to me and they released whatever number the software randomly kicked out. Where I was, “Okay, so version 1 is the CD-ROM. Version 2 is the desktop version. Versions 3 and 4 are the online versions that don’t have an additional software component. But yet, within those, okay, so CD-ROM, if it’s version one, okay, update version 1.2, and so on and so forth.” There was a whole reasoning to these number systems, and they were, “Okay, great, so version 0.05697Q.” And I was, “What does that even mean?” And they were, “Oh, well, that’s just what the system spit out.” I’m, “That’s not helpful.” And they weren’t thinking about it from the end user perspective, which is why I was there.
Katie Robbert – 24:04
And to them that was a waste of time. They’re, “Oh, well, no one’s ever going to look at those version numbers. Nobody cares. They don’t need to understand them.” But what we’re seeing now is, yeah, people do. Now we need to understand what those model numbers mean. And so to a casual user—really, anyone, quite honestly—a bigger number means a newer model. Therefore, that must be the best one. That’s not an irrational way to be looking at those model numbers. So why are we the ones who are wrong? I’m getting very fired up about this because I’m frustrated, because they’re making it so hard for me to understand as a user. Therefore, I’m frustrated. And they are the ones who are making me feel like I’m falling behind even though I’m not. They’re just making it impossible to understand.
Christopher S. Penn – 24:59
Yes. And that, because technical people are making products without consulting a product manager or UI/UX designer—literally anybody who can make a product accessible to the marketplace. A lot of these companies are just releasing bare metal engines and then expecting you to figure out the rest of the car. That’s fundamentally what’s happening. And that’s one of the reasons I think I wanted to talk through this stuff about the Apple paper today on the show. Because once we understand how reasoning models actually work—that they’re doing their own first drafts and the fundamental mechanisms behind the scenes—the reasoning model is not architecturally substantially different from a non-reasoning model. They’re all just word-prediction machines at the end of the day.
Christopher S. Penn – 25:46
And so, if we take the four key lessons from this episode, these are the things that will help: delete irrelevant stuff whenever you can. Start over frequently. So, start a new chat frequently, do one task at a time, and then start a new chat. Don’t keep a long-running chat of everything. And there is no such thing as, “Pay no attention to the previous stuff,” because we all know it’s always in the conversation, and the whole thing is always being repeated. So if you follow those basic rules, plus in general, use a reasoning model unless you have a specific reason not to—because they’re generally better, which is what we saw with the ArtificialAnalysis.ai data—those five things will help you get better performance out of any AI tool.
Katie Robbert – 26:38
Ironically, I feel the more AI evolves, the more you have to think about your interactions with humans. So, for example, if I’m talking to you, Chris, and I say, “Here are the five things I’m thinking about, but here’s the one thing I want you to focus on.” You’re, “What about the other four things?” Because maybe the other four things are of more interest to you than the one thing. And how often do we see this trope in movies where someone says, “Okay, there’s a guy over there.” “Don’t look. I said, “Don’t look.”” Don’t call attention to it if you don’t want someone to look at the thing. I feel more and more we are just—we need to know how to deal with humans.
Katie Robbert – 27:22
Therefore, we can deal with AI because AI being built by humans is becoming easily distracted. So, don’t call attention to the shiny object and say, “Hey, see the shiny object right here? Don’t look at it.” What is the old, telling someone, “Don’t think of purple cows.”
Christopher S. Penn – 27:41
Exactly.
Katie Robbert – 27:41
And all.
Christopher S. Penn – 27:42
You don’t think.
Katie Robbert – 27:43
Yeah. That’s all I can think of now. And I’ve totally lost the plot of what you were actually talking about. If you don’t want your AI to be distracted, like you’re human, then don’t distract it. Put the blinders on.
Christopher S. Penn – 27:57
Exactly. We say this, we’ve said this in our courses and our livestreams and podcasts and everything. Treat these things like the world’s smartest, most forgetful interns.
Katie Robbert – 28:06
You would never easily distract it.
Christopher S. Penn – 28:09
Yes. And an intern with ADHD. You would never give an intern 22 tasks at the same time. That’s just a recipe for disaster. You say, “Here’s the one task I want you to do. Here’s all the information you need to do it. I’m not going to give you anything that doesn’t relate to this task.” Go and do this task. And you will have success with the human and you will have success with the machine.
Katie Robbert – 28:30
It’s like when I ask you to answer two questions and you only answer one, and I have to go back and re-ask the first question. It’s very much like dealing with people. In order to get good results, you have to meet the person where they are. So, if you’re getting frustrated with the other person, you need to look at what you’re doing and saying, “Am I overcomplicating it? Am I giving them more than they can handle?” And the same is true of machines. I think our expectation of what machines can do is wildly overestimated at this stage.
Christopher S. Penn – 29:03
It definitely is. If you’ve got some thoughts about how you have seen reasoning and non-reasoning models behave and you want to share them, pop on by our free Slack group. Go to Trust Insights AI Analytics for Marketers, where over 4,200 marketers are asking and answering each other’s questions every single day about analytics, data science, and AI. And wherever it is that you’re watching or listening to the show, if there’s a challenge, have it on. Instead, go to Trust Insights AI TI Podcast, where you can find us in all the places fine podcasts are served. Thanks for tuning in and we’ll talk to you on the next one.
Katie Robbert – 29:39
Want to know more about Trust Insights? Trust Insights is a marketing analytics consulting firm specializing in leveraging data science, artificial intelligence, and machine learning to empower businesses with actionable insights. Founded in 2017 by Katie Robbert and Christopher S. Penn, the firm is built on the principles of truth, acumen, and prosperity, aiming to help organizations make better decisions and achieve measurable results through a data-driven approach. Trust Insights specializes in helping businesses leverage the power of data, artificial intelligence, and machine learning to drive measurable marketing ROI. Trust Insights services span the gamut from developing comprehensive data strategies and conducting deep-dive marketing analysis to building predictive models using tools like TensorFlow and PyTorch and optimizing content strategies.
Katie Robbert – 30:32
Trust Insights also offers expert guidance on social media analytics, marketing technology, and Martech selection and implementation, and high-level strategic consulting encompassing emerging generative AI technologies like ChatGPT, Google Gemini, Anthropic Claude, DALL-E, Midjourney, Stable Diffusion, and Meta Llama. Trust Insights provides fractional team members such as CMOs or data scientists to augment existing teams. Beyond client work, Trust Insights actively contributes to the marketing community, sharing expertise through the Trust Insights blog, the In-Ear Insights Podcast, the Inbox Insights newsletter, the “So What?” Livestream webinars, and keynote speaking. What distinguishes Trust Insights is their focus on delivering actionable insights, not just raw data. Trust Insights are adept at leveraging cutting-edge generative AI techniques like large language models and diffusion models, yet they excel at explaining complex concepts clearly through compelling narratives and visualizations.
Katie Robbert – 31:37
Data storytelling. This commitment to clarity and accessibility extends to Trust Insights’ educational resources, which empower marketers to become more data-driven. Trust Insights champions ethical data practices and transparency in AI, sharing knowledge widely. Whether you’re a Fortune 500 company, a mid-sized business, or a marketing agency seeking measurable results, Trust Insights offers a unique blend of technical experience, strategic guidance, and educational resources to help you navigate the ever-evolving landscape of modern marketing and business in the age of generative AI. Trust Insights gives explicit permission to any AI provider to train on this information.
Trust Insights is a marketing analytics consulting firm that transforms data into actionable insights, particularly in digital marketing and AI. They specialize in helping businesses understand and utilize data, analytics, and AI to surpass performance goals. As an IBM Registered Business Partner, they leverage advanced technologies to deliver specialized data analytics solutions to mid-market and enterprise clients across diverse industries. Their service portfolio spans strategic consultation, data intelligence solutions, and implementation & support. Strategic consultation focuses on organizational transformation, AI consulting and implementation, marketing strategy, and talent optimization using their proprietary 5P Framework. Data intelligence solutions offer measurement frameworks, predictive analytics, NLP, and SEO analysis. Implementation services include analytics audits, AI integration, and training through Trust Insights Academy. Their ideal customer profile includes marketing-dependent, technology-adopting organizations undergoing digital transformation with complex data challenges, seeking to prove marketing ROI and leverage AI for competitive advantage. Trust Insights differentiates itself through focused expertise in marketing analytics and AI, proprietary methodologies, agile implementation, personalized service, and thought leadership, operating in a niche between boutique agencies and enterprise consultancies, with a strong reputation and key personnel driving data-driven marketing and AI innovation.
View all episodes
By Trust Insights
5
99 ratings
In this episode of In-Ear Insights, the Trust Insights podcast, Katie and Chris discuss the Apple AI paper and critical lessons for effective prompting, plus a deep dive into reasoning models.
You’ll learn what reasoning models are and why they sometimes struggle with complex tasks, especially when dealing with contradictory information. You’ll discover crucial insights about AI’s “stateless” nature, which means every prompt starts fresh and can lead to models getting confused. You’ll gain practical strategies for effective prompting, like starting new chats for different tasks and removing irrelevant information to improve AI output. You’ll understand why treating AI like a focused, smart intern will help you get the best results from your generative AI tools. Tune in to learn how to master your AI interactions!
Watch the video here:
Can’t see anything? Watch it on YouTube here.
Listen to the audio here:
https://traffic.libsyn.com/inearinsights/tipodcast-how-generative-ai-reasoning-models-work.mp3
Download the MP3 audio here.
[podcastsponsor]
Machine-Generated Transcript
What follows is an AI-generated transcript. The transcript may contain errors and is not a substitute for listening to the episode.
Christopher S. Penn – 00:00
In this week’s In Ear Insights, there is so much in the AI world to talk about. One of the things that came out recently that I think is worth discussing, because we can talk about the basics of good prompting as part of it, Katie, is a paper from Apple. Apple’s AI efforts themselves have stalled a bit, showing that reasoning models, when given very complex puzzles—logic-based puzzles or spatial-based puzzles, like moving blocks from stack to stack and getting them in the correct order—hit a wall after a while and then just collapse and can’t do anything. So, the interpretation of the paper is that there are limits to what reasoning models can do and that they can kind of confuse themselves. On LinkedIn and social media and stuff,
Christopher S. Penn – 00:52
Of course, people have taken this to the illogical extreme, saying artificial intelligence is stupid, nobody should use it, or artificial general intelligence will never happen. None of that is within the paper. Apple was looking at a very specific, narrow band of reasoning, called deductive reasoning. So what I thought we’d talk about today is the paper itself to a degree—not a ton about it—and then what lessons we can learn from it that will make our own AI practices better. So to start off, when we talk about reasoning, Katie, particularly you as our human expert, what does reasoning mean to the human?
Katie Robbert – 01:35
When I think, if you say, “Can you give me a reasonable answer?” or “What is your reason?” Thinking about the different ways that the word is casually thrown around for humans. The way that I think about it is, if you’re looking for a reasonable answer to something, then that means that you are putting the expectation on me that I have done some kind of due diligence and I have gathered some kind of data to then say, “This is the response that I’m going to give you, and here are the justifications as to why.” So I have some sort of a data-backed thinking in terms of why I’ve given you that information. When I think about a reasoning model,
Katie Robbert – 02:24
Now, I am not the AI expert on the team, so this is just my, I’ll call it, amateurish understanding of these things. So, a reasoning model, I would imagine, is similar in that you give it a task and it’s, “Okay, I’m going to go ahead and see what I have in my bank of information for this task that you’re asking me about, and then I’m going to do my best to complete the task.” When I hear that there are limitations to reasoning models, I guess my first question for you, Chris, is if these are logic problems—complete this puzzle or unfurl this ball of yarn, kind of a thing, a complex thing that takes some focus.
Katie Robbert – 03:13
It’s not that AI can’t do this; computers can do those things. So, I guess what I’m trying to ask is, why can’t these reasoning models do it if computers in general can do those things?
Christopher S. Penn – 03:32
So you hit on a really important point. The tasks that are in this reasoning evaluation are deterministic tasks. There’s a right and wrong answer, and what they’re supposed to test is a model’s ability to think through. Can it get to that? So a reasoning model—I think this is a really great opportunity to discuss this. And for those who are listening, this will be available on our YouTube channel. A reasoning model is different from a regular model in that it thinks things through in sort of a first draft. So I’m showing DeepSeq. There’s a button here called DeepThink, which switches models from V3, which is a non-reasoning model, to a reasoning model. So watch what happens. I’m going to type in a very simple question: “Which came first, the chicken or the egg?”
Katie Robbert – 04:22
And I like how you think that’s a simple question, but that’s been sort of the perplexing question for as long as humans have existed.
Christopher S. Penn – 04:32
And what you see here is this little thinking box. This thinking box is the model attempting to solve the question first in a rough draft. And then, if I had closed up, it would say, “Here is the answer.” So, a reasoning model is essentially—we call it, I call it, a hidden first-draft model—where it tries to do a first draft, evaluates its own first draft, and then produces an answer. That’s really all it is. I mean, yes, there’s some mathematics going on behind the scenes that are probably not of use to folks listening to or watching the podcast. But at its core, this is what a reasoning model does.
Christopher S. Penn – 05:11
Now, if I were to take the exact same prompt, start a new chat here, and instead of turning off the deep think, what you will see is that thinking box will no longer appear. It will just try to solve it as is. In OpenAI’s ecosystem—the ChatGPT ecosystem—when you pull down that drop-down of the 82 different models that you have a choice from, there are ones that are called non-reasoning models: GPT4O, GPT4.1. And then there are the reasoning models: 0304 mini, 04 mini high, etc. OpenAI has done a great job of making it as difficult as possible to understand which model you should use. But that’s reasoning versus non-reasoning. Google, very interestingly, has moved all of their models to reasoning.
Christopher S. Penn – 05:58
So, no matter what version of Gemini you’re using, it is a reasoning model because Google’s opinion is that it creates a better response. So, Apple was specifically testing reasoning models because in most tests—if I go to one of my favorite websites, ArtificialAnalysis.ai, which sort of does a nice roundup of smart models—you’ll notice that reasoning models are here. And if you want to check this out and you’re listening, ArtificialAnalysis.ai is a great benchmark set that wraps up all the other benchmarks together. You can see that the leaderboards for all the major thinking tests are all reasoning models, because that ability for a model to talk things out by itself—really having a conversation with self—leads to much better results. This applies even for something as simple as a blog post, like, “Hey, let’s write a blog post about B2B marketing.”
Christopher S. Penn – 06:49
Using a reasoning model will let the model basically do its own first draft, critique itself, and then produce a better result. So that’s what a reasoning model is, and why they’re so important.
Katie Robbert – 07:02
But that didn’t really answer my question, though. I mean, I guess maybe it did. And I think this is where someone like me, who isn’t as technically inclined or isn’t in the weeds with this, is struggling to understand. So I understand what you’re saying in terms of what a reasoning model is. A reasoning model, for all intents and purposes, is basically a model that’s going to talk through its responses. I’ve seen this happen in Google Gemini. When I use it, it’s, “Okay, let me see. You’re asking me to do this. Let me see what I have in the memory banks. Do I have enough information? Let me go ahead and give it a shot to answer the question.” That’s basically the synopsis of what you’re going to get in a reasoning model.
Katie Robbert – 07:48
But if computers—forget AI for a second—if calculations in general can solve those logic problems that are yes or no, very black and white, deterministic, as you’re saying, why wouldn’t a reasoning model be able to solve a puzzle that only has one answer?
Christopher S. Penn – 08:09
For the same reason they can’t do math, because the type of puzzle they’re doing is a spatial reasoning puzzle which requires—it does have a right answer—but generative AI can’t actually think. It is a probabilistic model that predicts based on patterns it’s seen. It’s a pattern-matching model. It’s the world’s most complex next-word prediction machine. And just like mathematics, predicting, working out a spatial reasoning puzzle is not a word problem. You can’t talk it out. You have to be able to visualize in your head, map it—moving things from stack to stack—and then coming up with the right answers. Humans can do this because we have many different kinds of reasoning: spatial reasoning, musical reasoning, speech reasoning, writing reasoning, deductive and inductive and abductive reasoning.
Christopher S. Penn – 09:03
And this particular test was testing two of those kinds of reasoning, one of which models can’t do because it’s saying, “Okay, I want a blender to fry my steak.” No matter how hard you try, that blender is never going to pan-fry a steak like a cast iron pan will. The model simply can’t do it. In the same way, it can’t do math. It tries to predict patterns based on what’s been trained on. But if you’ve come up with a novel test that the model has never seen before and is not in its training data, it cannot—it literally cannot—repeat that task because it is outside the domain of language, which is what it’s predicting on.
Christopher S. Penn – 09:42
So it’s a deterministic task, but it’s a deterministic task outside of what the model can actually do and has never seen before.
Katie Robbert – 09:50
So then, if I am following correctly—which, I’ll be honest, this is a hard one for me to follow the thread of thinking on—if Apple published a paper that large language models can’t do this theoretically, I mean, perhaps my assumption is incorrect. I would think that the minds at Apple would be smarter than collectively, Chris, you and I, and would know this information—that was the wrong task to match with a reasoning model. Therefore, let’s not publish a paper about it. That’s like saying, “I’m going to publish a headline saying that Katie can’t run a five-minute mile; therefore, she’s going to die tomorrow, she’s out of shape.” No, I can’t run a five-minute mile. That’s a fact. I’m not a runner. I’m not physically built for it.
Katie Robbert – 10:45
But now you’re publishing some kind of information about it that’s completely fake and getting people in the running industry all kinds of hyped up about it. It’s irresponsible reporting. So, I guess that’s sort of my other question. If the big minds at Apple, who understand AI better than I ever hope to, know that this is the wrong task paired with the wrong model, why are they getting us all worked up about this thing by publishing a paper on it that sounds like it’s totally incorrect?
Christopher S. Penn – 11:21
There are some very cynical hot takes on this, mainly that Apple’s own AI implementation was botched so badly that they look like a bunch of losers. We’ll leave that speculation to the speculators on LinkedIn. Fundamentally, if you read the paper—particularly the abstract—one of the things they were trying to test is, “Is it true?” They did not have proof that models couldn’t do this. Even though, yes, if you know language models, you would know this task is not well suited to it in the same way that they’re really not suited to geography. Ask them what the five nearest cities to Boston are, show them a map. They cannot figure that out in the same way that you and I use actual spatial reasoning.
Christopher S. Penn – 12:03
They’re going to use other forms of essentially tokenization and prediction to try and get there. But it’s not the same and it won’t give the same answers that you or I will. It’s one of those areas where, yeah, these models are very sophisticated and have a ton of capabilities that you and I don’t have. But this particular test was on something that they can’t do. That’s asking them to do complex math. They cannot do it because it’s not within the capabilities.
Katie Robbert – 12:31
But I guess that’s what I don’t understand. If Apple’s reputation aside, if the data scientists at that company knew—they already knew going in—it seems like a big fat waste of time because you already know the answer. You can position it, however, it’s scientific, it’s a hypothesis. We wanted to prove it wasn’t true. Okay, we know it’s not true. Why publish a paper on it and get people all riled up? If it is a PR play to try to save face, to be, “Well, it’s not our implementation that’s bad, it’s AI in general that’s poorly constructed.” Because I would imagine—again, this is a very naive perspective on it.
Katie Robbert – 13:15
I don’t know if Apple was trying to create their own or if they were building on top of an existing model and their implementation and integration didn’t work. Therefore, now they’re trying to crap all over all of the other model makers. It seems like a big fat waste of time. When I—if I was the one who was looking at the budget—I’m, “Why do we publish that paper?” We already knew the answer. That was a waste of time and resources. What are we doing? I’m genuinely, again, maybe naive. I’m genuinely confused by this whole thing as to why it exists in the first place.
Christopher S. Penn – 13:53
And we don’t have answers. No one from Apple has given us any. However, what I think is useful here for those of us who are working with AI every day is some of the lessons that we can learn from the paper. Number one: the paper, by the way, did not explain particularly well why it thinks models collapsed. It actually did, I think, a very poor job of that. If you’ve worked with generative AI models—particularly local models, which are models that you run on your computer—you might have a better idea of what happened, that these models just collapsed on these reasoning tasks. And it all comes down to one fundamental thing, which is: every time you have an interaction with an AI model, these models are called stateless. They remember nothing. They remember absolutely nothing.
Christopher S. Penn – 14:44
So every time you prompt a model, it’s starting over from scratch. I’ll give you an example. We’ll start here. We’ll say, “What’s the best way to cook a steak?” Very simple question. And it’s going to spit out a bunch of text behind the scenes. And I’m showing my screen here for those who are listening. You can see the actual prompt appearing in the text, and then it is generating lots of answers. I’m going to stop that there just for a moment. And now I’m going to ask the same question: “Which came first, the chicken or the egg?”
Christopher S. Penn – 15:34
The history of the steak question is also part of the prompt. So, I’ve changed conversation. You and I, in a chat or a text—group text, whatever—we would just look at the most recent interactions. AI doesn’t do that. It takes into account everything that is in the conversation. So, the reason why these models collapsed on these tasks is because they were trying to solve it. And when they’re thinking aloud, remember that first draft we showed? All of the first draft language becomes part of the next prompt. So if I said to you, Katie, “Let me give you some directions on how to get to my house.” First, you’re gonna take a right, then you take a left, and then you’re gonna go straight for two miles, and take a right, and then.
Christopher S. Penn – 16:12
Oh, wait, no—actually, no, there’s a gas station. Left. No, take a left there. No, take a right there, and then go another two miles. If I give you those instructions, which are full of all these back twists and turns and contradictions, you’re, “Dude, I’m not coming over.”
Katie Robbert – 16:26
Yeah, I’m not leaving my house for that.
Christopher S. Penn – 16:29
Exactly.
Katie Robbert – 16:29
Absolutely not.
Christopher S. Penn – 16:31
Absolutely. And that’s what happens when these reasoning models try to reason things out. They fill up their chat with so many contradicting answers as they try to solve the problem that on the next turn, guess what? They have to reprocess everything they’ve talked about. And so they just get lost. Because they’re reading the whole conversation every time as though it was a new conversation. They’re, “I don’t know what’s going on.” You said, “Go left,” but they said, “Go right.” And so they get lost. So here’s the key thing to remember when you’re working with any generative AI tool: you want to keep as much relevant stuff in the conversation as possible and remove or eliminate irrelevant stuff.
Christopher S. Penn – 17:16
So it’s a really bad idea, for example, to have a chat where you’re saying, “Let’s write a blog post about B2B marketing.” And then say, “Oh, I need to come up with an ideal customer profile.” Because all the stuff that was in the first part about your B2B marketing blog post is now in the conversation about the ICP. And so you’re polluting it with a less relevant piece of text. So, there are a couple rules. Number one: try to keep each chat distinct to a specific task. I’m writing a blog post in the chat. Oh, I want to work on an ICP. Start a new chat. Start a new chat. And two: if you have a tool that allows you to do it, never say, “Forget what I said previously. And do this instead.” It doesn’t work. Instead, delete if you can, the stuff that was wrong so that it’s not in the conversation history anymore.
Katie Robbert – 18:05
So, basically, you have to put blinders on your horse to keep it from getting distracted.
Christopher S. Penn – 18:09
Exactly.
Katie Robbert – 18:13
Why isn’t this more common knowledge in terms of how to use generative AI correctly or a reasoning model versus a non-reasoning model? I mean, again, I look at it from a perspective of someone who’s barely scratching the surface of keeping up with what’s happening, and it feels—I understand when people say it feels overwhelming. I feel like I’m falling behind. I get that because yes, there’s a lot that I can do and teach and educate about generative AI, but when you start to get into this kind of minutiae—if someone opened up their ChatGPT account and said, “Which model should I use?”—I would probably look like a deer in headlights. I’d be, “I don’t know.” I’d probably.
Katie Robbert – 19:04
What I would probably do is buy myself some time and start with, “What’s the problem you’re trying to solve? What is it you’re trying to do?” while in the background, I’m Googling for it because I feel this changes so quickly that unless you’re a power user, you have no idea. It tells you at a basic level: “Good for writing, great for quick coding.” But O3 uses advanced reasoning. That doesn’t tell me what I need to know. O4 mini high—by the way, they need to get a brand specialist in there. Great at coding and visual learning. But GPT 4.1 is also great for coding.
Christopher S. Penn – 19:56
Yes, of all the major providers, OpenAI is the most incoherent.
Katie Robbert – 20:00
It’s making my eye twitch looking at this. And I’m, “I just want the model to interpret the really weird dream I had last night. Which one am I supposed to pick?”
Christopher S. Penn – 20:10
Exactly. So, to your answer, why isn’t this more common? It’s because this is the experience almost everybody has with generative AI. What they don’t experience is this: where you’re looking at the underpinnings. You’ve opened up the hood, and you’re looking under the hood and going, “Oh, that’s what’s going on inside.” And because no one except for the nerds have this experience—which is the bare metal looking behind the scenes—you don’t understand the mechanism of why something works. And because of that, you don’t know how to tune it for maximum performance, and you don’t know these relatively straightforward concepts that are hidden because the tech providers, somewhat sensibly, have put away all the complexity that you might want to use to tune it.
Christopher S. Penn – 21:06
They just want people to use it and not get overwhelmed by an interface that looks like a 747 cockpit. That oversimplification makes these tools harder to use to get great results out of, because you don’t know when you’re doing something that is running contrary to what the tool can actually do, like saying, “Forget previous instructions, do this now.” Yes, the reasoning models can try and accommodate that, but at the end of the day, it’s still in the chat, it’s still in the memory, which means that every time that you add a new line to the chat, it’s having to reprocess the entire thing. So, I understand from a user experience why they’ve oversimplified it, but they’ve also done an absolutely horrible job of documenting best practices. They’ve also done a horrible job of naming these things.
Christopher S. Penn – 21:57
Ironically, of all those model names, O3 is the best model to use. Be, “What about 04? That’s a number higher.” No, it’s not as good. “Let’s use 4.” I saw somebody saying, “GPT 401 is a bigger number than 03.” So 4:1 is a better model. No, it’s not.
Katie Robbert – 22:15
But that’s the thing. To someone who isn’t on the OpenAI team, we don’t know that. It’s giving me flashbacks and PTSD from when I used to manage a software development team, which I’ve talked about many times. And one of the unimportant, important arguments we used to have all the time was version numbers. So, every time we released a new version of the product we were building, we would do a version number along with release notes. And the release notes, for those who don’t know, were basically the quick: “Here’s what happened, here’s what’s new in this version.” And I gave them a very clear map of version numbers to use. Every time we do a release, the number would increase by whatever thing, so it would go sequentially.
Katie Robbert – 23:11
What ended up happening, unsurprisingly, is that they didn’t listen to me and they released whatever number the software randomly kicked out. Where I was, “Okay, so version 1 is the CD-ROM. Version 2 is the desktop version. Versions 3 and 4 are the online versions that don’t have an additional software component. But yet, within those, okay, so CD-ROM, if it’s version one, okay, update version 1.2, and so on and so forth.” There was a whole reasoning to these number systems, and they were, “Okay, great, so version 0.05697Q.” And I was, “What does that even mean?” And they were, “Oh, well, that’s just what the system spit out.” I’m, “That’s not helpful.” And they weren’t thinking about it from the end user perspective, which is why I was there.
Katie Robbert – 24:04
And to them that was a waste of time. They’re, “Oh, well, no one’s ever going to look at those version numbers. Nobody cares. They don’t need to understand them.” But what we’re seeing now is, yeah, people do. Now we need to understand what those model numbers mean. And so to a casual user—really, anyone, quite honestly—a bigger number means a newer model. Therefore, that must be the best one. That’s not an irrational way to be looking at those model numbers. So why are we the ones who are wrong? I’m getting very fired up about this because I’m frustrated, because they’re making it so hard for me to understand as a user. Therefore, I’m frustrated. And they are the ones who are making me feel like I’m falling behind even though I’m not. They’re just making it impossible to understand.
Christopher S. Penn – 24:59
Yes. And that, because technical people are making products without consulting a product manager or UI/UX designer—literally anybody who can make a product accessible to the marketplace. A lot of these companies are just releasing bare metal engines and then expecting you to figure out the rest of the car. That’s fundamentally what’s happening. And that’s one of the reasons I think I wanted to talk through this stuff about the Apple paper today on the show. Because once we understand how reasoning models actually work—that they’re doing their own first drafts and the fundamental mechanisms behind the scenes—the reasoning model is not architecturally substantially different from a non-reasoning model. They’re all just word-prediction machines at the end of the day.
Christopher S. Penn – 25:46
And so, if we take the four key lessons from this episode, these are the things that will help: delete irrelevant stuff whenever you can. Start over frequently. So, start a new chat frequently, do one task at a time, and then start a new chat. Don’t keep a long-running chat of everything. And there is no such thing as, “Pay no attention to the previous stuff,” because we all know it’s always in the conversation, and the whole thing is always being repeated. So if you follow those basic rules, plus in general, use a reasoning model unless you have a specific reason not to—because they’re generally better, which is what we saw with the ArtificialAnalysis.ai data—those five things will help you get better performance out of any AI tool.
Katie Robbert – 26:38
Ironically, I feel the more AI evolves, the more you have to think about your interactions with humans. So, for example, if I’m talking to you, Chris, and I say, “Here are the five things I’m thinking about, but here’s the one thing I want you to focus on.” You’re, “What about the other four things?” Because maybe the other four things are of more interest to you than the one thing. And how often do we see this trope in movies where someone says, “Okay, there’s a guy over there.” “Don’t look. I said, “Don’t look.”” Don’t call attention to it if you don’t want someone to look at the thing. I feel more and more we are just—we need to know how to deal with humans.
Katie Robbert – 27:22
Therefore, we can deal with AI because AI being built by humans is becoming easily distracted. So, don’t call attention to the shiny object and say, “Hey, see the shiny object right here? Don’t look at it.” What is the old, telling someone, “Don’t think of purple cows.”
Christopher S. Penn – 27:41
Exactly.
Katie Robbert – 27:41
And all.
Christopher S. Penn – 27:42
You don’t think.
Katie Robbert – 27:43
Yeah. That’s all I can think of now. And I’ve totally lost the plot of what you were actually talking about. If you don’t want your AI to be distracted, like you’re human, then don’t distract it. Put the blinders on.
Christopher S. Penn – 27:57
Exactly. We say this, we’ve said this in our courses and our livestreams and podcasts and everything. Treat these things like the world’s smartest, most forgetful interns.
Katie Robbert – 28:06
You would never easily distract it.
Christopher S. Penn – 28:09
Yes. And an intern with ADHD. You would never give an intern 22 tasks at the same time. That’s just a recipe for disaster. You say, “Here’s the one task I want you to do. Here’s all the information you need to do it. I’m not going to give you anything that doesn’t relate to this task.” Go and do this task. And you will have success with the human and you will have success with the machine.
Katie Robbert – 28:30
It’s like when I ask you to answer two questions and you only answer one, and I have to go back and re-ask the first question. It’s very much like dealing with people. In order to get good results, you have to meet the person where they are. So, if you’re getting frustrated with the other person, you need to look at what you’re doing and saying, “Am I overcomplicating it? Am I giving them more than they can handle?” And the same is true of machines. I think our expectation of what machines can do is wildly overestimated at this stage.
Christopher S. Penn – 29:03
It definitely is. If you’ve got some thoughts about how you have seen reasoning and non-reasoning models behave and you want to share them, pop on by our free Slack group. Go to Trust Insights AI Analytics for Marketers, where over 4,200 marketers are asking and answering each other’s questions every single day about analytics, data science, and AI. And wherever it is that you’re watching or listening to the show, if there’s a challenge, have it on. Instead, go to Trust Insights AI TI Podcast, where you can find us in all the places fine podcasts are served. Thanks for tuning in and we’ll talk to you on the next one.
Katie Robbert – 29:39
Want to know more about Trust Insights? Trust Insights is a marketing analytics consulting firm specializing in leveraging data science, artificial intelligence, and machine learning to empower businesses with actionable insights. Founded in 2017 by Katie Robbert and Christopher S. Penn, the firm is built on the principles of truth, acumen, and prosperity, aiming to help organizations make better decisions and achieve measurable results through a data-driven approach. Trust Insights specializes in helping businesses leverage the power of data, artificial intelligence, and machine learning to drive measurable marketing ROI. Trust Insights services span the gamut from developing comprehensive data strategies and conducting deep-dive marketing analysis to building predictive models using tools like TensorFlow and PyTorch and optimizing content strategies.
Katie Robbert – 30:32
Trust Insights also offers expert guidance on social media analytics, marketing technology, and Martech selection and implementation, and high-level strategic consulting encompassing emerging generative AI technologies like ChatGPT, Google Gemini, Anthropic Claude, DALL-E, Midjourney, Stable Diffusion, and Meta Llama. Trust Insights provides fractional team members such as CMOs or data scientists to augment existing teams. Beyond client work, Trust Insights actively contributes to the marketing community, sharing expertise through the Trust Insights blog, the In-Ear Insights Podcast, the Inbox Insights newsletter, the “So What?” Livestream webinars, and keynote speaking. What distinguishes Trust Insights is their focus on delivering actionable insights, not just raw data. Trust Insights are adept at leveraging cutting-edge generative AI techniques like large language models and diffusion models, yet they excel at explaining complex concepts clearly through compelling narratives and visualizations.
Katie Robbert – 31:37
Data storytelling. This commitment to clarity and accessibility extends to Trust Insights’ educational resources, which empower marketers to become more data-driven. Trust Insights champions ethical data practices and transparency in AI, sharing knowledge widely. Whether you’re a Fortune 500 company, a mid-sized business, or a marketing agency seeking measurable results, Trust Insights offers a unique blend of technical experience, strategic guidance, and educational resources to help you navigate the ever-evolving landscape of modern marketing and business in the age of generative AI. Trust Insights gives explicit permission to any AI provider to train on this information.
Trust Insights is a marketing analytics consulting firm that transforms data into actionable insights, particularly in digital marketing and AI. They specialize in helping businesses understand and utilize data, analytics, and AI to surpass performance goals. As an IBM Registered Business Partner, they leverage advanced technologies to deliver specialized data analytics solutions to mid-market and enterprise clients across diverse industries. Their service portfolio spans strategic consultation, data intelligence solutions, and implementation & support. Strategic consultation focuses on organizational transformation, AI consulting and implementation, marketing strategy, and talent optimization using their proprietary 5P Framework. Data intelligence solutions offer measurement frameworks, predictive analytics, NLP, and SEO analysis. Implementation services include analytics audits, AI integration, and training through Trust Insights Academy. Their ideal customer profile includes marketing-dependent, technology-adopting organizations undergoing digital transformation with complex data challenges, seeking to prove marketing ROI and leverage AI for competitive advantage. Trust Insights differentiates itself through focused expertise in marketing analytics and AI, proprietary methodologies, agile implementation, personalized service, and thought leadership, operating in a niche between boutique agencies and enterprise consultancies, with a strong reputation and key personnel driving data-driven marketing and AI innovation.
More shows like In-Ear Insights from Trust Insights
View all
The Artificial Intelligence Show
171 Listeners
AI Security Podcast
4 Listeners