In this episode of Selling the Cloud, Josh Payne—Founder & CEO of CoFrame and creator of GPT-Migrate—joins Mark Petruzzi and KK Anderson to explore how AI agents and continuously optimizing websites are changing the future of digital growth.
Josh shares how CoFrame is helping mid-market and enterprise companies drive 30%+ increases in conversion by building adaptive, AI-powered interfaces that test, learn, and optimize in real time. He also unpacks how his team partnered with OpenAI to create a state-of-the-art UI code generation model—and what this signals about the future of growth marketing.
Whether you’re a CMO, growth lead, or founder looking to build durable competitive advantage through AI, this episode will shift your mindset and your roadmap.
What You’ll Learn:
- From Static to Living Interfaces: Why websites are broken—and how AI agents can continuously optimize them based on real-time data and buyer behavior.
- The End of Manual A/B Testing: How Josh’s team automates experimentation at scale to unlock compound conversion lifts without traditional resource bottlenecks.
- Inside the OpenAI Partnership: How CoFrame trained a UI-specific code generation model that understands brand, layout, and style—then deployed it at scale.
- Building Competitive Advantage with AI: Why businesses that embrace continuous optimization today will outpace those stuck in manual, quarterly workflows.
- Designing the Future Growth Stack: How AI agents, data orchestration, and strategy automation will transform marketing roles and unlock new GTM velocity.
Key Topics:
- Living interfaces vs. static websites
- AI code generation for brand-consistent design
- Scaling experimentation with AI agents
- The future of CRO: conversion rate optimization
- AI in paid media, lifecycle marketing, and SEO
- Strategy design vs. manual execution
- How to evaluate AI models like Claude, Gemini, GPT-4
- Building “quant growth” systems for marketing
- Why the next big AI use case is growth engineering
Guest Spotlight: Josh Payne
Josh is the CEO of CoFrame and a repeat founder with deep roots in AI. He previously co-founded Autograph (a $2B+ unicorn), created GPT-Migrate, and authored over 20 patents and AI research papers. At CoFrame, he’s pioneering AI-first digital experiences that adapt to users in real time.
Resources & Mentions:
- CoFrame: https://coframe.com
- Article: The New Science of Growth Marketing – Every
- Tech: GPT-4 Vision, Claude, MoonDream, OpenAI fine-tuning
- Tools Mentioned: Optimizely, Adobe Target, VWO, Amplitude
- Topic: Quant growth, UI code generation, AI agents
🎧 Listen now and follow Selling the Cloud for cutting-edge conversations with the builders, strategists, and GTM leaders shaping the future of sales, growth, and AI.
Mark Petruzzi (00:02)
Welcome to today's episode of Selling the Cloud podcast. We are excited to welcome Josh Payne, founder and CEO of CoFrame. Josh is a serial entrepreneur and AI pioneer who has co-founded multiple successful companies, including Autograph, a 2 billion plus unicorn backed by A16Z and Access Bell, which has been acquired by Tata.
He's also the creator of GPT Migrate, one of the first popular coding agents and has authored over 20 papers and patents in AI. At CoFrame, Josh is building the future of digital interfaces, websites and apps that continuously optimize themselves using AI. The company recently raised 9.3 million from Coastal Ventures.
and partnered with OpenAI to develop breakthrough UI code generation technology that's driving a 30 % plus improvement in conversion rates for mid-market and enterprise clients alike. Today, Josh will share how AI agents are revolutionizing growth marketing and website optimization, what it means to build living interfaces, and how go-to-market teams should prepare for the autonomous
optimization era that we are in today. So welcome, Josh, welcome.
Josh (01:31)
Thanks so much for having me, Mark. Good to see you.
Mark Petruzzi (01:33)
Excellent. Thank you. Okay, four themes we'll focus on today. From static to living, the AI powered interface revolution, the OpenAI partnership building visually grounded AI at scale, autonomous growth, how AI agents are replacing manual optimization, and the future of marketing technology and growth engineering. So first area here, Josh, for us to jump into.
You describe CoFrame as creating living interfaces rather than static websites. What's fundamentally broken about how we think about digital experiences today?
Josh (02:15)
Yeah, so the problem with digital experiences as we know them, it comes on both sides. There's the problem from the business side, which is that it takes a lot of effort to make these digital experiences like websites or apps better for our customers. And people tend to spend a lot of money and time doing so. On the customer side, it's also an issue because you have these experiences that are one size fits all. They're not adaptive. They're not personal, you know?
They lack the context of you as a person. And the way that we're approaching this is the problem is, let's think about this from the ground up. If you could redesign the concept of digital interfaces today in a world where AI is possible and as powerful as it is, you know, look at the trend of where things are going in terms of the capabilities that it has. How would you design these customer experiences?
And that was really our impetus, which was, let's think about a world in which these digital interfaces were living in a sense, they could be adaptive and personal from the ground up. And so what we're doing essentially is, is enabling that capability for the businesses that we work with and their customer experiences, allowing their, their teams to not have to do all that grunt work and trying to continuously stay on top of.
⁓ of making those customer experiences better at great cost and ⁓ great time.
KK Anderson (03:51)
Like, what would a use case be or an example, just to go a little deeper into living interfaces? I think it sounds amazing.
Josh (03:59)
Absolutely. The biggest one that we focus on at the moment is website optimization. So you're probably very familiar with the idea of websites being a conduit for which consumers can interact and transact. And the problem with websites today is that, like I was mentioning, it takes a lot of effort to improve the conversion rate ⁓ of these sites. 95 % of
plus of traffic on the internet doesn't convert, right? It's people who go to a website and they check it out and they decided that maybe it's not a fit for them or they don't get the punchline, they don't get the idea. And this is truer in other industries than it is, know, for instance, in e-commerce, this is a huge, huge problem because the website is everything, right? And, you know, consumer finance tends to be a big one as well. And so,
What you end up seeing is ⁓ these teams spending a lot of time running experiments on these websites. You'll have a product manager or a marketer come up with a concept or an idea of what to test. They'll pass that on to a designer. They'll come up with designs and concepts. They'll then go and pass that on to an engineering team, which will go and implement the idea to be tested in the website. And that handoff process, that whole process takes weeks, sometimes months.
to do. And just an insane amount of time, when you look at the dollar value of time, an insane amount of cost and headcount. ⁓ And so it's a very broken process. It's still very beneficial and valuable, because every win that you get, every better version of that experience, tends to give you significantly more revenue, because you're stacking those conversions over time. You start to see the compounding benefits.
It's really, really painful. so the use case here that we're very focused on at CoFrame, at least, is bringing this idea of adaptive, personalized websites where different people can have different experiences that are suited to them to really help to increase that end KPI that the business cares about, usually a conversion KPI.
Mark Petruzzi (06:20)
So Josh, you have built multiple successful companies in different paradigms. What made you pivot from traditional SaaS to this AI-first approach with CoFrame?
Josh (06:35)
Yeah, I mean, this is a big question a lot of operators are facing right now, which is, how is the role of software going to evolve in this world of AI? I think we're seeing the commoditization of software. starting to see, on the flip side, we're seeing a lot of incredible new capabilities that AI is enabling. I actually think that one of the biggest opportunities right now in the space overall, in the world, is
taking processes that are manual and take a lot of work and then applying ⁓ sort of an AI first mindset to that to figure out how can we automate or optimize different parts of that process and then deliver that outcome with way less effort involved. So an example here of what we're doing, know, conversion optimization, like I was saying, that kind of takes, you know, design and engineering and product management and a bunch of different manual functions that are tedious. And so
Our approach was, can we break down that whole long process that is delivering the outcome, which is basically experiments for your website and use AI to significantly accelerate and then automate that ⁓ end to end. Now there's still human involvement for sure. And there's, there's like a gradient of, of involvement, right? But on the level of completely manual to completely automated, we're gradually moving toward that automation piece. And I think that that's.
really the biggest opportunity currently in the world of AI is looking at what is currently manual and trying to figure out systematically how can we automate that and ⁓ make it not as painful ultimately.
KK Anderson (08:17)
So really interesting. where in your professional opinion do most companies go wrong when they're trying to optimize their websites and their digital experiences? And I heard what you said earlier. And by the way, once four to eight weeks have passed and the new website is launched, that content is stale. there you are going back at it again.
Talk to me about where they go wrong and how you recommend that they shift.
Josh (08:53)
Yeah, so
I mean, a big thing that I see a lot is people are just not testing enough. know, there's that when you look at the math behind conversion optimization and experimentation, it's pretty astounding. you know, if you like, think about ⁓ a series of tests that you run, you find a couple of wins within those tests. It's not like those wins are going to give you, you know, a boost of revenue for that month and then you're done. No, that win stays with you for good, for the most part.
And they also compound over time. So if you find a 2 % conversion left here, a very small left here, you know, and then a 5 % over here, and then a 3 % over there, you suddenly have 10 % in aggregate conversion left. That could be 10 % more revenue, you know, over the course of forever. Right. And so that comp, that's why compounding is so powerful here, because you start to be able to see, you know, this like, this, this, this super linear growth.
⁓ of your business value.
Mark Petruzzi (09:56)
Yeah, what I love about that, Josh, is you're kind of taking us to continuous, instead of periodic testing. it's just, and that's the beauty of AI and it's the beauty of the intelligence that it's building in some of these large language models that, like you said, it compounds, they get better, they get smarter, and that's phenomenal. And, you know, I guess we're putting the A-B test out of
Josh (10:05)
Yes, exactly.
Mark Petruzzi (10:26)
at a commission at this point, because unless you want to look at it as A-B testing 24-7. yeah, so ⁓ what's the first thing a growth leader should understand about moving from this periodic optimization to this continuous AI-driven improvement?
Josh (10:32)
Yeah, A to Z testing.
KK Anderson (10:35)
Yeah.
Josh (10:46)
Yeah, so first thing is just like a mindset shift, right? A-B testing is valuable and it has its place. And you can still compound those wins over time with A-B testing. ⁓ We are now living in a world where the cost of creating new experiments is very low. So what I like to usually think through ⁓ when it comes to opportunities in the AI space is we talked a lot about automation, right?
like taking something that is manual and autumny. I think that's the biggest opportunity currently. ⁓ Now, there's another piece of this, which I think is also very valuable, we think of that a lot, which is what wasn't possible before that is now possible. So net new superhuman capabilities. So first is taking a human capability and then just performing that. And then the second bucket is
taking something that just wasn't possible before or feasible or reasonable to do and doing that. And so continuous optimization actually falls in that second bucket. The first bucket being here's how people are optimizing today with A-B testing. And we're just going to automate that. The second bucket is what new types of tests are now possible with AI. And I'll give you a simple example of this. ⁓
KK Anderson (11:54)
The superhuman?
Josh (12:11)
⁓ Let's see, Booking.com is actually probably one of the closest to this. They're a really, really advanced company when it comes to growth and optimization. They're running a thousand plus experiments at any given time. ⁓ But with their level of traffic, they could be running probably tens of thousands of experiments at any given time. ⁓ That's just not feasible to do ⁓ with ⁓ purely with human labor, right? It would just not make sense from an economic standpoint of time spent.
KK Anderson (12:34)
humans.
Josh (12:40)
And so yes, you'd have to cut tests off sooner. You'd probably have to use things like multi-armed bandits, advanced techniques to test lots more variants. ⁓ But in doing so, you're squeezing more value out of all that traffic that they have. And so if you can 10x your testing velocity, ⁓ that's A, not very feasible to do without the assistance of a technology like AI. But also B, it's very valuable if you can do it.
And so that's where we see this idea of continuous testing coming into play.
KK Anderson (13:16)
and that compounds over time. Yeah.
Josh (13:19)
yeah, that's the beauty
about compounding. It's like every win that you get, you know, builds on itself. And so the faster you compound, the more value your business is going to be receiving. And that's why a lot of our customers have been able to see like really big benefits very quickly with us.
KK Anderson (13:25)
Mm-hmm.
Well, and last question before we move on to the OpenAI partnership, which we're excited to learn about. So how hard is it to go from your periodic optimization or shall we say once a year optimization for some companies to this? Is it a big lift to switch? Do you have to change your whole website? What does that even entail?
Josh (13:58)
Yeah, it depends on how you're doing it. ⁓ I mean, I could speak to the co-frame piece of this, obviously. And for us, we come in and we're operational on day one with the companies that we work with. So it's no lift, basically, except for approving variations. The approval workflow still exists. And you have to make sure every variation and every experiment is given a sign off by the relevant stakeholders. for listeners at home, thinking about their own websites and how we can incorporate continuous optimization.
⁓ There ⁓ are tools out there that support this, testing tools. I think about tools like Optimizely or Adobe Target for the enterprise segment or VWO for the mid-market or lower market segments. ⁓ These are tools that we sometimes, at CoFrame, sit on top of ⁓ and we're the ones creating those tests. Really the important thing to keep in mind here is to enable continuous optimization.
It's a hungry engine. It's a very powerful engine, but it's very hungry. It requires a lot of shoveling coal in. In this case, the coal are variations. ⁓ So you have to make a lot of variations, and then those have to be based on what you're learning from the tests. And so the way that you can do this at home is by incorporating these models into how you're testing. So you're using Claude or using OpenAI, know, Chachiputti to create the variations, whether they be copy or.
or code or even images sometimes and things like that. It's not easy ⁓ to set all this up, but once you do, it's very beneficial.
KK Anderson (15:38)
Really cool. So let's go into, sorry, Mark, you go ahead. No, no. Okay. Yeah. I'm anxious to get into the OpenAI partnership. Okay. So you recently partnered with OpenAI. Congratulations. That's exciting. And you're developing what you call a first of its kind UI code, code generation. So break that down for us and tell us what this breakthrough actually means for business leaders.
Mark Petruzzi (15:39)
That sucks.
No, no kick us off. Bring us to topic topic two.
Josh (15:44)
I
Yeah, let's jump in.
Thank you.
Right, so one of the big challenges with models today is that they are very good at coding, but they're not as good at kind of like visually grounded coding. What I mean by that is when you look at a user interface, having the eye of a designer to figure out how is this code going to actually look when it's put into a user interface, especially when it comes to a pre-existing brand. So we're all operating in the world of
existing websites with existing assets and styles, and you have to stay on style and on brand and adhere to those guidelines. And a big shortcoming of today's models has been ⁓ how do we make sure that the code that we're writing when we create these variations from a layout or a user experience perspective are in adherence with these guidelines. So that was the technical challenge. ⁓ The way that we approached solving it
was we created a large data set of a bunch of different websites out there. And we trained a model to be specifically good at creating new sections of websites that were aligned with the existing brand of the existing websites. And so ⁓ our initial approach was to basically remove one of the sections and then create a text
prompt that describes what that section was in as much detail as possible, ⁓ and then use a model to create a new section that was based on that prompt. So you're basically masking out one section of the site, saying to a model, hey, look at this existing site with everything except for that section, and then create me that new section that I'm describing verbally to you. And then that's the task, and it actually creates that new section, and it's basically graded.
on how well it adheres to what the original section actually looked like. So that's the task that we use to train it. It's very similar to actually how transformers are trained in predicting the next token. But a little bit more of advanced predicting the next section instead of maybe the next token. And so that was the way that we set up the problem.
Mark Petruzzi (18:15)
Mm.
Josh (18:30)
And we were able to achieve state-of-the-art results with visually grounded cogeneration with this. And so what that means for businesses is that now, at least ones that work with CoFrame, but others maybe that would use a similar approach to train their own UI cogeneration models, ⁓ now you can have a model that really knows your brand, your visual identity, and can then create new experiences, new sections, new pages entirely, potentially. ⁓
based on what it knows about your brand. It's almost like having a designer and a front-end engineer paired up, but they've been on your team for 10 years and they know your brand really, really well.
KK Anderson (19:10)
That's really cool.
Mark Petruzzi (19:12)
So that's very cool. Josh, tell us, I guess the biggest thing that's jumping out on me is a little more of the playbook here. How did you convince OpenAI to work with you on the fine tuning of this GPT-4.0 vision? Knowing how fast they're growing, how distracted they are as an organization. So tell us a little bit about how you made that all work.
a little bit of the why. Like why are they so excited with your vision of brand consistent UI generation versus other things that may be popping around out there in the marketplace.
Josh (19:53)
Yeah, I think it comes down to just it being a really interesting problem. So in one that is interesting from a couple of different angles, one is of course the technical challenge. You know, this is a new domain ⁓ relatively, you know, like visual plus code. And so that's a really interesting combination of those two. But then there's also, it's interesting from a business standpoint, right? Like being able to create visually grounded user interfaces. That's like a really great.
⁓ thing to do. It's really core piece of this movement now, which we're seeing in like, you know, consumer products and even enterprise products that are creating, you know, code for new experiences. I think companies like Lovable and Replit and Bolt, like the ones that are just taking off right now, then ⁓ Claude and ⁓ Claude code and cursor, and there's a bunch of others too. And so
I think that that's why they were all so interested in this is because it's kind of a key piece of that equation, the front end or the user experience piece of this. How we ended up working with them is another story. had actually, I've been friends with several folks at OpenAI for a long time. ⁓ to school with a bunch of them who had joined early on and
I was at the time, I probably should have gone and done an internship there as well. I was off doing my own things usually, but it would have been probably pretty fun to be part of that early team. Anyway, I've been catching up with a bunch of them. And we were actually in the process of creating our own foundation model for UI cogeneration, right? Because this is pretty unique problem. And what we'd started to see was that ⁓
Visual ⁓ Vision Language models or VLMs were actually showing a lot of really strong promise in the open source world, ⁓ separate from the closed source models. ⁓ There was a model called Moon Dream that was out that was showing astonishingly powerful results despite it being very small. And so we were thinking, what if we made our own really strong coding LLM and then combine that with a strong vision component ⁓ using
you know, the technical term here is a clip model. And so that's that was like our initial thinking, which was if these closed source model companies like OpenAI are not going to release something that's really powerful here, ⁓ then why don't we do it ourselves? ⁓ The mistake there was, of course, thinking that they wouldn't release something as powerful. And so when I was catching up with with some of my friends there, ⁓
KK Anderson (22:39)
Yeah.
Josh (22:45)
I was mentioning this and they were like, hey, we're going to be doing something. And the moment I heard that, was like, okay, it's over. All this effort that we're putting in is going to be wasted. ⁓ You should not try to compete with these companies on that turf. ⁓ Luckily, they were, I think, impressed about how we were approaching it and asked to partner ⁓ on this. We had the data, they had the model and the team, and we kind of combined forces and did a really big sprint together for...
Mark Petruzzi (22:54)
All right.
Josh (23:14)
several weeks and got something out in time for the announcement. And then we announced it together.
KK Anderson (23:27)
That is really cool. Go ahead Mark.
Mark Petruzzi (23:31)
So, I think you have the next question, if you'd like.
KK Anderson (23:32)
So.
Okay. So now, so talk to me about kind of your go-to-market as it relates to, know, thinking about how you're going to go to market with this product. And obviously OpenAI Partnership gives you a competitive advantage. Like, you know, how defensible is it? You know, as other companies try to replicate this approach. you, do you have any competitors?
Josh (24:02)
Yeah, great question. The go-to-market approach has really been interesting because there's actually a number of different channels that we've been able to tap into. ⁓ And I know, Mark, you're actually an expert advisor at BCG, right? So you probably have seen a lot ⁓ of companies, a lot of SaaS companies approach BCG and say, hey, we'd like to work with you. BCG is actually a company that we're starting to work with as well. And that's been really beneficial.
Bain as well, we've done a little bit with. so partnerships through consulting firms have been interesting, especially on the enterprise level and looking at ways that we can bring value to these consulting firms, which are usually a little bit more, I would say hands off and sort of like advisory. But showing value quickly and getting outcomes quickly is something that is really beneficial for their model. And so that's how we've been able to partner with large consulting firms.
partnering with agencies as well, where maybe they don't have a conversion optimization practice. Maybe they're more in the paid media space or what have you, or SEO. And we can basically provide this as an added service to their offerings. ⁓ But most commonly, we're just going direct, right? We're finding people through conferences, word of mouth, customers will share and give references and stuff. ⁓ We'll have...
LinkedIn, know, campaign, not campaigns, but just like announcements of things and people will come in that way. ⁓ It's been, it's been a lot of just kind of like, you know, the momentum, keeping the momentum going and the energy in the space. think a lot of people are indeed excited about the AIs, you know, spin on it. And, the partnership with the OpenAI has actually been really helpful on the go-to-market front in terms of just like, guess, you know, credibility and they have that halo effect. so.
That's been awesome. wasn't our intention at all. It was really just let's build something cool with these folks, but ⁓ it's definitely been beneficial.
Mark Petruzzi (26:09)
that's cool, Josh. Yeah. And I've done some work with Vane over the years as well. And you're in the hands of great companies there. And it so fits into what their mission is versus what you all are doing at CoFrame as well. So let's talk about the technical challenges to orchestrating all this. What have you had to overcome in making AI generate on-brand visually consistent interfaces?
Josh (26:25)
Mm-hmm.
Mark Petruzzi (26:39)
mean, that's the key right there.
Josh (26:42)
Yeah, mean, that in itself is a technical challenge. I the models are, I have found, ⁓ they're very strong in writing code and kind of doing the quote unquote heads down work. They're actually less strong in the visual creativity side of it. They have less of a strong sense of sight than they have a sense of, I guess, logic, if you want to call it that. ⁓
That has posed a challenge, right? You know, our world is a lot about taste. It's a lot about making sure something fits in with the rest of the brand. ⁓ Conversion optimization is not simply, does this random experiment do better or not? It's about making sure that every experiment that we run is going to not only, you know, meet, but exceed the expectations of the people reviewing them on the customer side that we work with and making sure that they're actually impressed with the result before we even run the experiment.
And so that's a part of, you know, selling ourselves as Co-Frame actually. We'll go into sales calls and we will, before even working with someone, we'll show them some variations of their website. ⁓ That's kind of the beauty of the technology is that you're able to do that. Something that would otherwise take weeks of several people's time. ⁓ It's so easy for us that we can show them in a, just a qualifying call, which is kind of nice. ⁓ It's very helpful for sales.
But in order for models to be good at that, they have to be good at visuals. And most times, models off the shelves are just not good at seeing something and understanding how this is going to fit in naturally and natively with their brand. And so that was the core technical challenge here. Another challenge here ⁓ was gathering the data that was required for this. so luckily, ⁓ we were able to scrape the internet.
publicly available websites, ⁓ we basically, that's the nice part about this problem, which is that everything is basically out there for everyone to see. There's no concern ⁓ around data issues and stuff like that. But the challenge there was making a data set that actually reflected the problem that we're trying to solve, which was how do we get a model to more accurately create a new section of the site that is
aligned with the rest of website. So that was not an easy challenge to solve either. ⁓ but you luckily, when when the infrastructure is in place, you can kind of scale it up as much as you want to. And that becomes really, really powerful.
KK Anderson (29:26)
So.
Mark Petruzzi (29:26)
Great, all right, let's move, well,
we wanna move on to topic three, but I have, I keep kind of diving into one item that maybe we can do a quick sidebar in, at least in my own brain. So tell us a little more about what type of competitive advantage this creates, you know, and then how defensible is this for the companies, for your clients that jump in with you in this?
KK Anderson (29:47)
Yeah.
Josh (29:52)
Yeah, those are both great questions. we actually look at competitive advantage and defensibility a little bit in reverse. It's not what's competitive, what's like a defensible competitive advantage for a co-frame, but rather how are we creating that for our customers?
Mark Petruzzi (30:09)
Yes, and that's exactly where my question is focused on. Like how do you do that with your, how do you help your clients achieve that?
Josh (30:16)
Exactly. so that's the key piece. And I think flipping on that on its head creates this insane amount of alignment between us and our customers. ⁓ But basically with each customer, what we're doing is we're building essentially a very embedded, intelligent team member or team overall ⁓ that is working for them 24 seven. It's a team of, know, front end engineers and designers and PMs and marketers.
That's what CoFrame basically is. That is, over time as we're running more and more tests, we're learning more and more about what works for their business. And for the most part, what we've seen is, you know, what you learn about a specific business is pretty bespoke to that business. That's not the only reason why we completely silo the business data and so on. We do that also because it's just the right thing to do. you know, businesses would be concerned if we weren't doing that. But...
We certainly have found that within a given business, as this quote unquote team, embedded team gets more smart, that business has an increasing competitive advantage against other businesses that aren't 24 seven continuously optimizing and learning. You know, it's, it's, it's, it's one of those advantages. That's the beauty of compounding. We talked about earlier, but the cool part about compounding is that, you know, it might start slow, but then suddenly you have a runaway advantage. There's no way to catch up. And so.
That's what we are basically providing for our customers.
Mark Petruzzi (31:49)
Excellent, KK, will you bring us over to topic three?
KK Anderson (31:50)
Absolutely.
Yep. So let's talk about autonomous growth ⁓ and how AI agents are replacing manual optimization. And so you said that you've demonstrated 352 % improvement in click through rates with enterprise clients, which is incredible. ⁓ How should growth teams think about integrating
AI agents into their existing marketing stack.
Josh (32:21)
that's a fantastic
question. And that actually comes down to your own business. I actually have ⁓ a lot of thoughts here. marketing agents, agents in general, it's a very general topic, right? And so it's important to of like approach that from the right mindset and not just say like, here's, I'm not going to be prescriptive here because ultimately every business is different. Every workflow is different. First thing to kind of note is that
We are approaching this new world where basically ⁓ you have to make sure it impacts the bottom line. That's really, really important. ⁓ You have to make sure that it gives you an edge in whatever you're implementing agent technology into. And there is starting to become this organizational mandate as a result. The CEO of Shopify, Tobi Ludke, he recently put out a memo in which he was saying, before asking for resources, people have to demonstrate
why they can't do what they're trying to do ⁓ using AI. And so it's starting to become a mandate. So it's good that you're asking this question. The tactics that I have for evaluating whether or not to incorporate an agent into your workflow, how and when to use agents, ⁓ the first thing is to take a systems thinking mindset ⁓ to the workflow you're trying to automate here. ⁓ This is a little bit fuzzy, but
or fluffy, should say. you know, systems thinking is actually really, it's a really relevant thing to have. kind of, it's like, to me, it's this combination of AI ⁓ knowledge and knowing what the capabilities of these models are and these agents are. And then your domain knowledge, which in this case is marketing knowledge. And then finally, a general growth mindset. I know that's a fluffy term, but like having kind of that impetus to think about new ways of applying this stuff.
And so the systems thinking that comes into this is the combination of those three things. More tactically for marketing, there are several areas where this is applicable. Paid media is a big one. Looking at how ads can be, you can create different variations of ads and then optimize them over time. And there are companies that are doing this and providing ⁓ tooling for this. Coframe is going to be doing this at some point soon. ⁓ There's companies like Icon and ⁓
There's a bunch of them. Google and Meta are going to be doing themselves. There's conversion optimization for a website. That's of course what Coframe is doing. There's lifecycle marketing, emails, email campaign optimization. There's SEO, there's content operations, a bunch of stuff. The next tactic I have here is getting to know the capabilities of your AI colleagues. Trying to figure out where do we use certain models in what situations? For instance,
Claude is really great at general coding. So is now the 03 in the sort of like GPT series from OpenAI. Gemini from Google is great at long context. And so you can put a bunch of different collateral into it and have it analyze it. So my call to action for the listeners here would be get to know the different models out there and try to figure out
where are their limitations, where are their strengths. This is going to be evolving over time. So this is going to be really important to stay on top of. The next tactic I would have here is to prototype and see what's possible. You want to try to get your hands dirty and figure out what's going to show promise. ⁓ then just basically what you learn from prototyping, you should be able to have a sense for, OK, how do we apply this to my workflow and the agents that I'm trying to build?
At that point, it's really important to make sure that you're building things with what are called evals, making sure things are, you ⁓ you're actually measuring quantitatively the outcomes that you want your agent to be able to deliver. ⁓ And at the point where you do that, you know, you're in a really good place. You have a closed loop and you're ⁓ able to, with some level of reliability, automate a workflow, which is an amazing thing. And that's, think, like I was saying earlier,
the most promising part of the new AI world we're entering into.
Mark Petruzzi (36:48)
Excellent. All right, let's move us to the final topic here, and that's the future of marketing technology and growth engineering and how does this all come together for us? And the biggest question is, and this is gonna seem like I'm asking you to predict something that's 50 years in the future, because in this space, looking ahead even just 18 or 24 months is so difficult and the changes are happening daily, minute,
minute by minute. But let's say we look at 18, 24 months, how do you see the convergence of AI and growth marketing evolving? And what should marketing leaders be one, for, and secondly, driving within their companies to allow that information and those new capabilities to be executed on as quickly as possible for them?
Josh (37:42)
That's a fantastic area. You know, it's a, it's a really great question. I have this, this article, um, which I think would be a good one to check out. If you're looking to dig deeper into this topic for the listeners at home, it's on this newsletter called every by Dan shipper. Uh, it's called the new science of growth marketing. And in it, what I try to talk through is this correlation between growth marketing and finance. Interestingly enough.
So in finance, what we saw was previously, before the advent of computers and so on, people traded manually, traded stocks, bonds, other assets. And it was a very manual, very tedious process. It still is manual to an extent. People are certainly doing discretionary trading on their own. But what transformed finance in the 1980s and 90s and 2000s
⁓ was the emergence of quant finance. So quant trading and using quantitative methods, computers, ⁓ to do algorithmic trading of these assets. And what made that possible for finance was that the problem that you're trying to solve, it wasn't a simple problem, but it had a simpler output. It was either, you know, predict a price or
Mark Petruzzi (38:46)
Yeah.
Josh (39:10)
even simply a buy or sell indicator of a stock. Now the output for marketing is not something as simple as a price or a buy or sell. It is copy, is code, it is assets. ⁓ And until recently, that wasn't really possible to do with the models that we had at our disposal. But with the advent of generative AI, that is the native modality that AI is operating in.
And so what we're starting to see, and we will, I think, increasingly see, at least what I say in the article, is that just as quant finance ⁓ revolutionized the financial industry, we are going to start to see this idea of quant experimentation and quant quantitative growth methods revolutionize growth and marketing, where you'll have these strategies that are autonomous to an extent, ⁓ executed by machines.
being able to work 24 seven on your behalf and test things autonomously and make changes and personalize at a way deeper level than that was previously possible. And that's a really powerful thing because, know, to date it's been extremely tedious and time consuming to run these experiments. You require, you know, like an engineer and a designer and marketer and so on, just to get a single AB test out the door, can take weeks. What if it take minutes or hours?
you know, or less, you know, so ⁓ we're starting to enter a world where continuous optimization is then possible, all the new superhuman capabilities that we talked about earlier. ⁓ And that is what I like to think of as quant growth or quant experimentation. And I go into a bit more detail into specific strategies ⁓ in the article, very high level strategies, know, nothing super profound, but ⁓ I think it's gonna be very, very
impactful for the growth industry and for companies that use it.
KK Anderson (41:14)
So last question before we move on to our rapid fire ⁓ section. So what's going to be the highest value work for humans? Right?
Josh (41:27)
Yeah, so it's a great question.
the answer to this one, at least I kind of just borrow from this similarly borrow from what we've seen in the quant industry, right? Where instead of actually performing the trades, humans are responsible for coming up with the strategies, you know, ⁓ we have now more professionals in the financial industry than we ever have. ⁓
KK Anderson (41:46)
Okay.
Josh (41:53)
despite a lot of those manual tasks being taken by machines. ⁓ Similarly with the invention of the steam engine and electricity. ⁓ Well, the steam engine took a lot of the manual effort out of ⁓ transportation. ⁓ But we have now seen an entire new industry built up that has given a lot of jobs in performing higher level work that involves steam engines or is involved in the creation of steam engines.
There are a lot of such examples. So I think what we're going to end up seeing as applied to this industry is growth people, marketers, engineers alike, are going to have to focus less on the tedious work of writing the code, coming up with the specific assets, coming up with the specific analysis of the data and the results, and rather guiding the strategy for their businesses.
I also think we're going to see an explosion of businesses overall. We're going to, people are going to be way more empowered to be solopreneurs and, you know, small businesses and entrepreneurs and do more and offer more with less. And so we're going to start to see just like more opportunities where growth can be applied. I do think that growth teams at large, large companies might shrink because a lot of the grunt work is going to be taken on by AI. But ⁓
KK Anderson (43:00)
Yeah.
Josh (43:20)
there's going to be a lot more opportunity for many, many more types of businesses out there to use these advanced growth tactics as well.
Mark Petruzzi (43:28)
And I just want to point out that that last question that KK presented was submitted by a bot, by a robot, because they want to know how they can leverage us now into working for them instead of the other way around. it's already happening. That was not from a human. OK, we're going to go rapid fire over the last three minutes or so here. This has been great, Josh. So thank you.
KK Anderson (43:42)
Hahaha! ⁓
Josh (43:43)
already happening. wow. That's amazing.
Mark Petruzzi (43:57)
So Josh, what was the first product or service you were ever responsible for selling?
Josh (44:05)
wow. That's a great question. It has nothing to do with my current industry, but it has all to do with who I am, which is music. I had a little jazz group back in high school and we made some albums and reinvested the proceeds back into the band and made recordings, more recordings and did little touring. So yeah, that was my first exposure to business.
KK Anderson (44:14)
That's right.
Mark Petruzzi (44:28)
So everything you touch is successful, right? Is that how it works, Justin?
Josh (44:31)
Definitely not, but
I appreciate that.
KK Anderson (44:37)
Okay,
who's your favorite CEO to follow?
Josh (44:41)
man,
that's a, it's a great question. I'd have to say Eric Schmidt, ⁓ he's like next level, maybe the best operator in the whole world, ⁓ just insane story. And, ⁓ I've had the opportunity to learn some things from him directly and work with him through a couple of different things. And so, but I think he's awesome.
Mark Petruzzi (45:02)
Very cool. Okay, a tool other than CoFrame that every growth leader should be using.
Josh (45:11)
man. Well, I think it's really, really important to have a really solid sense of your data for sure. So some type of, some type of analytics platform, Amplitude's a really great one. know, Adobe analytics, ⁓ post-hoc, et cetera. I think that, you know, the more you have a sense of what's happening in your business, the more you're going to be able to improve it and measure, measure those outcomes.
KK Anderson (45:39)
Question, do you have to have one of those to be able to use CoFrame?
Josh (45:42)
No, no, we actually come with full data, you know, analytics included, but we do plug into these, these systems.
KK Anderson (45:49)
Okay, now for my favorite question. Advice that you would give your 21 year old self.
Josh (45:55)
that's a good
one. man. ⁓ Okay. So I think that what I would probably say is like, like basically hype is not the same as substance. You know, I've, I've, I've seen a lot of, a lot of hype in my career so far. ⁓ And, you know, easy come easy go. Like things will just like, there'll be bubbles, they'll collapse and there'll be things that
that what you want to be doing is basically subtracting the noise to try to find the true signal and keep this level of groundedness to try to figure out what is truly valuable here and think from first principles what is going to be durable and sustain over time. And so I'd really urge myself, I guess, to really focus on the substance.
Mark Petruzzi (46:45)
Oh, I love that, Josh. And as a, know, as a reoccurring, you know, individual who's launched different companies, been CEO of multiple companies at this point, I love to have you, I love to hear you thinking about that in those terms. Because, you know, I have a lot of friends who have done the same from, you know, different company to the next and...
they start doing it a few times at the CEO level and they start to think that's their job is to always drive that hype through the market. And even sometimes within their own organization, they think they need to have credibly positive energy in their business at all times. And that's not life as we know it. And ⁓ having that realistic approach is really impressive.
Josh, thank you so much for taking the time with us. This was great and we're excited as we go along to learn more about Code Frame and see how we can leverage that into some of the work we do with clients.
Josh (47:54)
It's been a pleasure, Mark. Thanks so much for having me, Mark and KK. Really good to see you both.
KK Anderson (47:54)
Yeah, absolutely.
Thank you, Josh.
Mark Petruzzi (48:01)
Perfect, all right. Thanks, all. Thanks, Josh. Thanks, KK. Cheers.
See Privacy Policy at https://art19.com/privacy and California Privacy Notice at https://art19.com/privacy#do-not-sell-my-info.