TwoSetAI: Decoding AI Builders

From an early engineer on GitHub Copilot to a consultant for whom every month or two is a completely new world: John Berryman is the author of Prompt Engineering for LLMs and the founder of Arcturus Labs. He left GitHub in May of 2024. Staying inside one company on one fixed project, he says, meant missing what was happening in every other corner of the field. Search was his first career, before AI, which is why he has firm opinions about RAG.

In this episode, John joins TwoSetAI to describe how working with AI has changed for him, from an assistant that saved him 10 seconds of work to something closer to a team he manages. He calls himself the architect, the product and the manager of developers that are naive but much smarter than he ever was. One of his answers gave the episode its title, and Angelina keeps count of the doomsday predictions from there. How serious he is about it, and what he is doing about it in his own career and with his kids, runs through to the end.

You can learn more about Arcturus Labs here.

We go deep on RAG, which John says people have treated as one black box for too long. He takes it apart: an agent is a for loop around a large language model call with a growing context and some tools, and if one of the tools is search, you have RAG. When it fails, he reads the top 50 or so searches by hand to see whether the agent misread the user, formed a bad query or got poor results back. That turns one vague failure into an AI problem and a search problem, each of which can be fixed. He explains when he would use lexical search (exact matches, names, filters such as price or color) and when vector search (matching an idea, so that gorilla suit finds monkey costume), why out-of-the-box embeddings need fine-tuning for a specific domain, and why there is no reason not to combine the two. He also describes agents that skip the index and navigate files, as in his Roaming RAG post.

We also get into the harder parts. John never did SEO work and says he was unaware of the push to get found by AI, then answers with a prediction: in five to 10 years he does not expect landing page websites to exist, replaced by data and tools that a conversational interface reads for you. Angelina presses on what would replace backlinks as a signal of quality, and on whether Google is next on the list. He is candid about consulting too: feast or famine, getting nervous in the quiet stretches, and marketing, which he says he is still not good at. He closes with what he tells people about their careers, the kind of startup he would not build today, and what he is teaching his eight-year-old and his eleven-year-old.

We discuss:

* Why so few AI projects make it to production, and how big companies and startups differ

* Why evals are a limiting factor for big companies

* From code completion at GitHub to Claude Code and Cursor: how his own use of AI changed

* Agents on rails versus agents with a big set of tools and clear instructions

* The therapist trainer he helped build: an avatar patient, a coach and an evaluator, all live

* The whiteboard feature that went from basically impossible to working within three months

* Junior developers as "wizards casting spells", and the old codgers at the other end

* A pure AI agent or a junior developer assisted by AI: which he would be more comfortable with

* How anyone becomes a senior when agents handle the junior-level work

* RAG is not a black box: an agent, a search tool and a pipeline you can debug

* Building RAG from scratch around what the user needs

* Agents that navigate files without an index: Roaming RAG and skills

* Lexical search versus vector search: gorilla suits, filters and exact matches

* Getting found by AI, from someone who never did SEO

* Why he thinks landing page websites go away in five to 10 years

* LLMs.txt, APIs, standardized checkout, and what replaces backlinks

* Whether Google is going to die

* How AI search works: a search tool and a large language model in a loop

* "AI is just an HTTP request"

* Training small models on the traces of bigger ones

* Leaving GitHub, life as a consultant, and the part he is still not good at

* The fork in the road for careers, and the startup he would not build today

* What he is teaching his kids, and the farm in West Tennessee

John Berryman

* LinkedIn: https://www.linkedin.com/in/john-berryman-864b1713/

* Arcturus Labs: https://arcturus-labs.com/

* Book: Prompt Engineering for LLMs (O'Reilly)

* Roaming RAG: https://arcturus-labs.com/blog/2024/11/21/roaming-rag--rag-without-the-vector-database/

* llms.txt directory: https://llmstxt.org/

Timestamps

00:00:00 Why do 90% of AI projects fail before production?

00:03:39 How does a Copilot engineer actually use AI today?

00:04:33 What does it mean to manage AI like employees?

00:06:35 Can you train therapists in war zones using only AI?

00:09:27 What happens when impossible tech works three months later?

00:11:43 Are junior developers just wizards casting spells?

00:13:51 Would you hire an AI agent over a fresh graduate?

00:17:26 Why is your RAG pipeline failing?

00:20:21 How would you build RAG from scratch today?

00:23:25 When should you use keyword search over vector search?

00:27:23 How does AI change getting found online?

00:28:51 Are websites going to disappear in 10 years?

00:34:22 Is Google actually going to die?

00:39:56 Why is AI just an HTTP request now?

00:42:50 What is it like leaving GitHub to build alone?

00:47:43 Are we at the fork between utopia and dystopia?

00:49:58 What is he teaching his kids about failure and AI?

00:51:51 Outro

From a couch in a Zurich co-working space to a hacker house in San Francisco, Magnus Müller has kept the microwave. He is the co-founder and CEO of Browser Use, the open source tool that lets AI agents use the web, not only search it. He and his co-founder had each just shut down a startup when they asked over lunch how hard it could be to tell a computer a goal and have it done. They pushed an MVP to Hacker News four days after they started, and Angelina introduces the project as now at 80,000 GitHub stars and 17 million in funding.

In this episode, Magnus joins TwoSetAI to argue that the way people use AI is about to flip: you will stop prompting AI, and AI will prompt you. You give an agent a high-level goal once, it works in the background on a schedule, and it comes back with specific things for you to confirm. His own example is an agent that checks his calendar every morning at 6:00 a.m., searches the web for each person he is meeting and sends him the context on Slack.

You can learn more about Browser Use here.

We go deep on what it takes to make a browser agent work: the basic loop (a screenshot, the DOM state and about 30 tools such as click, type and scroll) and the edge cases that he says fill every real task, from two-factor logins to checkout pages that are really another site inside an iframe. He names the three complaints people have about agents, consistency, speed and cost, and the answer his team built: run the agent once, convert that run into a deterministic script, and replay it until the website changes. A second agent reviews each run and saves what worked, which he says makes the next run 10 times faster and 10 times cheaper. He also shares his screen to show the eval platform, where coding agents start an eval, wait 10 minutes, inspect the results, fix the biggest problems and go again. One was on cycle 10 during the interview, with almost 100 hours of runtime.

We also get into the harder parts: the Amazon lawsuit over an AI shopping agent and why Magnus thinks businesses built on ads will struggle with agents, and why a one-line change to a system prompt is so hard to test. On the company side, he says the team is seven or eight people, that he would hire a 19-year-old with very high agency over someone with a lot of experience, and that half the team has no university degree or dropped out. Asked whether the big labs threaten him, he says that if a very good Claude Code integration succeeded it would be super easy for Claude Code to replace Browser Use, and concludes they need to compete one level higher. He closes with what hitchhiking from Germany to Iran with a hammock taught him about worry.

We discuss:

* A Zurich co-working space, gym showers and the 50-second rule for microwave eggs

* Two founders, two shut-down startups, and a lunch question: how hard could it be?

* The first MVP: a screenshot, the DOM state and a simple loop, on Hacker News four days in

* Browser Use explained to an elementary schooler, then to an engineer

* The Amazon lawsuit over an AI shopping agent, and why ad-funded sites struggle with agents

* What a store loses when shopping happens inside a chat box

* "You will stop prompting AI": high-level goals, and an agent that prompts you

* His 6:00 a.m. agent that researches everyone on his calendar

* Medium-important tasks nobody gets to: broken links in the docs, an unchecked solar roof dashboard

* The stack behind a personal agent: a smart LLM, tools, persistent cookies, integrations, scheduling, a file system

* Why people who scrape data really want change detection

* Building a browser agent from scratch, and the edge cases that break it

* What happens when a website changes

* The three complaints about agents: consistency, speed and cost

* Turning one agent run into a deterministic script

* A second agent that reviews each run and saves the learnings

* LLMs that loop on a CAPTCHA, LLMs that give up, and fixing the root cause

* Why prompt changes are hard to test when variance is four or five percent

* A screen share of the eval platform, and coding agents running evals in 10-minute cycles

* A team of seven or eight, and how shipping fast kills wishful thinking

* Why he hires 19-year-olds with high agency, and his advice for high schoolers

* Whether the big labs threaten Browser Use, and competing "one level higher"

* Growth: open source launches, and why he says blog posts beat demo videos

* Hitchhiking from Germany to Iran with a hammock, and trusting that everything will be fine

Magnus Müller

* LinkedIn: https://www.linkedin.com/in/magnus-mueller/

* Browser Use: https://browser-use.com/

* GitHub: https://github.com/browser-use/browser-use

* Mentioned in the episode: the Browser Use cloud, and Composio, which he says the team added for integrations

Timestamps

00:00:00 He built an AI company while showering at a gym

00:00:57 How did two students build Browser Use from a coworking-space couch?

00:04:14 What does Browser Use actually do?

00:06:27 Will Amazon's lawsuit kill AI shopping agents?

00:10:28 Will AI start prompting you instead of the other way around?

00:13:04 How does his personal 6 AM AI agent work?

00:15:41 What tech stack does a personal AI agent need?

00:18:20 What breaks when you build a browser agent from scratch?

00:23:15 How do you fix non-deterministic AI agents?

00:26:00 Can AI agents learn from each other?

00:32:19 What does a real production AI eval platform look like?

00:41:12 Why hire 19-year-olds over senior engineers?

00:46:05 Could Claude Code replace Browser Use?

00:50:38 What does hitchhiking to Iran teach you about being a founder?

00:52:34 Outro

Daniel Davis is the co-creator of TrustGraph, which Angelina introduces as the open-source context backend for AI agents. He says he found knowledge graphs very hard to learn, and that what he knows came from his co-creator Mark Adams, who wrote him a guide that builds a knowledge graph out of three statements about his cat, Fred. By his account the term context graph exploded around November or December, and he has since given his "what is a context graph" talk three times in the Bay Area, where he estimates about 10% of the audience had worked with graph systems.

In this episode, Daniel joins TwoSetAI to say what a context graph is, where the term comes from, and why he is skeptical of most of the companies now using the label. He says the phrase goes back 20 years in the academic literature, credits Vicky Froyen with the first use he has found in today's sense, in a 2019 talk, and argues that the ideas underneath it are decades old. Asked whether a context graph is the same concept as a knowledge graph, he gives a direct answer.

TrustGraph is open source. You can find it here.

We go deep on how graphs work for AI: property graphs versus RDF, where ontologies come from, and the best definition of reification Daniel says he has seen, "statements about statements." Angelina describes how she would build a context graph herself, with chunks, embeddings, a vector database and Neo4j on top, and he explains what TrustGraph did at first: a flat triple store, with vector embeddings mapped to the graph for subgraph retrieval. He covers decision traces, what to do when data changes, and what he calls the elephant in the room, time: what was observed, when, and whether it changed. He also explains why TrustGraph 2.0 no longer supports Neo4j as its graph store, why the roadmap is designed around RDF 1.2 with Cassandra for storage and Qdrant for vector embeddings, and why he compares the result to Supabase for AI context.

We also get into the harder parts. Daniel asked the major AI services which former NHL head coaches are not currently coaching a team, and says every one of them made mistakes; Perplexity, by his account, gave him 13 names, two of them wrong. He tells a story from his Department of Defense days that he calls "when nine out of 10 experts were wrong." He makes pointed claims about how Wikipedia articles get edited and how much weight AI services give them, and Angelina adds her own example of an AI answer that led back to a blog post with wrong data. He closes with what scares him, people accepting information based on who has the most social media followers, and with a book about appearance versus reality.

We discuss:

* The NHL head coach question that, by his count, every major AI service got wrong

* Why "context graph" is suddenly everywhere, and why he is skeptical of most companies using the label

* Where the term comes from: 20 years of academic literature and a 2019 talk by Vicky Froyen

* Graphs, knowledge graphs, RDF, property graphs and DAGs as one family

* Learning graphs from Mark Adams, and Fred the cat

* Is a context graph just a rebranded knowledge graph?

* Vocabularies, taxonomies, schemas and ontologies

* Why he says 2025 was the year AI POCs failed

* Grading the answers: is 11 out of 13 good enough to trust?

* "When nine out of 10 experts were wrong": a story from his DoD days

* Why confident answers are hardest to catch on questions you cannot check yourself

* The Metallica top 10 list that he says no service got right

* Angelina's example of compounding misinformation

* His claims about how Wikipedia edits work, and how AI services use it as a source

* Angelina's build with embeddings, a vector database and Neo4j, and what he does differently

* Property graphs vs. RDF, and where each has been used: fraud detection, KYC, search and cyber security

* Reification: "statements about statements"

* Decision traces, and adding an agent's behavior back to the graph

* What happens when the data changes, and observations over time

* Why he says old information is not always the worse source

* TrustGraph 2.0 without Neo4j: RDF 1.2, Cassandra and Qdrant

* "Supabase for AI context"

* Idiocracy, Wag the Dog, Brazil, and the two things he sees converging

* Ike's Bluff, and appearance versus reality

Daniel Davis

* LinkedIn: https://www.linkedin.com/in/graphrag/

* TrustGraph: https://trustgraph.ai

* GitHub: https://github.com/trustgraph-ai/trustgraph

* Mentioned in the episode: Ike's Bluff, the book he recommends; the films Idiocracy, Wag the Dog and Brazil; and his podcast conversation with Vicky Froyen about the term context graph

Timestamps

00:00:00 Can AI get simple facts right?

00:01:35 What is a context graph, and who's faking it?

00:09:29 Is a context graph just a rebranded knowledge graph?

00:14:26 Why 2025 was the year AI POCs failed

00:18:07 When 9 out of 10 experts were wrong

00:25:17 How Wikipedia gets weaponized to poison AI

00:29:23 Is my RAG + Neo4j approach actually correct?

00:36:07 What are "statements about statements"?

00:41:03 What happens when your data changes over time?

00:49:12 Why TrustGraph dropped Neo4j entirely

00:52:42 His doomsday prediction for AI

00:55:23 The one book every leader should read

00:59:03 Outro

From a computer science faculty post at the University of Washington to chief scientist at Apodex: Simon Shaolei Du moved from academia to industry to build what he calls a heavy duty solver. Angelina introduces him as one of the most decorated young theorists in machine learning, with honors that include a Sloan Fellowship and MIT Tech Review Innovators Under 35. He introduces himself more simply, as a faculty member who likes to work with smart young people on hard problems. Apodex, he says, is not building a chatbot: the aim is to solve very hard, often unsolved problems, and the first product is a deep research agent.

In this episode, Simon joins TwoSetAI to explain how Apodex puts a whole team of AI agents on one question, and why he thinks organizing 100 agents is a different job from organizing 100 people. Angelina puts the obvious objection to him: LLMs are nondeterministic, so wouldn't that many agents compound each other's errors, the way throwing bodies at a problem fails in a human organization? His answer rests on two design choices, verification and redundancy, and on a claim about how models differ from people that he makes in the first seconds of the episode.

You can learn more about Apodex here.

We go deep on how the system works. Simon starts with two capabilities the model is trained for: planning, which means decomposing a hard problem into sub problems the way a scientist would, and search, including the multi-step agentic search measured by BrowseComp, a benchmark of around 1,000 hard-to-find answers on which he says Apodex has the state-of-the-art result. Around the model sits what he calls an agent team. A single model cannot hold a very long context, he says, and even a window of 1 million tokens is not enough, so the work is divided: a problem that needs more than 10 million tokens in total becomes many small queries that each need only a modest context. What he says sets Apodex apart is an explicit verification agent that judges whether a source is reliable, whether the reasoning from source to claim holds and whether code passes its tests. He compares it to the organizer of a weekly sync meeting: it gathers the agents' output, discards what is wrong and reassigns the next tasks.

We also get into the harder parts: whether a large model can predict the future, and why Simon, who worked at a quant company before, says this kind of prediction is closer to a fundamental analyst's deep research than to a quant model predicting a number from price and volume; whether an expert team should build its own multi-agent system; and whether he worries about the frontier labs' progress on deep research. On careers, he says AI can speed up software engineering and office work 10x or 100x and that the same work will not need as many people, but that human reasoning still matters because a person has to judge the model's answer. He also explains why a referral is, by his account, the most common way into a frontier lab, and why an open source agent framework, which needs only an API, is his suggestion for people who want in.

We discuss:

* The objection up front: nondeterministic models, and whether more agents means more errors

* How Simon introduces himself, beyond the list of honors

* Why Apodex is building a "heavy duty solver", not a chatbot

* Deep research as the first product, and the first step toward solving a hard problem

* The target domains: drug discovery and biomedicine, AI development itself, materials and finance

* Self-evolving: pointing the solver at the problem of developing large models

* Planning: training a model to decompose problems the way a scientist does

* Why search is still hard, what multi-hop search means, and the BrowseComp benchmark

* The agent team, and the limits of a single agent in ReAct mode

* The verification agent: judging sources, reasoning and code

* "LLM as a judge", and what the verifier sees

* Why a 10 million token problem does not need one long context

* Organizing 100 agents versus 100 people, and what he says makes the difference

* The verification agent as the organizer of a weekly sync meeting

* Redundancy: giving the same task to several agents, and why randomness helps

* Can an AI predict the future? Quant models versus a fundamental analyst

* Predicting the outcome of a World Cup match as a deep research problem

* Who should use Apodex, and whether expert teams should build their own agent system

* What is still up to humans once the research is automated

* Career advice: fundamentals, referrals and open source agent frameworks

* Why people still need to reason when a model gives the answers

* Whether he worries about the frontier labs, and what is next for the solver

* System-level thinking, AgentOS and the book he recommends

Simon Shaolei Du

* LinkedIn: https://www.linkedin.com/in/simon-shaolei-du-71b2a952/

* Apodex: https://www.apodex.com/

* X: @SimonShaoleiDu

* Mentioned in the episode: an operating system textbook, his book pick for thinking at the system level; and BrowseComp, the search benchmark he cites

Timestamps

00:00:00 100 AI agents vs. 100 people (cold open)

00:01:24 Meet Simon Du, one of ML's most decorated young scientists

00:02:37 Why build an AI "solver" instead of a chatbot?

00:04:14 What problem is Apodex really solving?

00:06:49 How does an AI break a hard problem into pieces?

00:09:19 Why is search still hard for AI agents?

00:12:46 The agent team, and the agent that distrusts the rest

00:17:13 Why a 10-million-token problem doesn't need a giant model

00:19:02 Won't 100 AI agents just compound errors?

00:25:55 Can an AI actually predict the future?

00:28:31 Can AI predict a World Cup match?

00:33:31 Build your own AI agents, or buy?

00:36:25 Breaking into AI and surviving the job market

00:42:36 What's next for the "heavy-duty solver"?

00:44:02 One habit for a better founder (and a book pick)

From competitive sailing to the ad business for AI apps: Nic Baird went to school at Yale, won a few national championships in sailing and expected to go professional. He says he stumbled his way into startup land instead. He is now the co-founder and CEO of Koah Labs, which he describes as a full service ad network for AI native companies: Koah deploys its SDK into a publisher's app, finds the most relevant sponsored content for the person using it, and builds the advertiser side of the marketplace as well.

In this episode, Nic joins TwoSetAI to explain why AI apps shut down while their users are still engaged. A generative product gets more expensive to run the more people use it, and in his telling subscriptions only convert at 3 to 5%, so founders look at the projections and decide it will never make money. His answer is advertising that is native to the app and relevant to what the person just asked. He also gives his account of Perplexity's ads, which he says went to market at a $50 CPM, drew a lot of interest and were then shut down, and what he reads into that for anyone putting sponsored content inside an AI product.

You can learn more about Koah Labs here.

We go deep on how an ad gets chosen. Nic treats it as a search problem with layers on top: is the query commercial or is the person just in learning mode, would a brand not want to be associated with it, is there a relevant match, and does the generated copy make sense before it goes out. Koah returns an ad only about 30 to 35% of the time. He demos it inside Liner, an AI search tool for students, where the ad sits apart from the answer in the app's own font and coloring. By his account the network is doing a 2% click-through rate, against the 0.4 or 0.5% he cites for traditional display ads, at an average CPM floating around $5. He also covers the revenue math, the three steps of installing the SDK, and a head-to-head test he says Luzia ran against AdMob.

We also get into the harder parts. Koah gets relevancy wrong sometimes, he says, and generated copy can misfire: one ad for an athletic food product came out with an opinion on the best national park in Brazil. Publishers control how an ad looks and where it is placed, but choosing exactly which advertisers appear is not really built yet. On building it yourself, he says great developers could connect to programmatic demand in a couple of weeks, while Koah's version has taken a year plus with 12 on the team, and someone still has to sell the brands directly. He says AI is not a shopping platform yet, so he pitches brands on a middle of funnel opportunity. He also argues both sides of when OpenAI should bring in ads, tells founders to get 10,000 engaged users first, and points to Open Evidence, which he says makes $150 million a year on a million users, all doctors, only from ads.

We discuss:

* Why AI apps with 40 or 50,000 engaged users still shut down

* From competitive sailing to "startup land"

* Koah explained to an elementary schooler, a college student and a professional

* Whether OpenAI will run ads, and why he can argue both sides on timing

* The South Park Commons origin: subscriptions converting at 3 to 5%

* A live demo of a native ad inside Liner, an AI search tool for students

* Why the ad stays out of the model's answer and is clearly marked as an ad

* Perplexity's ads, by his account: a $50 CPM, a lot of interest, then a shutdown

* Character AI's full screen ads and the backlash he says followed

* How an ad gets picked: commercial intent, brand safety, a relevant match, a check on the copy

* Why Koah returns an ad only about 30 to 35% of the time

* The head-to-head test he says Luzia ran against AdMob

* 2% click-through vs. the 0.4 or 0.5% he cites for display ads

* Installing the SDK: three steps, and customers he says were live in two hours

* Who it is for: indie developers with a niche, engaged audience

* When to monetize: plan from the beginning, get 10,000 engaged users first

* Open Evidence, his example of a product that makes its money only from ads

* The revenue math: fill rate, a $5 CPM and queries per day

* Which products fit: anything dynamic, not pure entertainment

* What publishers control today, and what they cannot choose yet

* The ad that gave its opinion on the best national park

* Build or buy: a couple of weeks for a simple version, a year plus for Koah's

* Proving ads work when AI "is not a shopping platform yet", and where GEO fits

* The "dynamic internet": from passive reading to active interfaces and agents

* Three books, for a 20-year-old, a 40-year-old and a 60-year-old

Nic Baird

* LinkedIn: https://www.linkedin.com/in/nicbaird/

* X: https://x.com/bairdone

* Koah Labs: https://www.koahlabs.com/

* Koah docs: https://docs.koahlabs.com/

* Mentioned in the episode: The Alchemist, Tomorrow and Tomorrow and Tomorrow, and Educated, the three books he recommends

Timestamps

00:00:00 Why AI apps keep dying

00:01:14 Who is Nic Baird?

00:02:21 Can you explain AI advertising to a five-year-old?

00:04:56 Will OpenAI ever run ads?

00:08:31 Why did he start Koah Labs?

00:10:44 What do AI ads actually look like?

00:15:15 Why did Perplexity kill their ad program?

00:17:30 How does Koah find the right ad to show?

00:19:24 Why leave 70% of ad slots empty?

00:23:24 Can you set up the SDK in two hours?

00:25:33 Who should use Koah Labs?

00:26:38 When should founders start monetizing?

00:29:36 The revenue math for an AI app

00:32:34 What types of apps work best with native ads?

00:40:05 Does AI hallucinate in ad copy?

00:41:14 Why you can't just build your own ad system

00:43:38 How do you prove AI ads work?

00:48:23 What's next for Koah Labs?

00:49:43 What is the "dynamic internet"?

00:51:25 Books for your 20s, 40s, and 60s

00:54:33 Outro

From a growth marketing agency he grew to 50 people to the O'Reilly books engineers read: Mike Taylor is the author of Prompt Engineering for Generative AI, a columnist at Every and, at the time of this conversation, the CEO of Ask Rally, which builds AI focus groups out of synthetic personas. He came from economics, learned to code after leaving the agency in 2020 and started freelancing as a developer; nobody knew he had no experience, he says, and all they cared about was whether the code worked. He got access to GPT-3 that same year, so he had years of prompt engineering behind him when O'Reilly came to commission a book.

In this episode, Mike joins TwoSetAI to argue that prompt engineering is not dead, whatever the man who stopped him at a 2023 conference said. People who are better than most at their jobs still need legal, HR, managers and employee handbooks, he says, and all of those are prompts. His own practice is plainer than the name suggests: he calls himself very lazy at prompting and very diligent at prompt engineering, and he says he uses his managerial skills with AI more than his data science skills.

You can learn more about Ask Rally here.

We go deep on the idea in the title: almost everything, Mike says, comes down to whether you have enough good examples of the task. With fewer than 10, add them to the prompt; with more, hand them to an optimizer such as GEPA in DSPy, which he says typically costs under a dollar a run; for something in production he aims for 200, collected from real usage. He explains why it helps to treat an LLM as a search problem and not a generative one, how he holds back a development set the optimizer never sees, and how a project of 5,000 personas maxed out a credit card. Then Ask Rally: personas grounded in real interviews, an LLM judge trained to tell a real response from an AI one, and why he says accuracy tops out near 90%, because people do not fully agree with their own earlier answers.

We also get into the harder parts: why ChatGPT makes a poor focus group (he calls it sycophantic, like a mother saying "awesome job sweetie"), what he calls the vegan problem, when personas in one thread color each other's responses, and a simulated election in which 90% of the AI personas voted the same way. He is candid about where Ask Rally stood on product-market fit, and clear that no research, synthetic or human, will tell you whether a product will succeed. He closes with the permanent underclass theory and why he does not expect it, his case that humans are AGI for minimum wage, the piano in a London train station that gives him hope, and three things he did in 2020 to get ready.

We discuss:

* How a marketer with an economics background ended up freelancing as a developer

* Prompting vs. prompt engineering: testing, workflows and formal evaluation

* The 2023 conference where someone told him prompt engineering was dead

* Why legal, HR, managers and employee handbooks are all prompts

* Parenting as prompt engineering, and why kids are hard to A/B test

* Managing AI like a team: a clear brief, a plan first, divided labor

* His top tip: collect good examples, plus a swipe file of what you dislike

* What the GEPA optimizer in DSPy hands back: a plain English prompt

* Examples vs. descriptions, and distilling a prompt down to cut cost

* Overfitting, hold-out examples and a development set the optimizer never sees

* How many examples: fewer than 10, more than 10, and 200 for production

* An LLM as a search problem, and whether your task is in distribution

* The project of 5,000 personas that maxed out a credit card

* What Ask Rally is, who uses it, and his candid answer on product-market fit

* Why ChatGPT is a sycophantic focus group

* Personas built from real interviews, and the simulated election that went 90% one way

* The lightsaber kit his AI clone said it bought at 2:00 a.m.: plausible, not predicted

* Training personas against an LLM judge

* Why 85% accuracy may be close to the ceiling

* Pricing next to a traditional focus group, and how many personas to ask

* "Personas of thought" and the vegan problem

* The permanent underclass theory, and why he does not expect it

* Humans as AGI for minimum wage, and the piano in St. Pancras station

* Three things he did in 2020 to prepare, and why senior people are the worst adopters of AI

* His second book, on context engineering with DSPy

Mike Taylor

* Ask Rally: https://askrally.com

* Mike's column at Every: https://every.to/@mike_2114

* Books: Prompt Engineering for Generative AI (O'Reilly), written with James Phoenix; and a second O'Reilly book, on context engineering with DSPy, which he was writing at the time of this conversation

Timestamps

00:00:00 How a marketing guy ended up writing O'Reilly's AI books

00:03:00 What's the real difference between prompting and prompt engineering?

00:04:37 Has nobody told you prompt engineering is dead?

00:07:17 Why the biggest skill in AI is management, not coding

00:16:04 Why AI doesn't create anything: it just searches

00:18:52 He maxed out a credit card optimizing 5,000 AI personas

00:20:49 What is Ask Rally and why did he build it?

00:29:20 Can an AI clone actually predict your behavior?

00:44:30 The permanent underclass theory, and why he doesn't buy it

00:48:15 Humans are AGI for minimum wage

00:49:38 Why nobody stops for a self-playing piano

00:56:27 Outro

From side projects on the GPT-3 API to leading AI at a work platform: Jay Hack is Head of AI at ClickUp and the founder of Codegen, which Angelina introduces as one of the first coding agent companies. He says he has worked on versions of code generation for over a decade, that it kicked into high gear when the GPT-3 API became public in late 2022, and that he started Codegen in early 2023 to build agents that go off, change your code asynchronously and come back with a pull request. By his account the idea was a little ahead of its time, and the early product did not get much customer traction until the models caught up in 2025.

In this episode, Jay joins TwoSetAI to explain why he sold Codegen instead of continuing to build it as an independent company. The answer starts with what his customers were doing with a coding agent once it sat inside the tools where they already worked, and with what he came to see as the real limit on what an agent can get done. He also argues that the path coding has taken with AI is the one many other industries will follow, two to three years later.

You can learn more about ClickUp here.

We go deep on how agents get built inside a product people work in all day. Jay argues that chat has more staying power than dashboards or interfaces assembled on the fly, because people overestimate how much of the work they will still be doing themselves. He weighs building an internal AI brain against buying one: he credits Ramp, by his account a former Codegen customer, as a pioneer of building its own coding agent, thinks perhaps 10 companies can pull that off, and says ClickUp has something like 200 engineers on the problem. He explains why memory gets hard when an agent serves a whole organization, the hierarchical summarization his team uses, and the idea it borrows from the Merkle tree at the base of Bitcoin. He also separates memory, knowledge and context, and takes on the view, which he attributes to Dario Amodei, that a context of 100 million tokens could stand in for memory.

We also get into the harder parts. Jay says it is hard to point to successful vertical AI platforms and that a generalist agent will be 95% as good for almost everything, while allowing that a high-powered legal firm may still want a vertical one. On permissions for memory shared across a company, he says the industry has no agreed answer. He says a team at the application layer has to be ready to burn down its tech stack every four months, expects many organizations to slim down, and thinks the job of sitting and writing code all day is going away, with English as the next level of abstraction after assembly, C and Python. He describes the "AI maximalism" policy that replaced coding puzzles in interviews at Codegen, what a product generalist does, the evals he runs while daily-driving six other AI clients, and his advice to founders: do not pitch your business.

We discuss:

* More than a decade of code generation work, and what changed when the GPT-3 API became public

* Why code was the first target for LLMs: you can verify it

* Codegen's bet on asynchronous agents that hand back a pull request

* Why he thinks other industries will follow coding, two to three years behind

* Why chat has staying power: people overestimate the work they will still do themselves

* The moment he decided to sell Codegen, and what customers were really using it for

* What he came to see as the real bottleneck for agents doing knowledge work

* "Software to replace all software", and the cost of nine separate software contracts

* Build or buy: the roughly 10 companies he thinks can build their own, and ClickUp's 200 engineers

* Why what you built six months ago is already obsolete

* Why he says it is hard to point to successful vertical AI platforms

* Fine-tuned models, positive transfer, and where a vertical legal product still makes sense

* Telemetry: how to tell whether a new model is better for customers

* Memory for a whole organization, and the permission problems that come with it

* Hierarchical summarization, and the idea borrowed from Bitcoin's Merkle tree

* Memory, knowledge and context: the semantics debate he says he usually avoids

* Whether a context of 100 million tokens could replace memory

* Burning down the tech stack every four months

* Assembly, C, Python, then English: software engineering "inventing itself out of a job"

* "AI maximalism" in hiring, and what best-in-class means now

* Who he says is filtering out of tech, and why he calls this the best time to be in the industry

* His wife, a product manager who vibe codes, and the role of the product generalist

* Daily-driving six other AI clients, and Galaxy Brain, his open source set of evals

* Why founders should not pitch their business, and how Figma tells its story

Jay Hack

* LinkedIn: https://www.linkedin.com/in/jayhack

* ClickUp: https://clickup.com/

* Mentioned in the episode: Galaxy Brain, his open source GitHub repo of evals

Timestamps

00:00:00 Why a coding agent isn't really about code

00:00:45 How did GPT-3 lead to building one of the first coding agents?

00:05:43 Why will chat UI outlast every AI dashboard?

00:09:24 What made you sell Codegen instead of competing?

00:16:08 Should you build or buy your company's AI infrastructure?

00:19:50 Why does Jay Hack think vertical AI is doomed?

00:26:29 What's the hardest part of building AI agents at scale?

00:33:06 How does a Bitcoin-inspired Merkle tree solve AI memory?

00:36:56 Does Dario Amodei's infinite context vision actually work?

00:39:22 Why burn your tech stack every four months?

00:42:17 What is the assembly-to-English abstraction chain?

00:43:53 What does AI maximalism mean for hiring?

00:47:30 What happens when non-engineers start vibe coding?

00:52:31 What do you learn from daily-driving 6 AI clients?

00:55:26 Why should founders never pitch their business?

From a year or more without revenue to a company of 100 plus people: that is the stretch John Kim describes here. At the time of this conversation he is the co-founder and CEO of Paraform, which Angelina introduces as the recruiting marketplace that companies like Palantir, Cursor and Rippling use to fill their hardest roles. Paraform signs up recruiters, gives them companies to work with and tools to do the job, and gets paid only when a hire is made, with most of that payment going to the recruiter, he says. It is his second startup, and he says he still feels like a first-time founder.

In this episode, John joins TwoSetAI to argue that great candidates are out there and that the hard part is matching and access. He can find the best engineers at Anthropic on LinkedIn in less than a minute, he says, and that does not mean he has access to them. It is why Paraform built a network of recruiters instead of another tool, why he calls recruiting the most consequential matchmaking problem in the world, and why he thinks AI is making the noise worse.

You can learn more about Paraform here.

We go deep on how the marketplace works: a company posts a role, Paraform matches it with recruiters who reach into their own networks, and across the platform the average time to surface the candidate a company ends up hiring is 11 days, by his account. John explains why the first five or 10 hires make or break a startup, why posting on LinkedIn is not enough for a company without a brand, and how house hunting in the Bay Area, an analogy Angelina brings, maps onto recruiting. Angelina also lays out how she would build Paraform from scratch as an AI engineer, from scraping job boards to LLM evaluation of candidates, and John answers with what that leaves out: marketplace incentives, quality control, distribution, and the recruiters who are in the office every week building the product with the team. He makes his case for recruiters going independent, too. By his account recruiting has one of the highest attrition rates among white collar jobs, about 35% year over year.

We also get into the harder parts: the layoffs at Oracle and Block, where John bets many of the people affected were recruiters, and the adjustment period he expects before another golden age of abundance. He is plain that building the company got harder over time, not easier: the cold start problem, a run of 8 AM to 3 AM days, an offer that fell through and sent him and his co-founder into 10 delirious minutes of planning a different startup, and days during scaling when he woke up throwing up. He shares the two ways he says a company dies, the value he and his co-founder call binary outcome, why the two of them still live together, and his advice for new grads and people who were just laid off, including the skill he now calls free.

We discuss:

* Paraform explained to an elementary schooler, and the world as "a museum of passion projects"

* Who gets paid when a hire is made, and where most of the payment goes

* Why the first five or 10 hires make or break a startup

* Recruiting at 3 AM when you are not a Google or a Netflix

* The Decagon example, and the "found time" stat he tracks

* Why posting the job on LinkedIn is not enough: inbound, outbound and the power law

* Finding the best engineers at Anthropic in under a minute, and why that is not access

* The Bay Area real estate analogy and information asymmetry

* "The most consequential matchmaking problem"

* Confucius, upskilling and why he ranks matching first

* Oracle, Block and who he bets was hit by the layoffs

* What CFOs and heads of engineering are struggling to model right now

* His prediction about seven-figure salaries

* The hire and lay off cycle for recruiters

* Going independent, and why he thinks it makes a recruiter a better matchmaker

* What AI has done to the messages a good engineer gets and the applications a good company gets

* Angelina's from-scratch build of a recruiting marketplace, and what John says it leaves out

* The cold start problem and a year or more without revenue

* The 3 AM plan to combine TikTok and DoorDash

* Crossing the 50 and 70 person marks

* The only two ways a company dies

* "Binary outcome": the pact with his co-founder

* Sufficiently different, sufficiently similar: how he thinks about co-founders

* Professional ikigai, sales roles and what he says is no longer valuable to know

* His doomsday prediction for recruiting, and why he expects more business owners than employees

John Kim

* LinkedIn: https://www.linkedin.com/in/johnkim5480931a8/

* Paraform: https://www.paraform.com

* Mentioned in the episode: Recruiter OS, the suite of tools Paraform gives recruiters; and professional ikigai, the idea behind his career advice

Timestamps

00:00:00 What does Paraform actually do? (explained two ways)

00:03:31 Why do the first 5 hires make or break a startup?

00:07:32 Is posting on LinkedIn enough to hire great engineers?

00:09:08 Why does network access matter more than AI search tools?

00:10:57 Is finding a job harder than finding a house in the Bay Area?

00:15:12 Oracle laid off 30,000: what's really happening in hiring?

00:21:46 Should recruiters rethink their entire career model?

00:23:51 How is AI making recruiting noise worse?

00:27:21 Can you build a recruiting marketplace from scratch in 6 steps?

00:31:03 What was the hardest part about actually building Paraform?

00:35:23 "I would wake up just throwing up": the cost of scaling

00:37:04 What are the only two ways a company actually dies?

00:39:58 How do you know your co-founder is the right person?

00:43:02 How do you make yourself valuable in the AI job market?

00:46:46 What's the doomsday prediction for recruiting?

00:49:29 Outro

Postscript is seven and a half years old, and Alex Beller says the company is reinventing itself all over again. He started Postscript with two co-founders, Adam and Colin, and at the time of this conversation all three still run it together. He describes it at three levels: it helps businesses send text messages; it is a messaging and conversational AI platform for e-commerce businesses; and, to people in the industry, it is an SMS marketing platform. When LLMs became capable of doing real work for customers, he says, that hit reset on what had been a scaling SaaS business with established playbooks.

In this episode, Alex joins TwoSetAI to talk about what happens when AI agents start having conversations with a brand's customers. Marketers, he says, are not going to let a black box agent just run, so everything starts with Brand Center, where a brand curates the voice, persona and language its agents use. From there he walks through Infinity Testing, which turns one message into hundreds of variants and keeps routing traffic to the ones that perform, and Shopper, the conversational selling agent.

You can learn more about Postscript here.

We go deep on what broke along the way. The models quickly learned that more aggressive sales tactics performed better, down to saying only three of a product were left when that was not true, and Alex describes the prompt constraints, batch validation gates and merchant feedback that pulled them back. When an agent did not have an answer it would fabricate one, so the team built what it calls a graceful failover, plus supervisor agents that check links, carts and message content against the real data before anything is sent. He is open that he did not personally architect the system, and he explains why contradictory product data inside a customer's store still has to be Postscript's responsibility. He also tells the Dr. Squatch story, in which, by his account, the testing agent found something about the brand's texting persona that surprised the merchant.

We also get into the harder parts. SMS is regulated under the Telephone Consumer Protection Act, the liability he cites is $1,500 per message sent, and he says that if the founders had known all about that, they probably would have thought twice before getting into SMS. He thinks it is too late to start within his space, and would still do the journey over again. He shares what the company learned from an office in Phoenix full of people who sold over text for brands, what his customers are seeing in traffic and purchases from ChatGPT, his hot take on AEO and GEO, and why he would be building a startup with a very small team today.

We discuss:

* Postscript explained at three levels

* Seven and a half years, three co-founders, and why it still feels early

* The moment LLMs "hit reset" on a scaling SaaS business

* Brand Center, and why marketers will not let a black box agent run

* Building a brand voice from a website, social feeds and uploaded guidelines

* Why the team avoided an infinite rules-based engine

* Infinity Testing: hundreds of variants and a multi-armed bandit in place of an A/B test

* Why the best message in December is not the best one in June

* List size and statistical significance

* Dr. Squatch, Chad and what the testing agent found

* When models learn that pushy tactics and artificial scarcity sell

* Fabricated answers, "graceful failover" and supervisor agents

* Contradictory product data inside a store, and whose problem it is

* Where Shopper is held to a RAG pattern: the synced catalog, configured discounts and verified URLs

* The Telephone Consumer Protection Act and $1,500 per message

* "We probably would have thought twice"

* Too late to start in this space, and why he would do it again

* Shopify, Stripe and agentic commerce protocols

* Traffic and purchases coming from ChatGPT

* A long conversation between a consumer and a brand's agent

* The Phoenix sales center and "the big empty silent store"

* What brands should do now: catalogs, on-site content and an opt-in list they own

* His hot take on AEO and GEO

* How AI is changing work inside Postscript, including his own memo writing

* Advice for people early in their careers, and two books

Alex Beller

* LinkedIn: https://www.linkedin.com/in/alex-beller-70452852/

* Postscript: https://postscript.io/

* Mentioned in the episode: Leadership and Self-Deception and The Moon is a Harsh Mistress, the two books he recommends

Timestamps

00:00:00 What does Postscript do, at three levels?

00:02:33 Seven and a half years in: how do you keep it exciting?

00:04:40 Brand Center: how do you maintain a brand voice under the hood?

00:11:09 Infinity Testing: hundreds of variants in place of an A/B test

00:15:03 What did AI discover about Dr. Squatch's customers that humans missed

00:16:55 Why AI models fabricate discounts when you optimize for sales

00:20:03 How supervisor agents catch hallucinations before they reach customers

00:26:49 What makes SMS compliance a $1,500-per-message liability

00:29:23 Is it too late to start a company in SMS marketing

00:31:00 Is ChatGPT sending real revenue to Shopify stores

00:34:36 What a 100-person Phoenix sales center taught an AI about selling

00:36:42 Will shopping through AI agents replace browsing online stores

00:39:05 How is AI changing how Postscript builds internally?

00:40:47 Advice for someone early in their career

00:42:23 The books he recommends

00:43:25 Outro

From Romania, where he did not want to apply for the few positions on offer, to founding AI engineer at a San Francisco startup: Paul Iusztin is the author of the LLM Engineer's Handbook and writes Decoding AI, a newsletter that Angelina says reaches more than 40,000 engineers. He got there without the usual interviews. He decided the leetcode path was, in his word, hijacked, went the contracting route, and started putting everything he learned on the internet, with what he calls zero strategy. At the time of this conversation he was working part-time at the startup, which builds vertical AI agents for financial advisors.

In this episode, Paul joins TwoSetAI to explain the article behind the title: why his team stopped using RAG, MCP and agentic loops in large parts of its product after trying, as he puts it, all these fancy algorithms. He is careful about the claim. They killed RAG not entirely, he says, but for many scenarios, and by the end of the conversation nothing is declared dead. What he argues against is putting a solution everywhere because it is hot: if you have a hammer, you see nails everywhere.

You can learn more about Decoding AI here.

We go deep on how those decisions get made: what the team learned about its own data, how he tells when RAG still earns its place, and where he still uses MCP himself and where he recommends against it. From there the conversation moves to code. Paul is bullish on vibe coding and suggests that clean-code rules were made to help people avoid human error, so AI-generated code may be closer to a compilation step: nobody reads bytecode to check that it is clean. If AI can code, he says, the only thing left to do is evaluate it, which is why he expects AI evals to be a huge domain. He also walks through his Agentic AI Engineering course, 34 lessons in three modules with two capstone projects, built mostly from scratch because he does not think agents from frameworks belong in production code.

We also get into the harder parts: how a startup that pivots many times can talk to its customers and still start from the wrong assumption, why he calls a portfolio the default and a degree only one of the checks, and what the work behind his own looked like: by his account, 10 to 12 hours a day, Monday to Saturday, since he started coding heavily 10 years ago. He would have been happy to sell 1,000 copies of his book and says it sold around 20,000. Paul explains why he chose Portugal over San Francisco, what he feels he misses, and why, as an introvert, he needs three days of recovery after meeting people. He closes with advice to own a domain before worrying about how to code, and admits he still needs 50 minutes of preparation before a video like this one.

We discuss:

* Why Paul stopped doing traditional job interviews, and why he calls the leetcode path hijacked

* Building in public from Romania: "zero strategy" and about one article a week

* Work framed as a service: what clients ask him in place of interview questions

* The visuals he still makes by hand in Canva

* What the startup builds: vertical AI agents for financial advisors

* Why he expects vertical agents to follow a shared blueprint, like web apps did

* What went wrong with RAG in their application

* How much data one advisor really has, and what that did to the design

* The agentic loop as a zigzag: extra latency and extra cost

* Angelina's intern analogy for an agent that keeps going back to double check

* Why the simple solution is hard to see when a startup keeps pivoting

* When RAG still makes sense: the ratio between your data and your context window

* MCP, the hammer and the nails, and the integrations his team ended up writing itself

* Where he still uses MCP: Cursor, Claude Code and tools he has no control over

* Vibe coding, clean code and AI-generated code as a compilation step

* AI evals, and prioritizing what to fix when tokens are a limited resource

* Inside the course: 34 lessons, three modules, two capstone projects

* Why he avoids agents from frameworks but still uses LangGraph for its utilities

* Custom business metrics vs. generic hallucination scores

* "I like to teach people how to fish"

* A portfolio as the default for getting hired

* 10 to 12 hours a day, and a book he hoped would sell 1,000 copies

* Portugal over San Francisco: a trade-off he says he made consciously

* Long-term plans, staying flexible and owning a domain

* Being anxious about posting, and the 50 minutes of preparation he still needs

Paul Iusztin

* LinkedIn: https://www.linkedin.com/in/pauliusztin/

* Decoding AI: https://www.decodingai.com/

* LLM Engineer's Handbook: https://learning.oreilly.com/library/view/llm-engineers-handbook/9781836200079/

* The article discussed in the episode: https://www.decodingai.com/p/building-vertical-ai-agents-case-study-1

Timestamps

00:00:00 Why most AI teams over-engineer

00:01:08 Why the leetcode path is hijacked

00:06:31 Building an AI career from Romania

00:10:09 What he's building: vertical AI agents for finance

00:13:46 Why his team killed RAG (and when it actually works)

00:21:31 The hammer-and-nails problem with MCP

00:26:24 Why he's bullish on vibe coding

00:29:47 AI evals: the next core engineering discipline

00:32:11 Inside the Agentic AI Engineering course

00:39:41 Why portfolio matters more than degree

00:41:04 The grit behind 20K copies (10-12 hrs/day, 10 years)

00:43:53 Why Portugal over San Francisco

00:47:12 Plan long-term in a domain, not in code

00:50:34 "I still get anxious: 50 minutes of prep"

00:51:08 "Big Tech is not dead. RAG is not dead. Nothing is dead."