Forward Deployed

(We know the audio quality isn't great on this one :( But the conversation is still well worth it!)

Last week I hosted a fireside chat on what it actually takes to build AI agents in the enterprise with Natalie Meurer (Head of Agent Eng, Sierra), Harsh Trivedi (founding engineer, Mercor), Juhi Parekh (GM, Turing), and Kevin Lynch (Senior FDE, Fin).

We get into why new models aren't always better (and why you can't just swap in the latest release and assume your agent improves), how the data-labeling/RL environment business might only have a couple years left, why real-time voice-to-voice models still aren't production-ready, how cheaper inference is still causing prices to go up, how baking in a constellation of models into enterprise agents is so important for reliability,

and much, much, more!

Chapters below

00:00 Intro

00:21 Meet the panel

01:57 What everyone's actually using agents for day to day

06:10 The reality of forward deployed work

09:28 What agents couldn't do a year ago that they can now

12:29 Why you have to tell agents what NOT to do

16:26 What a harness actually is

22:03 RL environments explained

28:46 Does the data-labeling and RL environment business even last?

37:02 Why benchmarks don't tell you what works in production

38:12 Agent engineering vs forward deployed engineering

41:38 Deploying into 100-year-old enterprise systems

44:25 Why AI adoption is an org problem, not a tech problem

45:36 Hiring for judgment when engineers aren't really coding anymore

48:16 Why agents are a new kind of software

50:46 The first 90 days of an enterprise deployment

53:20 Why compliance environments break normal testing

56:58 Layering AI on AI to get to 99% accuracy

01:00:56 New models aren't always better — the swap problem

01:02:35 Improving agents without waiting for a new model

01:06:36 Does agent performance secretly degrade over time?

01:09:40 Why one model is never enough: the constellation approach

01:11:14 Building resilience when inference providers go down

01:13:48 When fine-tuning actually makes sense

01:14:53 Why voice-to-voice still isn't production-ready

01:16:25 The cascaded pipeline that real voice agents use

01:21:45 Audience Q&A: managing change inside the enterprise

01:23:24 Why inference getting cheaper makes things more expensive

01:26:54 Charging for outcomes instead of conversations

01:30:19 What the real moat is when everyone uses the same models

01:37:04 Synthetic data and where the data wall actually is

01:38:50 Closing thoughts

Today's episode is with Vince Signori, Sales Director at LangChain and one of the first sales hires at HashiCorp, where he watched the company grow from a small startup all the way to an IPO and get acquired by IBM.

He sees the exact same shift happening now with AI agents that happened with cloud back then, except 50x faster. And he's got the numbers to back it up — LangChain is downloaded more than the OpenAI SDK, and 45% of the Fortune 500 are now paying customers.

We get into how companies like Toyota and Home Depot are actually using AI agents in production today, why enterprises are building their own private versions of ChatGPT to own their data, and why memory is becoming the most valuable asset in AI.

We also talk about what it actually takes to get an agent from prototype to production, why selling open source is the hardest sale in software, and how Vince went from 3 reps doing 20-hour days to running the number one sales region at one of the fastest growing companies in AI.

You don't wanna miss this one.

Chapters:

00:00 Intro

01:38 From HashiCorp to LangChain

03:20 Cloud wave vs AI agent wave

05:30 Open source vs enterprise

06:23 How LangChain's product stack evolved

07:37 Why agents are finally in production

08:57 What companies build with LangGraph

10:12 LangSmith and Engine explained

13:23 The Agent Development Lifecycle

16:36 Build vs buy on voice agents

19:48 Why owning your data and memory layer matters

22:12 How open source users become paying customers

26:49 Why LangChain hired forward deployed engineers

31:34 What go-to-market looked like with 3 reps

35:31 From 39 employees to hypergrowth

36:37 Transitioning away from founder-led sales

37:07 Why the CEO joined every early call

37:38 The sandwich sale strategy explained

39:28 Signals that an open source user is ready to buy

41:08 Why outbound controls the narrative in enterprise

43:33 Why in-person selling still wins

45:57 Building an internal GTM agent to scale

47:37 What the GTM agent actually does

50:16 Why AI moves 50x faster than cloud did

53:46 The vendor consolidation wave that's coming

56:14 How to win the platform standardization deal

59:41 Why staying model-agnostic beats the hyperscalers

01:01:23 How Vince onboards new reps today

01:04:19 The sales and engineering feedback loop

01:09:49 Signals an account is ready to expand

01:12:57 How to prove early value before full commitment

01:14:40 How enterprises actually measure agent ROI

01:16:27 Why automation is expanding beyond support

01:17:59 Which industries are adopting agents fastest

01:19:07 Healthcare agent use cases live today

01:21:17 AI agents in finance and payments

01:22:40 The Visa partnership

01:25:17 What it takes to scale a sales team right now

01:27:44 How the GTM agent is changing the SDR role

01:32:06 Why the human element in sales still matters

01:34:28 Platform deals vs point solutions

01:36:43 Vince's predictions on memory and consolidation

01:39:23 How Engine helps teams iterate on agents faster

01:43:19 Forward deployed engineers vs Engine

01:46:20 Where to find Vince and LangChain's open roles

Today's episode is with Shrivu Shankar, VP of AI Strategy at Abnormal AI - a $5B cybersecurity company. What makes this one unique is that Shrivu joined as an intern in 2021 and got promoted every single year until he reached VP, so he's basically watched AI go from a niche engineering tool to something that's reshaping entire companies from the inside.

We get into how AI is catching cyberattacks so sophisticated that even humans can't tell they're fake, how engineers at a $5B company have basically stopped writing code themselves, and what that means for everyone else on the team.

We also go deep on why context engineering is replacing prompt engineering as the real moat, how they used GPT-3 with zero safety guardrails to generate fake phishing attacks as training data, and what it actually takes to become an AI native company at 1,500 people.

One of the most technical and eye-opening conversations I've had. You don't wanna miss this one.

Chapters:

00:00 Intro

00:58 Who is Shrivu and what is Abnormal AI

01:39 Why cybersecurity and machine learning

03:13 Intern to VP in 4 years — how it actually happened

05:44 What Abnormal AI does and how it started

09:10 The vendor fraud attack so convincing the victim didn't believe it was real

10:45 What GPT-3 changed for cybersecurity

13:01 Using synthetic data to train models — and how they measured it

16:49 How a 1,500 person company actually adopts AI internally

19:50 How engineering, PM, and platform roles are changing right now

23:17 The biggest AI misconception Shrivu keeps hearing

27:35 What Shrivu's day actually looks like as VP of AI Strategy

28:53 Engineers stopped writing code. Here's what they do instead.

32:28 Why product teams are getting much smaller

34:31 Why context engineering beats prompt engineering

36:31 Spec-driven development and how Nora Tech Plan works

39:14 How to scale context engineering across an entire eng org

40:30 What the manager role looks like in the agent era

42:17 What skills actually matter for managers now

43:29 AI is making orgs flatter. Is that a good thing?

45:08 How the C-suite is getting closer to the work

46:41 What agents actually are and how tool calling works

48:05 How agents improved Abnormal's detection pipeline

50:56 The AI phishing coach — how it works and why it matters

53:30 The internal AI data analyst agent

56:13 Dozens of internal agents — the ones Shrivu is most proud of

57:15 Where agents fail (it's usually not the model)

58:52 What Shrivu would tell a CEO just starting with agents

01:00:19 Sending sensitive security data to LLMs — how they handle it

01:01:47 What becoming AI native actually means in practice

01:03:37 What most people still get wrong about AI in the enterprise

01:04:31 How to write documents with AI without it sounding like AI

01:06:40 Claude Code vs Codex — which one and why

01:09:27 How Shrivu stays ahead and his take on MCPs

01:11:33 How the team uses Claude Code skills

01:12:47 Using hooks for shift-left validation in large codebases

01:13:42 How to manage context in a massive monorepo

01:14:56 Building tool-agnostic rules across Claude, Cursor, and Code Rabbit

01:16:55 Why infra teams are becoming agent harness teams

01:17:57 Wrap up

Today's episode is with Supriya Gupta, ex-VP of Product at Intuit Credit Karma and former Product Lead on the Ads team at Meta when that business was scaling like crazy.

She brings a really unique perspective on everything happening with GenAI right now because she's seen it from the inside at two of the biggest companies in tech.

We get into how ads are slowly going to be generated on the spot, per user, where the image, copy and offer will all be unique to you specifically. We talk about why you can't just infinitely scale ad testing even though it's now theoretically possible, and why flooding the internet with AI content might actually be the worst thing you can do for your brand.

We also go deep on what actually happened inside Credit Karma when they started building with GenAI, including what moved the needle and what didn't, and what that means for designers, PMs, and content teams everywhere.

And at the end, Supriya shares why she walked away from her VP role to start her own company, and what she's building now.

You don't wanna miss this one.

🔗 Find Supriya on LinkedIn: https://www.linkedin.com/in/supriyag/

🌐 Company's website: https://www.helloeve.co/

⏱ Chapters
00:00 Intro

01:27 Predictive AI vs. Generative AI: what actually changed

02:59 The next gen of dynamic ads — why every user could soon get a unique ad

06:52 Will content agencies survive the AI era?

09:17 Why you can't just test a million ad variants (the stats problem)

12:55 AI slop, UGC backlash, and should AI content be labeled?

17:02 Supriya joins Credit Karma: building the Lightbox targeting platform

21:23 Building the Credit Karma financial assistant

24:35 Handling hallucinations in a finance app at scale

27:26 What happened to content designers when AI started writing copy

29:56 Why AI copy still needs human taste and judgment

31:01 PMs are prototyping now — what that means for design and eng

33:30 Will there be fewer PMs? (Probably not — here's why)

35:33 Why Supriya left Credit Karma to start her own company

37:23 The principles she built her startup around

39:20 The rise of the "super IC" — managers becoming AI-powered operators

41:54 Why most AI projects fail before they even start

45:33 Enterprises vs. startups: how they approach AI differently

48:13 What a successful AI deployment actually looks like

51:02 Will everyone need to upskill on AI? (Spoiler: maybe not)

52:41 Most execs are thinking about cost-cutting — the smarter ones aren't

53:49 The executive digital twin: Supriya's startup vision

56:51 Current product, roadmap, and what's shipping next

58:52 Where to find Supriya

AI agents are everywhere right now. But are they actually working inside real engineering teams?

At AngelList’s Founders Cafe, I sat down with founders of Cognition ($898M raised), Semgrep ($204M raised), Factory ($70M raised), and Composio ($29M raised) to talk about what agentic engineering looks like in practice.

There’s a lot of hype around AI coding tools, but the reality is more nuanced. Some teams are moving 10x faster, others are slowing down. A big part of it comes down to whether your codebase is actually “agent-ready” (linting, type systems, guardrails, etc).

We also went deep on security, which is one of the biggest gaps right now. As more non-developers start “vibe coding,” the risk surface grows fast. We talked about MCP access control, layered security, and why you can’t rely on models alone to generate secure code.

Enterprise teams are also dealing with the operational side of this shift. They have to manage cost, run evals, and help thousands of engineers use these systems well. Tools like PR review agents, model routing, and internal orchestration are quickly becoming part of the stack.

⏱ Chapters

00:00 Welcome and setup

00:41 Panel introductions

02:00 Windsurf acquisition

03:52 Do agents boost productivity?

04:16 Agent-ready codebases

05:55 Real-world enterprise wins

07:28 Agents building integrations

09:41 Security risks (vibe coding)

11:04 LLM security tools landscape

12:21 Defense-in-depth

15:16 MCP security pitfalls

18:41 MCP vs CLI

23:08 RL for secure code

27:05 Auto research missions

27:57 Training your own models

31:33 Distillation and IP decay

34:50 Hybrid systems

37:14 Side projects vs enterprise

40:16 Forward deployed engineering

40:58 Agent orchestration

43:08 Cost controls

43:49 Auto model routing

45:52 Guardrails

47:02 Legacy code risks

48:16 Model poisoning

49:35 What is a harness?

51:47 Why build your own

52:58 Continuous learning loops

56:30 Security workflows

57:37 Validation and meta engineering

1:00:32 Running evals in practice

1:03:36 Teams reshaped by agents

1:12:17 Should you study CS?

1:14:11 Enterprise adoption

1:18:23 Local to cloud journey

1:20:51 Agent economy and prompting

1:22:22 Spec vs plan

1:25:12 Closing and thanks

Summary:

In this conversation, I talked to Ashish Shubham (VP of Engineering), who's been at ThoughtSpot for 10 years, about AI agents in enterprise analytics. ThoughtSpot started as a search-based analytics company trying to make data accessible to regular business users. In 2019, they tried building natural language interfaces using BERT, but only hit about 50% accuracy. For a product where enterprise customers make billion-dollar decisions, that wasn't good enough. They shelved the project.

When ChatGPT came out, ThoughtSpot was ready. Ashish walked me through how they pivoted: they built a 25-30 person team, decided to use prompting instead of fine-tuning, and leveraged their existing semantic data modeling layer to get accuracy into the high 90s. We got into the technical evolution from monolithic systems to agent architectures with tools, how they went from manual human judges to using LLMs to evaluate their outputs, and how enterprise security requirements shaped what they built.

We also talked about how software engineering is changing. Ashish said 50-60% of his code is AI-generated now, and he thinks system design is becoming the critical skill, even for junior engineers. He had an interesting take on the "95% of AI deployments fail" stat too.

Chapters:

0:00 Intro and Ashish's journey to ThoughtSpot from GoDaddy

0:13 ThoughtSpot's mission to democratize data analytics for business users

1:26 Early search-based analytics before natural language processing

2:36 ThoughtSpot vs Tableau and the promise of self-service analytics

4:40 The analyst bottleneck problem and how ThoughtSpot aimed to solve it

5:49 Early technical challenges with in-memory databases and data migration

8:11 Semantic data models, joins, and creating abstraction layers for users

11:39 Who builds the data models and the role of analysts

12:22 Pre-LLM natural language processing using BERT and word2vec in 2018-2019

14:43 The accuracy problem and ambiguity in translating user queries

16:58 Trust challenges and why the early NLP product never became core

19:59 Competition with Tableau, Looker, and Power BI

22:44 How analyst roles changed with self-service analytics tools

25:30 The ChatGPT moment and pivoting to LLM-powered natural language

27:48 Early prompt engineering days and generating SQL with LLMs

31:09 Training vs prompting debate and why fine-tuning was eventually abandoned

34:28 Organizational changes and building the NLS team

37:16 Coaching systems for company-specific terminology vs training models

39:02 Evolution of evaluation methods from human judges to LLM-as-judge

43:23 Moving to LangFuse and GCP for agent infrastructure

46:29 How LLM context windows and capabilities evolved their product

50:07 From 30-column limits to agentic systems with 90%+ accuracy

52:52 RAG, column selection, and using proprietary data indexes

54:59 Multi-model support and enterprise data security concerns

59:14 How AI has changed Ashish's personal engineering workflow

1:02:42 Impact of AI on the broader engineering organization

1:04:15 Measuring AI productivity and the challenge of metrics

1:07:26 50-60% AI-generated code and the changing nature of coding

1:09:18 System design skills becoming more important than coding

1:13:00 Junior engineers doing senior-level work and interview changes

1:14:37 Customer conversations about Gen AI adoption across industries

1:17:26 The MIT report on 95% agent failures and why it misses the point

1:22:12 Agent architecture with LangGraph vs Google ADK and building internal agent platform

1:24:26 Where value lies in the next two years: tools, skills, and optimization

1:28:05 Startup opportunities in making AI accessible to non-technical users

1:29:26 Closing remarks

In this conversation, Tatu discusses the transformative impact of AI on game development, drawing from his extensive experience in the gaming industry. He highlights the shift from traditional game development processes to a more agile, AI-driven approach that allows for rapid prototyping and iteration. Tatu emphasizes the importance of organizational change and the need for leaders to embrace AI as a core part of their strategy. He also explores the evolving role of product managers, the challenges of user acquisition, and the future of marketing in a saturated gaming market. The discussion culminates in Tatu's vision for his new AI-native game studio, aiming to disrupt the industry by leveraging cutting-edge technology to create high-quality games at unprecedented speed.

Takeaways:

• AI is condensing the time and resources needed for game development.
• Organizational inertia can hinder the adoption of AI in large companies.
• The future of game development will require T-shaped professionals with diverse skills.
• AI will fundamentally change the economics of the gaming industry.
• Smaller companies can leverage AI to outmaneuver larger competitors.
• The role of product managers will evolve as AI takes over prioritization tasks.
• Marketing strategies will need to adapt to a more saturated market.
• User acquisition costs are expected to rise due to increased competition.
• Novelty may not be as valuable as familiarity in a saturated market.
• The future of entertainment will see a rise in fast, iterative game development.

Chapters:

00:00 The Evolution of Game Development with AI

03:07 From Web Design to Gaming: A Career Journey

05:50 The Impact of AI on Knowledge Work

09:07 The Changing Landscape of Game Development

11:53 Organizational Inertia and the Future of Gaming Companies

14:55 The Role of AI in Transforming Game Development

17:57 Navigating the Challenges of AI Adoption

21:08 The Future of Game Development Methodologies

23:46 The Role of Product Managers in an AI-Driven World

26:47 Marketing Strategies in the Gaming Industry

29:59 The Role of Publishers in Game Development

33:05 The Future of User Acquisition in Gaming

36:02 The Changing Economics of Game Development

38:56 The Future of Software Development

42:13 The Role of Novelty in Game Development

45:04 The Importance of Familiarity in a Saturated Market

48:12 The Future of Fast Entertainment

50:59 Leveraging Licensing for Success

54:02 The Journey from Rovio to AI Native Gaming

57:02 Building Tools for Rapid Game Development

59:57 The Vision for Future Games

01:03:04 AI Adoption in Organizations: A Leader's Perspective