Sign up to save your podcasts
Or
Share The Right Answer Isn't Enough—with Karthik Narayan (Komodo Health)
Share to email
Share to Facebook
Share to X
Share to email
Share to Facebook
Share to X
Or
The Right Answer Isn't Enough—with Karthik Narayan (Komodo Health)
If an AI agent gives you the correct answer but took the wrong path to get there, can you actually trust it?
In this episode of Futureproof, Prakash Chandran sits down with Karthik Narayan, Director of Product Management at Komodo Health, where he leads Marmot, an enterprise AI product for life sciences. Marmot's promise is that life sciences companies no longer need to send their data to McKinsey and wait months for an answer—they can ask complex healthcare questions and get answers directly. The catch? The answers aren't binary. Together, Karthik and Prakash unpack why grading an agent on whether it got the right answer is only half the story, how Komodo uses parallel critique agents and friction detection to close the gap between AI confidence and analyst rigor, and what changes when AI makes product leaders more powerful than they've ever been.
Topics covered include:
- Why the path matters more than the answer: How an agent can arrive at the correct number through the wrong query, pass traditional evals, and then fail catastrophically on the next question—and why trajectory evals are the real measure of trustworthiness.
- Steering, not just answering: How Marmot uses research plans, follow-up questions, and full code transparency to give analysts maximum control over subjective healthcare methodology decisions.
- Friction detection over thumbs up/down: Why users rarely use explicit feedback mechanisms, how Komodo infers dissatisfaction from behavioral patterns, and how that drove a complete platform rewrite at the six-month mark.
- Build vs. buy when AI makes prototypes easy: Why a junior engineer's weekend demo isn't the same as a production system with fallback models, context compaction, token optimization, and continuous evaluation—and how to think about total cost of ownership.
Episode ID: 19252007-the-right-answer-isn-t-enough-with-karthik-narayan-komodo-health
Subscribe to Futureproof wherever you get your podcasts.
From Xano - The fastest way to create a production-ready backend for any app or agent. Xano unifies AI speed, code control, and visual clarity, so you never trade reliability for velocity. Sign up for free today.
View all episodes
By Prakash Chandran, CEO & Co-Founder of XanoIf an AI agent gives you the correct answer but took the wrong path to get there, can you actually trust it?
In this episode of Futureproof, Prakash Chandran sits down with Karthik Narayan, Director of Product Management at Komodo Health, where he leads Marmot, an enterprise AI product for life sciences. Marmot's promise is that life sciences companies no longer need to send their data to McKinsey and wait months for an answer—they can ask complex healthcare questions and get answers directly. The catch? The answers aren't binary. Together, Karthik and Prakash unpack why grading an agent on whether it got the right answer is only half the story, how Komodo uses parallel critique agents and friction detection to close the gap between AI confidence and analyst rigor, and what changes when AI makes product leaders more powerful than they've ever been.
Topics covered include:
- Why the path matters more than the answer: How an agent can arrive at the correct number through the wrong query, pass traditional evals, and then fail catastrophically on the next question—and why trajectory evals are the real measure of trustworthiness.
- Steering, not just answering: How Marmot uses research plans, follow-up questions, and full code transparency to give analysts maximum control over subjective healthcare methodology decisions.
- Friction detection over thumbs up/down: Why users rarely use explicit feedback mechanisms, how Komodo infers dissatisfaction from behavioral patterns, and how that drove a complete platform rewrite at the six-month mark.
- Build vs. buy when AI makes prototypes easy: Why a junior engineer's weekend demo isn't the same as a production system with fallback models, context compaction, token optimization, and continuous evaluation—and how to think about total cost of ownership.
Episode ID: 19252007-the-right-answer-isn-t-enough-with-karthik-narayan-komodo-health
Subscribe to Futureproof wherever you get your podcasts.
From Xano - The fastest way to create a production-ready backend for any app or agent. Xano unifies AI speed, code control, and visual clarity, so you never trade reliability for velocity. Sign up for free today.