So What About AI Agents

AI spent the last few years competing on intelligence. Now the next major AI race is about speed.

As AI shifts from simple chatbots to agents that reason, write code, call tools, search the web, and coordinate other agents, latency compounds. A task that requires dozens or hundreds of sequential model calls can quickly turn seconds into minutes — or even hours. That makes inference speed a fundamentally different problem in the agentic AI era.

In this episode of So What About AI Agents?, Philippe Trounev sits down with Vasanth Mohan of SambaNova Systems to unpack what actually makes AI faster — from model architecture and memory bandwidth to batching, specialized accelerators, and next-generation inference hardware.

Vasanth explains the two major stages of inference — prefill and decode — and why they create very different hardware bottlenecks. They discuss operator fusion, parallelization across chips, memory bandwidth, and why reducing latency often comes with a significant cost tradeoff.

They also explore why coding agents are one of the clearest use cases for premium inference, how AI providers may eventually offer commodity, premium, and ultra-fast tiers of tokens, and why businesses willing to pay for speed may gain a meaningful productivity advantage.

Vasanth also shares early performance figures for SambaNova's upcoming SN50 architecture, including benchmark results around 800 tokens per second on a large model, compared with roughly 300–400 tokens per second on GPUs in the cited comparison.

We also get into a slightly crazier question: are AI agents already helping engineers design the next generation of AI hardware? The answer is increasingly yes — although humans are still very much in the loop.

In this episode:

• Why AI agents make inference speed dramatically more important
• Why sequential agent workflows create a latency bottleneck
• Prefill vs. decode explained
• GPUs vs. specialized AI accelerators
• The relationship between speed, batching, throughput, and cost
• Why memory bandwidth matters for large AI models
• What SambaNova's SN40 and SN50 architectures are designed to solve
• 800-token-per-second AI inference
• Why coding agents benefit so much from faster models
• Commodity vs. premium vs. ultra-fast AI inference
• Whether faster AI becomes a competitive advantage
• How AI agents are already being used in hardware engineering
• Why AI infrastructure may become just as important as the models themselves

Chapters

00:00 — The AI race is shifting from intelligence to speed
00:39 — Why AI agents suddenly need faster inference
02:38 — What actually makes an AI model run faster?
05:30 — When does ultra-fast inference matter?
08:08 — Does model architecture determine inference speed?
10:05 — How do you optimize AI from model to hardware?
12:44 — How fast can AI inference actually get?
15:28 — The sequential latency problem with AI agents
16:39 — Where faster AI creates the most value
18:30 — Will premium AI inference become a competitive advantage?
20:11 — What does AI inference actually cost?
23:10 — How many tokens can AI hardware generate?
24:46 — Building private AI infrastructure
28:02 — How much faster can AI eventually become?
30:14 — Are AI agents already designing AI hardware?
32:50 — Why today's “fast” AI will eventually feel slow
33:34 — What happens next in AI infrastructure

AI attackers can now move at a speed and scale that traditional security operations were never designed to handle.In this episode of **So What About AI Agents**, Philippe Trounev sits down with **Neal Iyer, Director of Product Management for AI at Splunk Security**, to talk about what happens when autonomous AI agents become part of both the attack and defense side of cybersecurity.Neal breaks the new threat landscape into three forces: **scale, speed, and sophistication**. AI can give relatively unsophisticated attackers capabilities that previously required much more expertise, while autonomous attacks can move so quickly that a security team celebrating a five-minute detection time may already be too late.We get into:• How AI changes the economics of cyber attacks• Why cheap AI agents can dramatically increase the number and sophistication of attacks• Why traditional SOC response times may no longer be fast enough• How prompt injection creates a new attack surface for defensive AI agents• Why simply deploying more security agents isn't enough• Building a security “harness” around autonomous agents• Runtime monitoring, guardrails, observability, and human escalation• When security agents should act autonomously — and when a human needs to stay in the loop• Using business context to distinguish between compromising an intern's laptop and locking out a CEO• Why security tools need better integration for agentic operations• How organizations can start testing their agents against prompt injection and other attacks today• Why Splunk and Cisco are exploring purpose-built small language models for cybersecurity• The role of open-source, self-hosted, and multi-model AI strategies• Why AI security economics may force companies to rethink how they store and process security dataWe also get into the broader question of whether frontier AI companies can really replace specialized enterprise software — or whether impressive demos fall apart when customers need reliability, support, governance, and measurable outcomes.The bigger takeaway is that the future SOC probably isn't humans versus AI attackers.It's **agents fighting agents — with humans designing the systems, permissions, guardrails, and escalation paths that keep those agents under control.**Subscribe to **So What About AI Agents** for conversations with the people actually building and deploying AI agents in production.https://www.docsie.io

In this episode of **So What About AI Agents**, we break down **10 AI agents and workflows we’ve used to drive traffic, generate demand, nurture leads, and automate parts of our go-to-market operation**.This episode was recorded a little while ago, so my own setup has evolved since then. Today I use **fewer, more capable agents with much more advanced workflows**, rather than trying to automate everything with a huge number of separate agents. But the core ideas in this conversation are still useful if you're trying to figure out where agents can actually create leverage in marketing and GTM.We cover agents for:• LinkedIn intent detection and outbound• Lead nurturing based on actual customer behavior• Automated newsletters• Social media content planning and distribution• Paid-ad creative and optimization• Turning videos into blogs, clips, and other content• Programmatic SEO and AI-search visibility• Building glossary and topic-cluster content at scale• Google Search Console and analytics-driven optimization• Cross-channel content repurposingWe also talk about what **doesn’t** work particularly well: generic cold email, blindly automating social media, expensive always-on AI employees, bad AI-generated content, and agents that cost more to operate than the value they create.The bigger lesson is that you probably don’t need one magical “AI employee” doing everything.You need a small number of well-defined agents with clear jobs, good data, specific triggers, and enough human oversight to keep them from doing something stupid.If you're building an AI-native marketing or GTM stack, this episode gives you a practical starting point for deciding what is actually worth automating — and what probably isn't.Subscribe to **So What About AI Agents** for conversations about how companies are actually building, deploying, and operating AI agents in the real world.https://www.docsie.io

AI agents can build a demo in a day. But what happens when they touch a production system with years of technical debt, undocumented decisions, security requirements, and real customers?In this episode of So What About AI Agents, Philippe Trounev sits down with Krishna Kumar Sharma, former Head of Engineering for AI at Amazon and founder of Omokai, to talk about what agentic software development looks like outside of greenfield demos and AI hype.Krishna introduces his D3 framework — Discover, Define, Deliver — an approach to AI-assisted engineering based on the same principles used by mature software teams: understand the system, define the work, execute deliberately, and review everything.We get into:• Why greenfield AI coding demos don't represent enterprise software development• How AI-generated technical debt can compound at enormous speed• Why spawning 20, 50, or 100 agents usually isn't the answer• “Token maxing” versus ROI maxing• Using different AI models to review and challenge each other's work• Why cheaper and local models can often handle implementation after good planning• Claude, Codex, Gemini, GLM and local/edge models• Prompt caching and whether context-optimization tools actually save money• Security risks created by executives and teams vibe coding directly into production• Why human review still matters in agentic engineering• The D3 framework for AI-assisted brownfield development• Why boring, structured engineering practices become even more important with AIIn the second half, we move from software agents into the physical world.Krishna explains how Omokai is developing voice-driven command-and-control systems for robots and drones, including autonomous systems capable of operating with AI at the edge.We discuss:• Voice-controlled robots and drone swarms• Running small language models directly on robotic systems• Human-in-the-loop controls for safety-critical actions• Guardrails for autonomous machines• Robotics interfaces such as ROS2, MAVLink and PX4• Operating robots without continuous cloud connectivity• Sensor fusion, LiDAR, vision and GPS-independent navigation• Defense, security, inspection, disaster response and caregiving applications• What happens when AI agents move from software into the physical worldThe central argument of the conversation is simple:More agents aren't automatically better. More tokens aren't automatically better. The goal should be producing more value for every dollar, model call, and engineering hour you spend.Subscribe to So What About AI Agents for conversations with founders, researchers, engineers and operators actually building and deploying AI agents in the real world.https://www.docsie.io

Every CIO is asking the same question: What happens to IT when AI starts doing the work?In this episode of So What About AI Agents, Philippe Trounev sits down with Shayde Christian, SVP of Data & Analytics at Cloudera, to discuss how one of the world's largest enterprise data companies is using AI internally—not just to automate tasks, but to redesign how IT operates.Instead of focusing on AI hype, this conversation explores what actually happens inside a large enterprise when AI agents become part of daily operations.Topics include:• How Cloudera built internal AI agents for enterprise workflows• Why AI assistants and autonomous agents are fundamentally different• AI governance, testing, and production deployment• Building trustworthy enterprise AI systems• Why Cloudera reinvested AI productivity instead of laying off employees• How data teams are evolving into AI engineering teams• Measuring ROI from enterprise AI• The future role of IT departments• What CIOs and technology leaders should be preparing for todayIf you're responsible for enterprise AI, digital transformation, IT leadership, or building AI products, this episode offers practical lessons from real production deployments—not theory.GuestShayde ChristianSVP, Data & AnalyticsClouderaLinkedIn:https://www.linkedin.com/in/shaydechristian/Subscribe for weekly conversations with CTOs, CIOs, AI founders, enterprise architects, and technology leaders building production AI systems.Chapters00:00Introduction to AI Agents and Cloudera02:35The Role of AI Agents in Data Management05:46Challenges in Building AI Agents08:14AI Test Beds and Governance11:13Redesigning Roles in the Age of AI14:16The Future of AI in Business Workflows16:48AI Trust and Human Interaction19:39Agent TAM and Decision Intelligence22:28Governance and Accountability in AI25:21Concrete ROI Examples from AI Agents28:18The Future of IT and AI Integration30:50Final Thoughts and Advice for Leaders#AI #EnterpriseAI #Cloudera #CIO #ITLeadership #DataAnalytics #ArtificialIntelligence #AIAgents #Automation #digitaltransformation https://www.docsie.io

In this episode of So What About AI Agents, Philippe Trounev sits down with Geoff Mattson, CEO of SecureAuth, to explore one of the biggest unanswered questions in enterprise AI:How do you secure autonomous AI agents?As organizations rapidly deploy AI agents across customer service, operations, engineering, and internal workflows, traditional identity and security models are beginning to break down. Systems designed for human users were never built for autonomous software capable of making decisions, invoking tools, spawning sub-agents, and operating at machine speed.Geoff shares his perspective on:• Why AI agents fundamentally challenge traditional identity systems• The difference between authentication and authorization in agentic environments• Agent control planes, permissions, and governance• Prompt injection and agent hijacking risks• Multi-agent architectures and delegation chains• Why "vibe coding" executives are creating unexpected security concerns• The future of enterprise AI security and autonomous digital workers• What organizations should do before giving AI agents real authorityWhether you're building AI agents, deploying enterprise AI systems, or responsible for security and governance, this conversation explores the emerging challenges that come with autonomous software operating inside modern organizations.Guest: Geoff Mattson, CEO of SecureAuth

In this episode, Nick Sathe, CTO of Black Hawk Network, shares insights on the evolving landscape of agentic commerce, standards development, and the future of AI-driven e-commerce and payments. Discover how standards like AP2 are shaping secure, interoperable transactions and the implications for brands, consumers, and regulators.keywordsagentic commerce, AI standards, e-commerce, payments, gift cards, loyalty, chatbots, API standards, regulation, future of AIkey topicsThe evolution of agentic commerce and standards like AP2The role of loyalty, rewards, and gift cards in AI recommendationsChallenges and opportunities in standardizing discoverability and transactionsThe impact of regulation and market power on agentic payment systemsThe speed of technological change and the importance of guardrailsChapters00:00Introduction to Agentic Commerce03:38Evolving Consumer Behavior and Chatbots07:20The Role of Loyalty and Rewards in E-commerce11:52Standards and Discoverability in E-commerce16:30The Future of E-commerce Standards and Regulation19:09Implementing AP2 and Agentic Transactions21:50The Role of Agents in E-commerce25:06Consumer Trust and the Future of Recommendations29:40Consumer Influence in AI Interactions31:14Trust and Emotional Connection with Bots35:03Deterministic Systems vs. AI Variability40:42The Role of Agents in Business Processes46:25The Future of AI and Consumer Experience52:39Final Thoughts on Innovation and Responsibility

In this episode, Philippe Trounev sits down with Joe Locandro, EVP and CIO at Rimini Street, to discuss what actually happens when organizations begin deploying AI and autonomous agents at scale.The conversation explores the real-world challenges enterprise leaders face around governance, compliance, security, agent orchestration, and operational control — including why many AI initiatives struggle to move from proof of concept into production environments.Topics covered include:• AI governance and enterprise policy frameworks• Managing autonomous agents securely• Compliance and separation-of-duties challenges• The growing attack surface of agentic AI• Why vulnerabilities now evolve in minutes• Scaling AI across enterprise organizations• Low-code and no-code development trends• The future of software engineering and AI operationsJoe also shares practical recommendations for CIOs navigating enterprise AI adoption today.Guest: Joe LocandroEVP & CIO, Rimini StreetTimestamps:00:00 Introduction to AI in Enterprise IT02:02 Governance and Policy in AI Implementation06:07 The Evolution of AI Usage in Organizations09:51 Access Control and Agent Development14:15 Compliance Challenges with Automation17:49 Tiered Access and Governance Structures21:06 Exploring Autonomous Agents in the Workplace24:33 The Evolution of Control Planes26:45 AI in Support and Sales30:08 Understanding the Real Costs of AI Adoption31:21 Security Challenges with Agentic AI33:09 The Speed of Vulnerabilities45:39 Strategic Recommendations for CIOs#AI #EnterpriseAI #AIGovernance #Cybersecurity #AutonomousAgents #CIO #GenerativeAI #AgenticAI

We let an autonomous AI penetration testing agent run against our production application — and the results were unexpected.In this episode, we sit down with Grant McCracken from Dark Horse Security, who built Vulcan, an AI-powered pentesting agent capable of autonomously discovering vulnerabilities in real-world systems.Instead of traditional scanners or manual pentests, Vulcan ran continuously, explored the app like a human tester, and uncovered issues we hadn’t considered — including unexpected behavior in our AI workflows.We cover:How AI penetration testing actually worksWhy traditional scanners miss critical vulnerabilitiesWhat Vulcan found in our systemHow we fixed the issues it uncoveredThe future of continuous, autonomous security testingIf you're building with AI, deploying SaaS products, or working on secure infrastructure — this is a glimpse into what security will look like going forward.

In this conversation, Philippe Trounev and Jill Heinze (https://www.linkedin.com/in/jill-stover-heinze/) discuss the critical aspects of responsible AI governance, emphasizing the importance of stakeholder impact, human-centered design, and the balance between innovation and regulation. Jill shares her insights on the decision points organizations must consider when implementing AI, the role of governance in expediting development processes, and the necessity of understanding risk tolerance. They also explore the evolving regulatory landscape and the future of AI governance frameworks, highlighting the need for organizations to proactively address ethical considerations and user safety.takeawaysResponsible AI governance is essential for stakeholder impact.Organizations in regulated industries are more likely to invest in AI governance.Human-centered design is crucial for deploying AI responsibly.Understanding the risk profile of AI systems is necessary.Governance can expedite the development process when designed effectively.Organizations must assess their risk tolerance regarding AI use.Regulatory frameworks are evolving to address AI risks.AI governance should facilitate innovation rather than hinder it.Proactive governance can prevent potential litigation and rework.Engaging cross-functional teams is vital for effective AI governance.titlesNavigating the Future of AI GovernanceThe Role of Human-Centered Design in AISound Bites"AI governance is a design problem.""Every organization needs to ask these questions.""We need to surface the ground truth information."Chapters00:00Introduction to Responsible AI Governance02:56The Importance of Stakeholder Impact05:32Understanding AI Governance Decision Points08:13The Role of Human-Centered Design in AI10:57Balancing Innovation and Governance13:52Navigating Regulatory Frameworks16:47The Future of AI Regulation19:26Building Effective AI Governance Frameworks22:13Overcoming Objections to AI Governance24:56Conclusion and Resources for AI Governancehttps://www.docsie.io