Organizations are investing in AI tools, tracking usage, and providing training—but are those measures telling us whether people can actually work with AI effectively?
In this episode of Tangents with TorranceLearning, Megan Torrance talks with Sam Rogers, founder and CEO of PAICE.work, about measuring People + AI Collaboration Effectiveness. Sam explains why tool adoption, course completion, and self-reported confidence offer only a limited view of AI capability—and how behavioral observation can help organizations understand what people actually do when AI is incomplete, uncertain, or wrong.
They explore AI collaboration as a performance system, the importance of governance readiness, and the role of privacy-first measurement in identifying organizational risks without turning assessment into employee surveillance. Sam also shares why effective AI use depends less on technical knowledge than on judgment, accountability, verification, and the ability to improve the work produced with AI in the loop.
Key topics discussed
- Why AI usage and training completion do not necessarily demonstrate capability
- What PAICE—People + AI Collaboration Effectiveness measures
- The five dimensions of the PAICE framework: performance, accountability, integrity, collaboration, and evolution
- The difference between using AI and collaborating effectively with AI
- Why behavioral observation can reveal more than knowledge tests or self-assessments
- How PAICE evaluates performance through an adaptive, work-like AI simulation
- Governance readiness, including guardrails, review cycles, ownership, and escalation paths
- Measuring human-level AI risk rather than focusing only on the technology
- Protecting individual privacy while giving organizations an aggregate view of capability gaps
- Designing AI systems to support privacy, equity, accessibility, and responsible participation
- Sam’s multi-model workflow, Obsidian system, paper notebooks, and resistance to doing everything on a phone
Hosts: Megan Torrance and Meg Fairchild
Producers: Meg Fairchild and Dean Castile
Music: Original music by Dean Castile
Resources and links from this episode
Learning Transfer Evaluation Model: Will Thalheimer’s framework for evaluating learning outcomes and performance
PAICE Framework Whitepaper: Measuring People + AI Collaboration Where It Actually Happens
PAICE.work: Learn more about the behavioral assessment of People + AI Collaboration Effectiveness.
Voice AI in Firms: A Natural Field Experiment on Automated Job Interviews
AI Transparency Statement: AI was used to generate the first draft of the transcript and the show notes for this episode. It was then edited by real humans.