UX - The User Experience Podcast

I'd love to hear from you. Get in touch!

Evaluating AI Agents, Claude's Computer Access & Prompt-Only Enterprise Software

🔬 EVA — A Framework for Evaluating Voice Agents

• I hadn't realised we lacked a proper evaluation framework for voice agents — this one from Hugging Face caught my attention
• What I like: it combines two dimensions I've always thought should go together — accuracy (task completion, faithfulness, speech fidelity) and experience (conciseness, conversational flow, turn-taking)
• My question: is the "experience" side actually measured with real end users, or just by the designers?
• This connects to a three-step evaluation model I keep coming back to: define your ingredients, evaluate internally, then validate with users — and compare the gap
• I'll dedicate a full episode to this, but the short version is: if you want to elicit trust or satisfaction, you need to know which product attributes actually produce those outcomes

🤖 Claude + Cowork — AI With Access to Your Computer

• Cowork now lets you authorise Claude to access your files and folders so it can act on your behalf even when you're away
• I'm genuinely torn — amazed by the technology, but uncomfortable with the direction
• My concern isn't the capability itself, it's the pattern: LLMs arrive, and suddenly we open the gates to everything — recording, transcription, computer access — as if these things naturally belong together
• My rule of thumb: always assume your data is being used to improve the product — if you have doubts, assume yes
• I'd love to see more push for private, self-hosted LLMs — but the honest tension is that commercial ones will keep winning on convenience because they have more data to train on
• It's not even apples to apples — and that's what makes this hard

🖥️ Aragon — What If Enterprise Software Was Just a Prompt?

• Startup Aragon raised $12M at a $100M valuation to replace enterprise tools like Salesforce, Jira, and Tableau with a single LLM interface
• Their thesis: buttons and menus are dead, future business is done by prompt
• My honest reaction: I get why this is being explored — we're mapping the edges of a new territory and seeing what sticks
• But one modality for everything? I'm not convinced — when I was building my own website, I actually wanted both: LLM for generation, drag-and-drop for fine-tuning — and that product barely exists yet
• Users have 10+ years of muscle memory with their tools — strip that away and you're not simplifying, you're adding friction
• Nielsen's heuristics exist for a reason: people need control, exit doors, and multiple ways to accomplish a task

Support the show

Help me improve the show HERE

I'd love to hear from you. Get in touch!

UX Digest | EP. 1 — Google Stitch, Microsoft's AI Rollback & the Rise of AI Note-Takers

In this episode of UX Digest, I react to three stories at the intersection of AI, product design, and user experience research — sharing my perspective as a UX researcher working daily with AI-driven tools.

🎨 Google Stitch — "Vibe Design" Goes Mainstream

Google's AI-powered UI design tool is making waves, but is it really a Figma killer? UX design cannot be reduced to the tool you use — Stitch handles maybe 5–10% of what a real UX designer does. Useful? Yes. Disruptive? Not sure.

🪟 Microsoft Dials Back Copilot — The "Less Is More" Lesson

Microsoft rolled back AI integrations across Windows 11 — photos, widgets, notepad, and more — following consumer pushback against what's being called "AI bloat." I reflect on why this keeps happening: technology gets deployed before user needs are validated. A reminder that AI doesn't automatically create value just because it's everywhere.

🎙️ AI Note-Taking Devices — Convenience vs. Privacy

From credit-card-sized recorders to AI pendants and earbuds, the market for ambient transcription hardware is growing fast — alongside software tools like Fireflies, Granola, and Notion AI. I raise a question worth sitting with: why does recording someone feel more acceptable in 2026 than it did a decade ago? And what are we losing cognitively when we stop taking notes by hand?

Support the show

Help me improve the show HERE

I'd love to hear from you. Get in touch!

Welcome back to UXChange, I’m Jeremy — and today we’re talking about something I’ve been wrestling with personally: when should we actually use AI?

We all love what it can do — speed, automation, magic. But at what point does using AI stop helping us think, and start replacing the thinking itself?

In this episode, I share my personal heuristics for using AI at work — not as a researcher, but as a knowledge worker trying to stay effective, curious, and human in an AI-powered world. I unpack questions like:

• When is AI genuinely adding value, and when is it just noise?
• How much control are you willing to give away for efficiency?
• Where’s the line between delegation and disconnection?

I’ll talk about why some tasks must stay human — especially those that shape your understanding, your craft, and your credibility — and how I’ve learned to decide where AI fits in my creative and research process.

🎧 Listen now for a raw, reflective episode about the mindset, trade-offs, and self-trust we all need in this new era of AI-driven productivity.

Support the show

Help me improve the show HERE

I'd love to hear from you. Get in touch!

Welcome back to UXChange — this is Part 2 of our series on Measuring the User Experience of AI.

In the first episode, we explored vanity vs. sanity metrics — the surface-level KPIs that look good on dashboards versus the deeper indicators that truly define trust, satisfaction, and control.

Today, we’re going a step further. I’ll break down why we need a new approach to measure AI experiences, and why our traditional UX frameworks simply don’t fit anymore.

If you’ve ever asked yourself “how do I even measure if my AI is working well?” — this episode will give you the foundation to start answering that question the right way.

Support the show

Help me improve the show HERE

I'd love to hear from you. Get in touch!

In this new series, we’ll dive deep into what it really means to measure the success of AI-enabled user experiences. Beyond surface-level KPIs and “vanity metrics,” we’ll talk about the sanity metrics that actually reveal whether your product is usable, reliable, trustworthy, and meaningful for real people.

I’ll break down complex ideas like trust, perceived control, satisfaction, and holistic measurement in a way that’s practical for designers, researchers, and product teams.

If you’ve ever wondered how to move from counting clicks to understanding confidence, or how to measure what truly matters in AI, this series is for you.

Support the show

Help me improve the show HERE

I'd love to hear from you. Get in touch!

In this episode, I explore how much of user experience happens beneath the surface of awareness — and why tapping into the unconscious is essential for better product design. 

• Why users often say one thing but do another
• The limits of relying only on interviews or only on observation
• How unconscious motivations, needs, and fears shape technology use
• The power of co-creation and generative workshops to reveal hidden dimensions
• Practical ways to integrate unconscious insights into product decisions

Support the show

Help me improve the show HERE

I'd love to hear from you. Get in touch!

In this episode I explore how product teams can better anticipate and measure user experience in a world where interaction patterns are shifting—from clicks and forms to voice and chat. 

• Why AI is transforming the way users interact with technology
• The importance of treating design choices like “ingredients” in a recipe
• How to build internal benchmarks to predict user experience outcomes
• Lessons from AI tools like ChatGPT and the rise of “Generative Engine Optimization”
• Practical ways to align intended vs. actual user experience through experiment design

Support the show

Help me improve the show HERE

I'd love to hear from you. Get in touch!

I break down the difference between vanity metrics (what looks good) and sanity metrics (what actually matters), plus the new measurement challenges when users collaborate with AI instead of just using a product. Essential for anyone designing or evaluating AI features.

Topics:

• Why traditional UX metrics fail with AI systems
• The dangerous difference between vanity and sanity metrics
• How collaboration changes everything we measure
• Essential metrics you're probably missing

Support the show

Help me improve the show HERE

I'd love to hear from you. Get in touch!

📝 Episode Description

• Why AI breaks traditional UX rules
• The costly mistake of treating AI like automation
• How that 5% error rate could lose you big deals
• Essential if you're adding AI to your product

Support the show

Help me improve the show HERE

I'd love to hear from you. Get in touch!

In today's episode, I shared my thoughts on AI from my dual perspective as a UX professional and admittedly picky user. Here's what we covered:

🧠 My UX Philosophy

• I'm that friend who gets frustrated when interfaces don't work as expected (yes, I might scream at my phone occasionally 😅)
• This pickiness led me naturally to UX 
• My low patience = high empathy for users!

💡 AI vs. LLMs - Let's Get Clear

• AI is a broad field, but I'm concerned when we call LLMs "artificial intelligence"
• LLMs predict the next word in a sequence - fascinating but limited!
• Setting proper expectations is crucial ⚠️

🎨 My AI UX Guidelines

1. Be honest about limitations
   • Tell users when AI might be wrong
   • Careful with the mysterious "magic" sparkles ✨
2. Offer control & feedback
   • Always include options to give feedback
   • Provide manual backup options
   • Show clear examples of what to ask
3. Respect privacy
   • Be transparent about how user data is used
   • Remember the Figma incident? Yikes! 😬

🛠️ Tools Doing It Right

• Perplexity: Shows its reasoning process - love this!
• Contextual AI in everyday apps > standalone chat interfaces

🔮 Coming Up Next

I'll be tackling trust in AI, the job-replacement debate, and more practical UX patterns!

🤗 Stay Connected!

Subscribe, enable notifications, and share with friends who care about thoughtful UX design.

Until next time!

Support the show

Help me improve the show HERE