Okay, But Why?

We return to the original support problem and follow the complete journey: discovery, scoping, prototyping, architecture, integration, evaluation, production, adoption, and feedback. Along the way, we explain Forward Deployed Engineering and compare it with software engineering, AI/ML engineering, solutions architecture, and DevRel. The final realization: technology is not the requirement. The problem is the requirement.

Real users have arrived, and so have latency, rate limits, privacy questions, and bills. We explore model choice, caching, fallbacks, logging, and operational ownership before comparing hosted APIs with local and self-hosted models. Sabrina brings the open-weight question; GPUs and quantization get plain-language explanations. The model turns out to be one dependency inside a much larger software system.

The demo worked. Unfortunately, that’s not a test strategy. We try to break Support Assistant with missing policies, conflicting documents, unavailable services, malicious instructions, and unauthorized refund requests. Learn how ordinary tests, AI evaluations, human review, monitoring, and traces work together—and why a convincing answer or impressive average score can hide a serious failure.

When does a chatbot with tools become an agent—and would that actually help? We compare tool assistants, predefined workflows, and systems that let a model choose what happens next. Planning, state, memory, orchestration, and multiple agents all get a turn, but so do budgets, stopping conditions, and the humble if statement. The robot’s lanyard remains under review.

We already have APIs and tool calling. What problem is MCP trying to solve? Parisa, Jules, and Sabrina untangle hosts, clients, servers, tools, resources, and prompts as Support Assistant connects to more systems. The USB-C analogy makes an appearance—with limits—because a shared connection format doesn’t guarantee compatibility, trustworthy integrations, or permission to access everything.

Reading the shipping policy is useful. Checking a real order requires a different capability. We follow a tool call from the model’s proposed arguments through application-controlled validation, authorization, execution, and results. Then someone asks for a refund, and suddenly permissions, human approval, accessible confirmations, and avoiding duplicate transactions become very relevant.

Brenda’s 40,000 documents are back, and this time we’re retrieving the useful ones. We explain retrieval-augmented generation, embeddings, vectors, semantic search, and chunking without turning the episode into a mathematics lecture. Learn how RAG differs from keyword search and fine-tuning—and why retrieving a passage doesn’t automatically make the resulting answer correct.

How does a model become a feature in your own application? Follow a request from the browser through your server to a model provider, with APIs, SDKs, streaming, token costs, and failure handling along the way. There’s a Chef Nervous Robot callback, a firm boundary around secret API keys, and a familiar realization: wait, that’s just JavaScript.

Can we replace elaborate prompt rituals with a useful brief? Sabrina joins Parisa and Jules to discuss instructions, examples, context windows, randomness, and structured output. Brenda proposes pasting all 40,000 company documents into the prompt. We explain why that’s a problem—and why better prompting doesn’t mean training, guaranteed accuracy, or enforceable security.

Is a language model a digital mind or autocomplete wearing an expensive coat? We explore tokens, next-token prediction, transformers, attention, and context to understand what happens when an LLM generates an answer. Support Assistant discovers that sounding like an experienced employee is not the same as knowing the company’s actual refund policy.