System Design on the Road

System Design on the Road

By System Design on the RoadTechnology
Download on the App Store

System Design on the Road episodes

  • S2 E26 — Design an AI Evaluation Platform: Reproducibility, Grading, and Release Gates

    An evaluation score improves even though the model has not changed. Daniel and Maya design a platform that can trace the difference to data, graders, configuration, infrastructure, or actual model behavior.


    Frozen inputs, task attempts, grading versions, failure classification, and aggregation form a reproducible run. Infrastructure errors and retries must not distort model comparisons. The interview ends with uncertainty and release gates, asking what evidence a higher score really provides for a deployment decision.


    Chapters

    00:00:00 Welcome and an unexplained score gain

    00:02:26 Frozen datasets, models, and run inputs

    00:05:29 Task execution, grading, and aggregation

    00:08:46 Grader changes and comparison validity

    00:12:32 Worker failures, retries, and fairness

    00:16:33 Uncertainty and reproducible comparisons

    00:20:15 Release gates and evaluation operations

    00:31:14 Final score interpretation and recap


    Daniel is the interviewer, voiced by Liam. Maya is the engineer, voiced by Bella. AI narration at a calm pace, with three thinking pauses.


    Opening music: Stealth Groover by Kevin MacLeod (incompetech.com). Licensed under Creative Commons Attribution 4.0: https://creativecommons.org/licenses/by/4.0/ . Edited excerpt with EQ, volume automation, a smooth fade, and narration.

    35 min
  • S2 E25 — Design an AI Research Assistant: Sources, Budgets, and Checkpoints

    Three sources tell two different stories and the research deadline is approaching. Daniel and Maya design an assistant that can choose what to investigate, preserve its evidence, and stop within time and spending limits.


    A working lookup expands into parallel research, conflicting sources, checkpoints, and recovery. Tool results and pages remain evidence rather than authorities. Evaluation asks whether the answer is supported and useful, including when uncertainty or insufficient evidence is the conclusion the assistant should report.


    Chapters

    00:00:00 Welcome and conflicting research sources

    00:02:30 Answer requirements and evidence records

    00:05:47 A complete research lookup

    00:09:29 Conflicting sources and uncertainty

    00:13:24 Parallel research and bounded budgets

    00:17:15 Checkpoints, tools, and untrusted content

    00:21:29 Evaluating useful, supported research

    00:30:22 Final answer and evidence trail


    Daniel is the interviewer, voiced by Liam. Maya is the engineer, voiced by Bella. AI narration at a calm pace, with three thinking pauses.


    Opening music: Stealth Groover by Kevin MacLeod (incompetech.com). Licensed under Creative Commons Attribution 4.0: https://creativecommons.org/licenses/by/4.0/ . Edited excerpt with EQ, volume automation, a smooth fade, and narration.

    34 min
  • S2 E24 — Design Enterprise RAG: Retrieval, Permissions, and Citations

    An employee may read a document when it is retrieved, but loses access before the generated answer is released. Daniel and Maya design enterprise RAG around that permission race rather than relevance alone.


    Document versions, ingestion, retrieval, evidence selection, and answer generation form the baseline. Deletion and indexing delays test the copies between source and response. Citations and evaluation ask whether the final answer has adequate evidence and whether the user is still authorized to receive it.


    Chapters

    00:00:00 Welcome and a wrongly disclosed answer

    00:02:24 Source records, versions, and permissions

    00:05:41 Ingestion, retrieval, and generation

    00:09:12 Revocation races and release-time checks

    00:13:14 Relevance, evidence, and citations

    00:17:08 Index failures, deletion, and freshness

    00:20:58 Evaluating retrieval and answer evidence

    00:30:50 Final authorized answer flow and recap


    Daniel is the interviewer, voiced by Liam. Maya is the engineer, voiced by Bella. AI narration at a calm pace, with three thinking pauses.


    Opening music: Stealth Groover by Kevin MacLeod (incompetech.com). Licensed under Creative Commons Attribution 4.0: https://creativecommons.org/licenses/by/4.0/ . Edited excerpt with EQ, volume automation, a smooth fade, and narration.

    35 min
  • S2 E23 — Design a Cloud Coding Environment: Sandboxing, Secrets, and Workspace Recovery

    A repository's test script runs hostile code inside a cloud development workspace. Daniel and Maya design isolation while preserving useful files and developer progress.


    Sandboxing, network access, secrets, and permission to cause external effects require distinct boundaries. A machine vanishing before a patch is saved tests durable workspace state. Prepared images, capacity, cleanup, and recovery connect the security model to the everyday experience of opening a project, running code, and returning to saved work.


    Chapters

    00:00:00 Welcome and an untrusted test script

    00:02:32 Workspace state and execution boundaries

    00:05:48 A complete cloud development session

    00:09:07 Sandboxing hostile repository code

    00:13:11 Secrets, network access, and side effects

    00:17:03 Workspace persistence and machine loss

    00:20:56 Images, capacity, and cleanup

    00:30:42 Final recovery and isolation guarantees


    Daniel is the interviewer, voiced by Liam. Maya is the engineer, voiced by Bella. AI narration at a calm pace, with three thinking pauses.


    Opening music: Stealth Groover by Kevin MacLeod (incompetech.com). Licensed under Creative Commons Attribution 4.0: https://creativecommons.org/licenses/by/4.0/ . Edited excerpt with EQ, volume automation, a smooth fade, and narration.

    35 min
  • S2 E22 — Design a Browser Agent: Sessions, Permissions, and Recovery

    A browser agent clicks Submit and disconnects before seeing the result. Daniel and Maya design recovery that preserves the uncertainty instead of automatically repeating a potentially consequential action.


    Goals, sessions, attempts, observations, and permissions have separate records and authorities. Untrusted page instructions test that separation. Checkpoints, human takeover, cleanup, and monitoring complete the lifecycle, with a clear distinction between an attempted click and a verified result.


    Chapters

    00:00:00 Welcome and an uncertain submit action

    00:02:32 Goals, sessions, and permission boundaries

    00:05:40 An action-observation session baseline

    00:08:56 Uncertain effects and safe retries

    00:12:57 Untrusted pages and prompt injection

    00:17:07 Checkpoints, human takeover, and cleanup

    00:21:01 Monitoring attempts and verified outcomes

    00:30:17 Final authority and recovery boundaries


    Daniel is the interviewer, voiced by Liam. Maya is the engineer, voiced by Bella. AI narration at a calm pace, with three thinking pauses.


    Opening music: Stealth Groover by Kevin MacLeod (incompetech.com). Licensed under Creative Commons Attribution 4.0: https://creativecommons.org/licenses/by/4.0/ . Edited excerpt with EQ, volume automation, a smooth fade, and narration.

    34 min
  • S2 E21 — Design a Collaborative Prompt Playground: Versioning and Reproducible Runs

    A teammate changes a prompt while an experiment is running, and later nobody knows which instructions produced the answer. Daniel and Maya design a collaborative playground that allows editable drafts without losing immutable run inputs.


    Concurrent edits, run snapshots, model settings, and result history establish traceability. Reconnection and revoked access test the shared room. Comparing outputs becomes a recorded experiment with explicit conditions, rather than a comparison between whatever happens to be in the editor now.


    Chapters

    00:00:00 Welcome and an untraceable experiment

    00:02:22 Drafts, versions, and run identities

    00:05:44 Editing and running a prompt

    00:09:02 Concurrent edits and save conflicts

    00:13:10 Immutable snapshots for reproducible runs

    00:17:20 Reconnection and revoked workspace access

    00:21:15 Comparing runs under recorded conditions

    00:30:01 Final draft-to-result traceability


    Daniel is the interviewer, voiced by Liam. Maya is the engineer, voiced by Bella. AI narration at a calm pace, with three thinking pauses.


    Opening music: Stealth Groover by Kevin MacLeod (incompetech.com). Licensed under Creative Commons Attribution 4.0: https://creativecommons.org/licenses/by/4.0/ . Edited excerpt with EQ, volume automation, a smooth fade, and narration.

    34 min
  • S2 E20 — Design a Model Distribution System: Downloads, Rollouts, and Rollbacks

    The model files have downloaded, but the worker is not ready to serve. Daniel and Maya design distribution as a sequence of transfer, verification, staging, loading, and traffic activation.


    Immutable manifests and content identities define the release. Fleet-wide downloads, interrupted transfers, partially staged files, and mixed versions test the plan. A rollout needs readiness evidence and a usable rollback target; moving all the bytes successfully is only one step toward safely changing the model behind a request.


    Chapters

    00:00:00 Welcome and a model that is not ready

    00:02:41 Immutable releases and artifact identity

    00:06:08 Download, verify, load, and activate

    00:09:19 Fleet-wide downloads and bandwidth

    00:12:42 Resuming downloads and staging safely

    00:16:37 Mixed versions, rollout, and rollback

    00:20:41 Security, readiness, and operational cost

    00:30:06 Final controlled release and recap


    Daniel is the interviewer, voiced by Liam. Maya is the engineer, voiced by Bella. AI narration at a calm pace, with three thinking pauses.


    Opening music: Stealth Groover by Kevin MacLeod (incompetech.com). Licensed under Creative Commons Attribution 4.0: https://creativecommons.org/licenses/by/4.0/ . Edited excerpt with EQ, volume automation, a smooth fade, and narration.

    34 min
  • S2 E19 — Design an LLM Inference Service: Batching, Admission Control, and Fairness

    A short question waits behind a prompt containing an entire book. Daniel and Maya design a shared inference service that uses GPUs efficiently while protecting latency and tenant fairness.


    Batch formation, request length, resource budgets, admission, and queueing shape the working path. Slow consumers and worker failures complicate an answer already streaming to a client. The final design explains which throughput gains are worth their waiting time and what can still be promised after partial output.


    Chapters

    00:00:00 Welcome and a blocked short request

    00:02:45 Latency contracts and request ownership

    00:06:17 A working inference request path

    00:10:01 Batching, long prompts, and latency

    00:13:54 Admission budgets and tenant fairness

    00:18:22 Worker failure during streamed output

    00:22:25 Monitoring and safe service changes

    00:29:30 Final inference flow and tradeoffs


    Daniel is the interviewer, voiced by Liam. Maya is the engineer, voiced by Bella. AI narration at a calm pace, with three thinking pauses.


    Opening music: Stealth Groover by Kevin MacLeod (incompetech.com). Licensed under Creative Commons Attribution 4.0: https://creativecommons.org/licenses/by/4.0/ . Edited excerpt with EQ, volume automation, a smooth fade, and narration.

    34 min
  • S2 E18 — Design a Video Generation Service: GPU Scheduling and Job Recovery

    A video-generation job is preempted, canceled, and then completed by an obsolete worker. Daniel and Maya design the lifecycle that keeps this late result from becoming the customer's published output.


    Durable jobs, GPU placement, fair admission, checkpointing, and resumption establish the baseline. Scarce compute makes wasted work and queue policy visible costs. The interview separates producing output bytes from holding current authority to publish them, then follows cancellation and recovery through that boundary.


    Chapters

    00:00:00 Welcome and an obsolete video result

    00:03:07 Inputs, budgets, and durable job state

    00:07:04 Prompt-to-download baseline

    00:10:57 GPU placement, admission, and fairness

    00:15:23 Preemption, checkpoints, and resumption

    00:19:29 Cancellation and result-publication fencing

    00:23:38 Monitoring and protecting scarce compute

    00:30:48 Final job and publication guarantees


    Daniel is the interviewer, voiced by Liam. Maya is the engineer, voiced by Bella. AI narration at a calm pace, with three thinking pauses.


    Opening music: Stealth Groover by Kevin MacLeod (incompetech.com). Licensed under Creative Commons Attribution 4.0: https://creativecommons.org/licenses/by/4.0/ . Edited excerpt with EQ, volume automation, a smooth fade, and narration.

    35 min
  • S2 E17 — Design an AI Chat Service: Streaming, Reconnection, and Cancellation

    An AI answer is still being generated when its browser connection breaks. Daniel and Maya design a chat service that resumes the existing answer rather than silently starting and charging for another generation.


    Conversation branches, generation IDs, durable stream events, cursors, and worker ownership connect the interface to model execution. Cancellation has its own outcome; closing a socket is not enough. Admission, latency, and cost limits test which work the system accepts and how it handles a worker that keeps running.


    Chapters

    00:00:00 Welcome and an interrupted AI answer

    00:03:08 Conversation branches and generation IDs

    00:07:09 A complete streamed generation

    00:11:05 Durable events, cursors, and reconnection

    00:15:41 Cancellation and stale-worker ownership

    00:19:40 Admission, latency, and generation cost

    00:23:41 Lifecycle monitoring and recovery

    00:31:01 Final resume and cancellation guarantees


    Daniel is the interviewer, voiced by Liam. Maya is the engineer, voiced by Bella. AI narration at a calm pace, with three thinking pauses.


    Opening music: Stealth Groover by Kevin MacLeod (incompetech.com). Licensed under Creative Commons Attribution 4.0: https://creativecommons.org/licenses/by/4.0/ . Edited excerpt with EQ, volume automation, a smooth fade, and narration.

    34 min

About System Design on the Road

From the publisher's feed

Learn system design wherever you are. Each episode builds from first principles to the decisions engineers face in interviews and real systems, using concrete examples, trade-offs, recall checks, and…