In this episode, I’m talking with Vincent Warmerdam about treating LLMs as just another API in your Python app, with clear boundaries, small focused endpoints, and good monitoring. We’ll dig into patterns for wrapping these calls, caching and inspecting responses, and deciding where an LLM API actually earns its keep in your architecture.

Seer: AI Debugging, Code TALKPYTHON

NordStellar

Talk Python Courses

Links from the show

Vincent on X: @fishnets88

Vincent on Mastodon: @koaning

LLM Building Blocks for Python Co-urse: training.talkpython.fm

Top Talk Python Episodes of 2024: talkpython.fm

LLM Usage - Datasette: llm.datasette.io

DiskCache - Disk Backed Cache (Documentation): grantjenks.com

smartfunc - Turn docstrings into LLM-functions: github.com

Ollama: ollama.com

LM Studio - Local AI: lmstudio.ai

marimo - A Next-Generation Python Notebook: marimo.io

Pydantic: pydantic.dev

Instructor - Complex Schemas & Validation (Python): python.useinstructor.com

Diving into PydanticAI with marimo: youtube.com

Cline - AI Coding Agent: cline.bot

OpenRouter - The Unified Interface For LLMs: openrouter.ai

Leafcloud: leaf.cloud

OpenAI looks for its "Google Chrome" moment with new Atlas web browser: arstechnica.com

Watch this episode on YouTube: youtube.com

Episode #528 deep-dive: talkpython.fm/528

Episode transcripts: talkpython.fm

🥁 Served in a Flask 🎸: talkpython.fm/flasksong

YouTube: youtube.com/@talkpython

Bluesky: @talkpython.fm

Mastodon: @[email protected]

X.com: @talkpython

Michael on Bluesky: @mkennedy.codes

Michael on Mastodon: @[email protected]

Michael on X.com: @mkennedy