The Stateless Founder

Build AI-First SOPs That Survive Model Changes

Listen Later
Build AI-First SOPs That Survive Model Changes

When models change on provider schedules you don't control, your prompts break. Today we build the fix: an AI-first SOP template that treats prompts as versioned assets.

The Problem: Brittle Prompts in a Moving Target Environment
  • OpenAI retired GPT-4o from ChatGPT February 13, 2026 (hard cutoff)
  • Traditional SOPs say "use GPT-4o" with no version, expiration, or fallback
  • Result: contractors debugging prompts that aren't broken when models disappear
    • The AI-First SOP Schema
    Header (Metadata Block)
    • Owner name + backup owner (critical for async teams)
    • Status: draft/approved/deprecated
    • SOP version number
    • Model tag with specific release date
    • Temperature band (0-0.2 for compliance, 0.3-0.6 for creative)
      • Steps with Versioned Prompts
      • Each model call gets unique prompt key + version number
      • Semantic versioning: major.minor.patch
        • Major: Output shape changes (text → JSON)
        • Minor: Instructions change, output contract same
        • Patch: Typo fixes, threshold tweaks
        • Full label: [email protected]#model_tag+dataset_hash
          • Input/Output Schemas
          • Field name, type, required/optional, description
          • JSON Schema for technical teams, simple tables for everyone else
          • Contractors don't guess what prompts expect
            • Failure Modes & Guardrails
            • OWASP Top 10 for LLM Applications (v2.0, 2025) catalogs common risks
            • Document specific failure modes for each workflow
            • Attach guardrail policy IDs (AWS Bedrock, NeMo Guardrails)
            • Version guardrail policies too
              • Tooling Options
              Small Teams (≤3 people): Pure Notion
              • Database with owner, status, SemVer, model tag, last edited time
              • Page history provides diffs for rollback
              • Button stamps changelog entry when publishing new version
              • Setup time: 45 minutes
                • Bigger Teams: Dedicated Platforms
                • PromptLayer: Registry with release labels, rollback, analytics
                  • Speak scaled 1→11 markets training non-technical teams to version prompts
                  • Humanloop: Version control with .prompt files that sync to Git
                    • Note: Platform sunset notice flagged in 2025 docs
                      • Platform Risk Mitigation
                      • Keep SemVer convention, model tags, changelog in your SOP
                      • These survive any platform migration
                      • Tool can disappear; versioning scheme persists
                        • The 30-Day Change Review Process
                        What to Check Monthly
                        • OpenAI deprecations page
                        • Azure model retirement tables
                        • Anthropic deprecation docs
                        • Vertex AI deprecation page
                          • When Something's Flagged
                          1. Pull affected SOPs
                          2. Rerun evals on replacement model (even just 5 test cases)
                          3. If outputs hold: update model tag, bump version
                          4. If outputs don't hold: patch prompt before deadline
                            5. Real Example: Meticulate
                            • Scaled to 1.5M LLM requests using PromptLayer
                            • Tagged every call by function and model
                            • When prompts regressed: search failing runs, find working version, rollback
                            • Versioned workflow enabled hotfixes in hours vs days
                              • The Cost of Not Having This
                              • 3AM messages from confused contractors
                              • 2 hours debugging prompts that aren't broken
                              • Client complaints on LinkedIn in front of 11K followers
                              • Margins drifting as pricing changes go unnoticed
                                • Implementation

                                This week: Pick your most critical AI workflow—the one that would hurt most if it broke tomorrow. Build its SOP first. Pin the model version, write the failure modes, set the 30-day review date.

                                Template: Grab the AI-First SOP template in the show notes. Duplicate it, fill in your model tag and inputs, get versioned prompts with built-in changelog by end of day.

                                Resources
                                • AI-First SOP Template (Notion) - Complete template with 3 worked examples
                                • OpenAI API Deprecations
                                • OWASP Top 10 for LLM Applications v2.0
                                • Semantic Versioning Spec
                                  • Case Studies Mentioned
                                  • Speak: Language learning app scaled 1→11 markets using PromptLayer for non-technical prompt editing
                                  • Meticulate: Scaled to 1.5M LLM requests with versioned prompt workflow for rapid rollbacks
                                    • ...more
                                    View all episodesView all episodes
                                    Download on the App Store
                                    The Stateless FounderBy Santi, Kira