Prompt versioning is configuration management
Prompts are code that changes behaviour in production. Treat them with the same discipline as feature flags and config files.
Daniel Reyes2 min read
If a prompt can change what your product does, it is configuration — and it deserves the same discipline as any other configuration: version control, review, staged rollout and the ability to roll back in seconds.
Keep prompts in the repository
Prompts that live in a database or a vendor dashboard are invisible to code review and impossible to diff. Store them as files next to the code that uses them, with the model name and parameters alongside.
id: reply-draft
version: 7
model: large-2026-06
temperature: 0.3
max_output_tokens: 600
system: |
You draft replies for customer-support agents.
Use the knowledge-base passages provided. If they do not answer
the question, say so and suggest escalating.
changelog: "Shorter replies; explicit escalation path (eval +4.1%)"Reference prompts by version
Application code should ask for reply-draft@7, not "the latest". That makes deploys reproducible and lets you run two versions side by side in an experiment.
Roll out like a feature flag
New prompt versions start at a small percentage of traffic, with dashboards comparing quality, cost and latency against the previous version. Rolling back is a config change, not a deploy.
Understand what you are configuring
It helps to have a mental model of how the model reads your prompt. This visual explanation of how transformers process text is the best introduction we know of:
Log the version with every response
Every model response we store includes the prompt ID and version. When someone reports a strange answer, the first question — "which prompt produced this?" — has an immediate answer.
Keep reading
Evals before vibes: a practical starting point
You cannot improve what you only feel. A lightweight evaluation loop you can set up in an afternoon, and why it pays for itself in the first week.
Maya Okafor3 min read
Retrieval that actually retrieves: a RAG checklist
Most retrieval-augmented generation failures are retrieval failures. Twelve checks to run before you blame the model.
Maya Okafor & Daniel Reyes3 min read
Structured outputs without tears
Getting reliable JSON out of a language model is a solved problem — if you combine schemas, validation and a sensible retry policy.
Maya Okafor2 min read