How caching works
On the first request with a system prompt, SERV generates a reasoning prompt and stores it. A later request in the same organization can reuse it when the original system prompt, SERV transformation type, and relevant prompt versions match. The requested application model is not part of the cache key. The customer request still goes to the upstream model each time; caching applies only to SERV’s reasoning-prompt generation. Cache entries are organization-scoped and expire after 30 days. A cache hit avoids a new reasoning-prompt generation charge and can reduce latency. It does not cache the model’s answer. For Chat Completions and Responses, bypass the cache for one request withmetadata.prompt_cache:
X-OpenServ-Prompt-Cache: false. The Messages endpoint currently always uses the SERV prompt cache and ignores these opt-out controls.
Keep system prompts stable when possible. Put changing data in user messages or tool results so SERV can reuse the generated reasoning prompt. A changed system-prompt string creates a different cache key automatically.
Use the opt-out when you need to regenerate an unchanged prompt during testing or incident response. Caching does not make a system prompt a safe place for secrets; follow your organization’s data-handling policy.