Skip to main content
A stable prompt entering a cache vault and branching into reusable request paths
SERV caches generated reasoning prompts per organization. The cache is enabled by default.

​
How caching works

On the first request with a system prompt, SERV generates a reasoning prompt and stores it. A later request with the same organization, prompt, and relevant reasoning configuration can reuse that generated prompt. The customer request still goes to the upstream model each time; caching applies to the SERV reasoning-generation step. Cache entries are organization-scoped and expire automatically. A cache hit can reduce latency and generation cost. It does not cache the model’s answer. Disable it for a request with metadata.prompt_cache:
Keep system prompts stable when possible. Put changing data in user messages or tool results so generated reasoning can be reused safely. Use metadata.prompt_cache: "disabled" when the system prompt represents a newly changed policy, when you are testing a fresh prompt version, or when you need to avoid reusing a previous reasoning transformation. Do not put secrets in a system prompt merely because caching is enabled; follow your organization’s data-handling policy.