# How we investigated prompt cache misses with Datadog

DevFeed: [How we investigated prompt cache misses with Datadog](<https://devfeed.tech/articles/how-we-investigated-prompt-cache-misses-with-datadog-122067.md>)

Original publisher: [Read original article](<https://www.datadoghq.com/blog/prompt-cache-investigation-ccm-agent-observability/>)

Author: Bruno Volpato; Will Potts; JC Mackin

Published: 2026-11-06T00:00:00Z

Content type: article

Language: en

Sources: [Datadog](<https://devfeed.tech/sources/datadog-the-monitor-blog.md>)

Topics: [Cache](<https://devfeed.tech/topics/cache.md>), [LLM observability](<https://devfeed.tech/topics/llm-observability.md>), [agent observability](<https://devfeed.tech/topics/agent-observability.md>), [cloud cost management](<https://devfeed.tech/topics/cloud-cost-management.md>), [datadog](<https://devfeed.tech/topics/datadog.md>), [Prompt Engineering](<https://devfeed.tech/topics/prompt-engineering.md>)

Tags: [added](<https://devfeed.tech/tags/added.md>), [agent](<https://devfeed.tech/tags/agent.md>), [agent-observability](<https://devfeed.tech/tags/agent-observability.md>), [agents](<https://devfeed.tech/tags/agents.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-applications](<https://devfeed.tech/tags/ai-applications.md>), [also](<https://devfeed.tech/tags/also.md>), [anthropic](<https://devfeed.tech/tags/anthropic.md>), [applications](<https://devfeed.tech/tags/applications.md>), [cache](<https://devfeed.tech/tags/cache.md>), [cloud](<https://devfeed.tech/tags/cloud.md>), [cloud-cost-management](<https://devfeed.tech/tags/cloud-cost-management.md>), [cost](<https://devfeed.tech/tags/cost.md>), [cost-management](<https://devfeed.tech/tags/cost-management.md>), [datadog](<https://devfeed.tech/tags/datadog.md>), [debugging](<https://devfeed.tech/tags/debugging.md>), [investigation](<https://devfeed.tech/tags/investigation.md>), [management](<https://devfeed.tech/tags/management.md>), [observability](<https://devfeed.tech/tags/observability.md>), [prompt](<https://devfeed.tech/tags/prompt.md>), [with](<https://devfeed.tech/tags/with.md>)

## AI overview

Datadog Cloud Cost Management helped identify provider spending, while Agent Observability exposed request-level causes of prompt cache misses. Reuse depends on a matching request prefix: changing details placed before stable instructions or nondeterministic tool and file ordering can disrupt it. The investigation also found caching options that failed to send required provider controls, and Anthropic compatibility endpoint traffic that did not support caching; native Messages API traffic showed cache reads. The team proposed sorting tools and files, putting stable instructions first, passing cache controls through, and checking outgoing requests and later cache reads. Evaluation should compare the same routes and task types before and after, including task cost, cache reads after tool calls, latency, accuracy, retries, fallback calls, and successful tool use. Token estimates help explain behavior, but provider bills are needed to confirm savings; cache writes can add cost, and longer histories can increase input volume.

## Source excerpt

An investigation of prompt cache misses in internal AI applications by using Datadog Cloud Cost Management and Agent Observability.