# ai observability Published articles for ai observability. This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles. ## Beyond the 200 OK: Architecting Observability for AI DevFeed: [Beyond the 200 OK: Architecting Observability for AI]() Original publisher: [Read original article]() Author: Adriano Mota Published: 2026-09-11T07:00:00Z Content type: article Language: en Sources: [Nordic APIs]() Topics: [ai observability](), [observability](), [Application Performance Management (APM)](), [AI Agent](), [Artificial Intelligence]() Tags: [ai](), [ai-agent](), [ai-agents](), [ai-observability](), [api-logging](), [api-metrics](), [api-monitoring](), [api-security](), [api-testing](), [apm](), [artificial-intelligence](), [blog](), [generative-ai](), [llm](), [mcp](), [observability]() ### AI overview An article about designing observability for AI systems beyond traditional APM metrics, with emphasis on tracking quality, cost, retrieval, and agent behavior. ### Source excerpt Traditional monitoring tools, such as application performance monitoring (APM), were engineered to monitor deterministic software where specific inputs reliably lead to predictable outputs through hard-coded logic. When a traditional API fails, it usually throws a 500 Internal Server Error. But when an AI agent fails, it might return a perfectly healthy 200 OK status code ... ## Monitor prompt caching to optimize your token usage DevFeed: [Monitor prompt caching to optimize your token usage]() Original publisher: [Read original article]() Author: Thomas Sobolik Published: 2026-09-02T00:00:00Z Content type: tutorial Language: en Sources: [Datadog | The Monitor blog]() Topics: [Caching](), [Large Language Model](), [Latency](), [Traces](), [SIEM, Security, Observability](), [anthropic]() Tags: [agent-observability](), [agents](), [ai](), [ai-engineering](), [ai-observability](), [anthropic](), [caching](), [cost](), [how-to](), [latency](), [llm](), [models](), [monitor](), [openai](), [traces]() ### AI overview A tutorial on prompt caching for LLM and agent workloads, covering cache behavior, provider differences, and monitoring token use and latency. ### Source excerpt Learn how to use prompt caching effectively and monitor your models and agents to troubleshoot cache invalidations. ## From traces to experiments: A loop for improving AI agents DevFeed: [From traces to experiments: A loop for improving AI agents]() Original publisher: [Read original article]() Author: Adam Virani; Lukas Goetz-Weiss; Natasha Silva Published: 2026-09-01T00:00:00Z Content type: tutorial Language: en Sources: [Datadog | The Monitor blog]() Topics: [ai observability](), [AI Bots]() Tags: [agent-observability](), [agents](), [ai](), [ai-agents](), [ai-observability](), [evaluation](), [experiments](), [latency](), [optimization](), [production](), [telemetry](), [traces]() ### AI overview The article explains how teams can use AI-agent trace data, evaluations, and production experiments to identify performance issues and test whether changes improve outcomes. ### Source excerpt Learn how to read AI agent traces as a roadmap and how to run production experiments that measure whether improvements hold in production. ## AI gateway best practices: Model routing, reliability, and budget controls for production agents DevFeed: [AI gateway best practices: Model routing, reliability, and budget controls for production agents]() Original publisher: [Read original article]() Author: Thomas Sobolik Published: 2026-07-23T00:00:00Z Content type: tutorial Language: en Sources: [Datadog | The Monitor blog]() Topics: [AI, ML & Data Engineering](), [LLM evaluation / benchmarking](), [ai observability](), [Azure OpenAI]() Tags: [agent-observability](), [agents](), [ai-gateway](), [ai-observability](), [benchmarking](), [cost](), [llm](), [monitor](), [observability](), [production](), [routing]() ### AI overview The article explains how AI gateways centralize access to multiple LLM providers for production agents. It focuses on model routing and evaluation, reliability controls such as retries and fallbacks, and budget governance. ### Source excerpt Learn how AI gateways help you scale your agents to consume multiple LLM services reliably, and how to monitor these systems to ensure you're getting the best performance and cost. ## Harness AgentTrace: An Observability and Guardrail Framework DevFeed: [Harness AgentTrace: An Observability and Guardrail Framework]() Original publisher: [Read original article]() Author: Sunil Gattupalle Sanjay Nagaraj Published: 2026-07-21T00:00:00Z Content type: article Language: en Sources: [Harness Blog]() Topics: [agent observability](), [ai observability](), [Framework](), [Testing](), [tracing]() Tags: [agents](), [ai](), [ai-observability](), [apache](), [ci](), [evaluation](), [observability](), [testing](), [tracing]() ### AI overview Harness describes AgentTrace, an internal framework for observing, evaluating, and governing AI agents in production. It connects production monitoring with evaluation, allows failures to become regression test cases, and includes open-source harness-sdk and harness-evals layers under Apache 2.0 that work with any OpenTelemetry backend. ### Source excerpt Harness AgentTrace unifies AI observability, evaluation, and guardrails to detect failures, improve quality, and secure AI agents in production. | Blog ## From Traditional ML to AI Agents: How Booking.com Scales AI Observability With Arize AI DevFeed: [From Traditional ML to AI Agents: How Booking.com Scales AI Observability With Arize AI]() Original publisher: [Read original article]() Author: Amir Bitaraf Published: 2026-07-10T07:52:18Z Content type: article Language: en Sources: [Booking.com Data Science]() Topics: [ai observability](), [agent observability](), [Machine learning](), [observability](), [human review]() Tags: [agents](), [ai](), [ai-agents](), [ai-observability](), [blog-posts](), [data-quality](), [human-review](), [latency](), [ml](), [observability]() ### AI overview Booking.com describes building an AI-native observability stack for traditional machine learning systems and agentic AI workflows. The article explains that its diverse systems require observability to detect changes, regressions, data quality issues, misconfigurations, and responsible-AI concerns across different operating constraints and user contexts. ### Source excerpt Building an AI-native observability stack for agentic AI and traditional ML at Booking.com Authors: Amir Bitaraf, Shahaf Veber Why AI Observability Matters at Booking.com At Booking.com, AI helps travellers and partners in every step of their journey, from how people discover destinations to the way we support them while they're on the road. Rather than a single flagship model, we rely on a large and growing collection of systems that each solve a specific problem at scale. To make this concrete, consider a few examples: Trip planning assistants that help travelers turn vague ideas ("somewhere warm in April with good hiking") into concrete, bookable itineraries. On-site helpers that turn property details, amenities, reviews, and options into plain-language guidance, so people can choose the right stay with confidence. Partner copilots that help accommodation partners and other suppliers respond to guest messages faster and more consistently, while still staying in control of the final reply. Ranking systems that decide which options to show first in search and recommendation to surfaces, balancing user relevance with experimentation needs. Fraud detection models that quietly protect customers and partners in the background by flagging suspicious activity before it turns into real harm. Each of these systems is built and iterated on by different teams, uses different data, and runs under different constraints such as real-time vs batch, strict latency budgets vs more relaxed ones, fully automated vs human-in-the-loop. As we scale this ecosystem, observability becomes a first-class requirement, not a nice-to-have as we need to: Know when something changes in the real world, a new travel pattern, a data quality issue, a misconfiguration and how that affects model behaviour and user experience. Detect regressions early: slower responses, more confusing answers, drops in relevance or conversion, or subtle shifts that only show up for specific geographies, devices, or use ## Making agentic token costs visible in production DevFeed: [Making agentic token costs visible in production]() Original publisher: [Read original article]() Author: Natasha Silva Published: 2026-07-10T00:00:00Z Content type: tutorial Language: en Sources: [Datadog | The Monitor blog]() Topics: [AI Bots]() Tags: [agent-observability](), [agentic](), [ai](), [ai-observability](), [cloud-cost-management](), [cost](), [finops](), [instrumentation](), [mcp](), [monitor](), [production](), [tools]() ### AI overview A guide to identifying, reducing, and monitoring token costs in production agentic systems. It explains how tool schemas, session history, and retrieval loops add recurring token usage, and recommends loading only the tools required for each task. ### Source excerpt Learn where agentic token costs come from, how to reduce them across tool definitions, session history, and retrieval loops, and how to monitor spend. ## Braintrust's Ankur Goyal: Code review doesn't cover prompts DevFeed: [Braintrust's Ankur Goyal: Code review doesn't cover prompts]() Original publisher: [Read original article]() Author: info@1password.com (Chris Fowler) Published: 2026-06-30T00:00:00Z Content type: article Language: en Sources: [Blog on 1Password Blog]() Topics: [ai observability](), [AI, ML & Data Engineering]() Tags: [agents](), [ai](), [ai-observability](), [developers](), [evaluation](), [podcasts](), [review](), [security]() ### AI overview A Zero-Shot Learning episode examines AI observability and the security risk of treating prompt changes differently from code changes. It argues that prompts can alter agent behavior, data exposure, tool use, and credential access without passing standard review controls. ### Source excerpt Zero-Shot Learning is a podcast about how AI gets built, secured, and deployed. Hosted by Nancy Wang, 1Password CTO, and Dev Tagare, Senior Director of Engineering at Google, it's a builder's view of the architecture and the decisions it takes to ship with AI. Ankur Goyal, Founder and CEO of Braintrust, which bills itself as "the AI observability platform," joined Zero-Shot Learning to talk about the problem every team shipping AI eventually faces, you can build something that works and then watch it quietly become something that doesn't. Braintrust sits in the iteration loop for AI products, helping teams trace production events, turn behavior into eval datasets, compare prompt or model changes, and catch regressions before they reach users. For teams building agents, quality depends on whether the system's behavior remains useful and safe as prompts, models, tools, and user inputs change. In this episode, what begins as a conversation about evaluation frameworks and production feedback loops reveals a security gap many teams may leave open: prompt changes are behavior-shaping production artifacts. They can change what agents do, what data they surface, which tools they call, and how they use access and credentials. Why don't prompt changes go through the same security review as code? "I think enterprise developers are more comfortable iterating quickly on prompts than they are changing the underlying code," Nancy said. Before code is sent to production, a developer commits to version control, opens a pull request, waits for peer review, passes automated security scans, and gets sign-off before anything merges. The process documents who changed what, when, and why. By contrast, prompt changes often live outside the codebase, stored in a database row, prompt-management tool, or platform dashboard that a product manager or operations team can update directly. There are often no pull requests for review, no security scans, and sometimes even no recorded change a secur ## Debug and evaluate your AI app from your coding agent with Datadog Agent Observability DevFeed: [Debug and evaluate your AI app from your coding agent with Datadog Agent Observability]() Original publisher: [Read original article]() Author: Michael Bevilacqua-Linn; Till W; Tanguy Renaudie; Mehul Sonowal; Gabriele Lorenzo; Alex Barksdale Published: 2026-06-30T00:00:00Z Content type: article Language: en Sources: [Datadog | The Monitor blog]() Topics: [agent observability](), [AI-assisted coding](), [AI Engineering](), [observability](), [MCP](), [MCP Server](), [Command-line interface](), [LLM evaluation / benchmarking](), [datasets](), [Traces](), [debug](), [experiments]() Tags: [agent-observability](), [agent-skills](), [agents](), [ai](), [ai-engineering](), [ai-observability](), [analysis](), [api](), [claude](), [claude-code](), [cli](), [code](), [codex](), [coding](), [command-line](), [cursor](), [datasets](), [debug](), [eval](), [how-to](), [observability]() ### AI overview This article explains how to use Datadog Agent Observability from coding agents such as Claude Code, Cursor, and Codex CLI. It presents the Datadog MCP Server, Pup CLI, and Agent Skills as ways to access traces, evaluation results, experiment metrics, and other telemetry for classifying sessions, debugging production failures, creating evaluation datasets, and generating fixes. ### Source excerpt Learn how to give your coding agent access to Datadog Agent Observability data to classify failures, run RCA, bootstrap evaluators, and generate fixes. ## Datadog achieves GovRAMP High authorization DevFeed: [Datadog achieves GovRAMP High authorization]() Original publisher: [Read original article]() Author: Rukshan Gunawardana; Sophie Wang; Jason Hansen Published: 2026-06-29T00:00:00Z Content type: release Language: en Sources: [Datadog | The Monitor blog]() Topics: [Security](), [observability](), [Monitoring](), [systems](), [Cloud](), [Resilience](), [telemetry](), [ai observability]() Tags: [agent-observability](), [ai-observability](), [aws](), [cloud](), [dashboards](), [database-monitoring](), [fedramp](), [government](), [infrastructure-monitoring](), [log-management](), [logs](), [metrics](), [monitoring](), [networks](), [observability](), [resilience](), [security](), [systems](), [telemetry]() ### AI overview Datadog announced GovRAMP High authorization for its government offering, extending its FedRAMP High certification and supporting state, local, tribal, territorial, and education organizations. The article explains how unified observability and security can consolidate monitoring data across applications, logs, networks, databases, infrastructure, and AI initiatives. ### Source excerpt Datadog achieves GovRAMP High authorization, bringing unified observability and security to state and local government agencies for critical systems. ## Using Evaluation Frameworks with Agent Observability DevFeed: [Using Evaluation Frameworks with Agent Observability]() Original publisher: [Read original article]() Author: Jennifer Mickel; Eddie Cai Published: 2026-06-22T00:00:00Z Content type: article Language: en Sources: [Datadog | The Monitor blog]() Topics: [agent observability](), [observability](), [Large Language Model](), [Pydantic](), [experiments](), [Traces](), [Retrieval Augmented Generation (RAG)]() Tags: [agent-observability](), [ai-observability](), [code](), [deployment](), [development](), [evals](), [integration](), [llm](), [observability](), [traces]() ### AI overview This article explains how Datadog Agent Observability integrates existing DeepEval and Pydantic Evals frameworks. It covers running evaluations in experiments, connecting evaluation scores to production traces, and continuously monitoring LLM evaluation quality across development and deployment. ### Source excerpt Run DeepEval and Pydantic Evals natively in Datadog Agent Observability. Track regressions and connect eval scores to production traces. ## DASH 2026 Harnessing AI: Guide to Datadog's newest announcements DevFeed: [DASH 2026 Harnessing AI: Guide to Datadog's newest announcements]() Original publisher: [Read original article]() Author: Datadog Published: 2026-06-09T00:00:00Z Content type: release Language: en Sources: [Datadog | The Monitor blog]() Topics: [AI Chat](), [dashboards]() Tags: [agentic](), [ai](), [ai-observability](), [announcements](), [autonomous](), [bits-ai](), [coding](), [conversational-ai](), [dash](), [dashboards](), [incident](), [logs](), [mcp](), [metrics](), [observability](), [telemetry](), [traces]() ### AI overview A DASH 2026 release roundup covering Datadog AI features, including Bits Chat for natural-language search, incident investigation, and dashboard generation. ### Source excerpt A roundup of everything we announced at DASH 2026, including Datadog MCP Apps, Bits Code, AI Observability, and Pup CLI. ## Introducing AI observability notebooks for Managed ClickStack in Private Preview DevFeed: [Introducing AI observability notebooks for Managed ClickStack in Private Preview]() Original publisher: [Read original article]() Author: Mike Shi Published: 2026-03-18T00:00:00Z Content type: article Language: en Sources: [ClickHouse Blog]() Topics: [ai observability](), [observability](), [Claude](), [clickhouse](), [log management](), [Grafana Cloud Metrics](), [Traces](), [anthropic]() Tags: [ai](), [ai-observability](), [anthropic](), [clickhouse](), [logs](), [metrics](), [observability](), [production](), [sre](), [traces]() ### AI overview ClickStack AI Notebooks bring AI-assisted investigation to Managed ClickStack in ClickHouse Cloud. Engineers can investigate logs, metrics, and traces in a persistent, structured workspace, using natural-language questions, editable ClickHouse queries, intermediate reasoning, and branching investigations while retaining control of the workflow. ### Source excerpt AI-native observability notebooks for Managed ClickStack let engineers investigate logs, metrics, and traces with AI as a collaborator -- keeping humans in control at every step. ## How Raindrop Scales AI Observability with Tinybird's Real-Time Analytics Infrastructure DevFeed: [How Raindrop Scales AI Observability with Tinybird's Real-Time Analytics Infrastructure]() Original publisher: [Read original article]() Author: Tinybird Published: 2025-04-24T00:00:00Z Content type: article Language: en Sources: [Tinybird]() Topics: [ai observability](), [AI Development](), [real-time](), [observability]() Tags: [ai](), [ai-observability](), [customer-stories](), [net-11](), [observability](), [real-time]() ### AI overview This case study describes how Raindrop, an AI observability platform, processes hundreds of millions of requests daily using Tinybird's real-time analytics infrastructure to help companies understand interactions with their AI applications. ### Source excerpt Learn how this AI observability platform processes hundreds of millions of requests daily to help companies understand how users interact with their AI applications, powered by Tinybird's real-time analytics infrastructure.