# Monte Carlo

Trust your agents in production

This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles.

## Agent Observability Example: Tracing One Bad Answer to Its Root Cause

DevFeed: [Agent Observability Example: Tracing One Bad Answer to Its Root Cause](<https://devfeed.tech/articles/agent-observability-example-tracing-one-bad-answer-to-its-root-cause-146867.md>)

Original publisher: [Read original article](<https://montecarlo.ai/blog-agent-observability-example>)

Author: Virna Sekuj

Published: 2026-11-20T19:19:28Z

Content type: tutorial

Language: en

Sources: [Monte Carlo](<https://devfeed.tech/sources/monte-carlo.md>)

Topics: [agent observability](<https://devfeed.tech/topics/agent-observability.md>), [tracing](<https://devfeed.tech/topics/tracing.md>), [OpenTelemetry](<https://devfeed.tech/topics/opentelemetry.md>)

Tags: [agent-observability](<https://devfeed.tech/tags/agent-observability.md>), [agent-trace-span-example](<https://devfeed.tech/tags/agent-trace-span-example.md>), [agent-tracing-example](<https://devfeed.tech/tags/agent-tracing-example.md>), [ai-agent-observability-example](<https://devfeed.tech/tags/ai-agent-observability-example.md>), [ai-observability](<https://devfeed.tech/tags/ai-observability.md>), [behavior](<https://devfeed.tech/tags/behavior.md>), [context](<https://devfeed.tech/tags/context.md>), [context-layer](<https://devfeed.tech/tags/context-layer.md>), [example](<https://devfeed.tech/tags/example.md>), [llm-agent-monitoring-example](<https://devfeed.tech/tags/llm-agent-monitoring-example.md>), [performance](<https://devfeed.tech/tags/performance.md>), [root-cause](<https://devfeed.tech/tags/root-cause.md>), [tracing](<https://devfeed.tech/tags/tracing.md>), [what-to-monitor-in-ai-agents](<https://devfeed.tech/tags/what-to-monitor-in-ai-agents.md>)

### AI overview

In the hypothetical billing-agent case, customers got a 30-day refund answer because the vector index served an outdated policy, even though latency, tool behavior, and the judge's faithfulness score looked healthy. Tracing retrieval lineage upstream revealed that a CMS field rename caused the nightly embedding job to write zero rows while reporting success, freezing the index on policy v7. The described resolution updates the job, backfills missed content, adds freshness, volume, and schema-change monitoring linked through lineage, then replays affected questions to verify the agent cites v8 and answers "14 days." The example recommends monitoring context, performance, behavior, and output, and notes that answer evaluations inherit the blind spots of retrieved data.

### Source excerpt

A step-by-step agent observability example: how one bad answer was traced across context, performance, behavior, and output to its root cause, and fixed.

## What Is a Vector Database? How It Works and Why AI Needs One

DevFeed: [What Is a Vector Database? How It Works and Why AI Needs One](<https://devfeed.tech/articles/what-is-a-vector-database-how-it-works-and-why-ai-needs-one-145883.md>)

Original publisher: [Read original article](<https://montecarlo.ai/blog-what-is-a-vector-database>)

Author: Virna Sekuj

Published: 2026-11-19T17:18:16Z

Content type: article

Language: en

Sources: [Monte Carlo](<https://devfeed.tech/sources/monte-carlo.md>)

Topics: [Vectors & LLMs](<https://devfeed.tech/topics/vectors-llms.md>), [Embeddings](<https://devfeed.tech/topics/embeddings.md>), [vector embeddings in AI](<https://devfeed.tech/topics/vector-embeddings-in-ai.md>), [Retrieval Augmented Generation (RAG)](<https://devfeed.tech/topics/retrieval-augmented-generation-rag.md>), [ann](<https://devfeed.tech/topics/ann.md>), [Database](<https://devfeed.tech/topics/database.md>), [information retrieval](<https://devfeed.tech/topics/information-retrieval.md>), [hybrid-search](<https://devfeed.tech/topics/hybrid-search.md>)

Tags: [agent-trust](<https://devfeed.tech/tags/agent-trust.md>), [agent-trust-glossary](<https://devfeed.tech/tags/agent-trust-glossary.md>), [agents](<https://devfeed.tech/tags/agents.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-agents](<https://devfeed.tech/tags/ai-agents.md>), [data](<https://devfeed.tech/tags/data.md>), [database](<https://devfeed.tech/tags/database.md>), [embeddings](<https://devfeed.tech/tags/embeddings.md>), [glossary](<https://devfeed.tech/tags/glossary.md>), [rag](<https://devfeed.tech/tags/rag.md>), [rag-systems](<https://devfeed.tech/tags/rag-systems.md>), [retrieval](<https://devfeed.tech/tags/retrieval.md>), [retrieval-augmented-generation](<https://devfeed.tech/tags/retrieval-augmented-generation.md>), [retrieval-augmented-generation-rag](<https://devfeed.tech/tags/retrieval-augmented-generation-rag.md>), [vector](<https://devfeed.tech/tags/vector.md>), [vector-database](<https://devfeed.tech/tags/vector-database.md>), [vector-database-examples](<https://devfeed.tech/tags/vector-database-examples.md>), [vector-database-for-rag](<https://devfeed.tech/tags/vector-database-for-rag.md>), [vector-database-vs-relational-database](<https://devfeed.tech/tags/vector-database-vs-relational-database.md>), [vector-search](<https://devfeed.tech/tags/vector-search.md>), [vector-store](<https://devfeed.tech/tags/vector-store.md>)

### AI overview

Vector databases store content embeddings and use similarity search to retrieve relevant passages even when query wording differs. In a typical RAG flow, source content is chunked and embedded, stored with metadata, and indexed for approximate nearest-neighbor search; the user query is embedded and matching chunks are passed to the LLM as context. ANN indexing trades a small amount of precision for speed, and metadata filters can narrow results. Options include dedicated databases, database extensions, and vector search built into data platforms; choice depends on scale, latency, data location, and operational overhead. Retrieval can silently degrade when source syncs become stale, ingestion or chunking breaks, embedding models change without re-embedding, duplicate versions compete, or permission filters are missing. The text recommends monitoring freshness and ingestion volume, checking chunk quality, tracking embedding versions, measuring relevance and grounding, and tracing which chunks each agent request retrieves.

### Source excerpt

A vector database is a database built to store embeddings, which are numeric representations of the meaning of text, images, or other data, and to quickly find the embeddings most similar to a query. It's the retrieval layer behind most retrieval-augmented generation (RAG) systems and many AI agents, letting them find relevant information by meaning instead of exact keywords.

## What Is Tool Calling? How LLMs and AI Agents Take Action

DevFeed: [What Is Tool Calling? How LLMs and AI Agents Take Action](<https://devfeed.tech/articles/what-is-tool-calling-how-llms-and-ai-agents-take-action-145886.md>)

Original publisher: [Read original article](<https://montecarlo.ai/blog-what-is-tool-calling>)

Author: Virna Sekuj

Published: 2026-11-19T16:58:24Z

Content type: article

Language: en

Sources: [Monte Carlo](<https://devfeed.tech/sources/monte-carlo.md>)

Topics: [function calling](<https://devfeed.tech/topics/function-calling.md>), [Multi-step AI processes](<https://devfeed.tech/topics/multi-step-ai-processes.md>), [LLM observability](<https://devfeed.tech/topics/llm-observability.md>), [API](<https://devfeed.tech/topics/api.md>), [Tool](<https://devfeed.tech/topics/tool.md>)

Tags: [2023](<https://devfeed.tech/tags/2023.md>), [2026](<https://devfeed.tech/tags/2026.md>), [actions](<https://devfeed.tech/tags/actions.md>), [agent](<https://devfeed.tech/tags/agent.md>), [agent-behavior](<https://devfeed.tech/tags/agent-behavior.md>), [agent-frameworks](<https://devfeed.tech/tags/agent-frameworks.md>), [agent-trust](<https://devfeed.tech/tags/agent-trust.md>), [agent-trust-glossary](<https://devfeed.tech/tags/agent-trust-glossary.md>), [agents](<https://devfeed.tech/tags/agents.md>), [ai](<https://devfeed.tech/tags/ai.md>), [api](<https://devfeed.tech/tags/api.md>), [database](<https://devfeed.tech/tags/database.md>), [function](<https://devfeed.tech/tags/function.md>), [function-calling](<https://devfeed.tech/tags/function-calling.md>), [glossary](<https://devfeed.tech/tags/glossary.md>), [large-language-model](<https://devfeed.tech/tags/large-language-model.md>), [llm-tool-calling](<https://devfeed.tech/tags/llm-tool-calling.md>), [llms](<https://devfeed.tech/tags/llms.md>), [querying](<https://devfeed.tech/tags/querying.md>), [searching](<https://devfeed.tech/tags/searching.md>), [structured](<https://devfeed.tech/tags/structured.md>), [tool-calling](<https://devfeed.tech/tags/tool-calling.md>), [tool-calling-examples](<https://devfeed.tech/tags/tool-calling-examples.md>), [tool-calling-vs-mcp](<https://devfeed.tech/tags/tool-calling-vs-mcp.md>), [tool-use-ai-agents](<https://devfeed.tech/tags/tool-use-ai-agents.md>), [uses](<https://devfeed.tech/tags/uses.md>)

### AI overview

Tool calling lets an LLM request an action by returning a structured tool name and arguments; the application validates and executes the request, sends back the result, and the model uses it to answer or make another call. Function calling and tool use are alternate names for the same mechanism. MCP standardizes how compatible applications discover and connect to tools; it builds on tool calling rather than replacing it. Calls can fail through wrong tool choice, bad arguments, skipped checks, retries, mishandled errors, changed responses, or unintended access, often without obvious errors. The text recommends tracing each call and its result, validating arguments and permissions before execution, watching for unexpected patterns and loops, tracking cost and latency, monitoring the underlying data, and limiting or requiring approval for high-impact actions.

### Source excerpt

Tool calling, also called function calling, is how a large language model asks an application to run a function, such as querying a database, calling an API, or searching documents, and then uses the result in its answer. The model doesn't run the tool itself. It produces a structured request naming the tool and its arguments, the application executes it, and the result goes back to the model.

## What Is an LLM? Large Language Models Explained

DevFeed: [What Is an LLM? Large Language Models Explained](<https://devfeed.tech/articles/what-is-an-llm-large-language-models-explained-145885.md>)

Original publisher: [Read original article](<https://montecarlo.ai/blog-what-is-an-llm-large-language-models-explained>)

Author: Virna Sekuj

Published: 2026-11-19T16:43:33Z

Content type: article

Language: en

Sources: [Monte Carlo](<https://devfeed.tech/sources/monte-carlo.md>)

Topics: [Large Language Model](<https://devfeed.tech/topics/llm.md>), [LLM observability](<https://devfeed.tech/topics/llm-observability.md>), [LLM evaluation / benchmarking](<https://devfeed.tech/topics/llm-evaluation-benchmarking.md>), [Retrieval Augmented Generation (RAG)](<https://devfeed.tech/topics/retrieval-augmented-generation-rag.md>), [context window](<https://devfeed.tech/topics/context-window.md>), [knowledge cutoff](<https://devfeed.tech/topics/knowledge-cutoff.md>), [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>)

Tags: [agent-trust](<https://devfeed.tech/tags/agent-trust.md>), [agent-trust-glossary](<https://devfeed.tech/tags/agent-trust-glossary.md>), [agents](<https://devfeed.tech/tags/agents.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-agent](<https://devfeed.tech/tags/ai-agent.md>), [ai-agents](<https://devfeed.tech/tags/ai-agents.md>), [architecture](<https://devfeed.tech/tags/architecture.md>), [attention](<https://devfeed.tech/tags/attention.md>), [chatgpt](<https://devfeed.tech/tags/chatgpt.md>), [claude](<https://devfeed.tech/tags/claude.md>), [code-generation](<https://devfeed.tech/tags/code-generation.md>), [coding-assistants](<https://devfeed.tech/tags/coding-assistants.md>), [context](<https://devfeed.tech/tags/context.md>), [context-window](<https://devfeed.tech/tags/context-window.md>), [glossary](<https://devfeed.tech/tags/glossary.md>), [how-do-llms-work](<https://devfeed.tech/tags/how-do-llms-work.md>), [language-models](<https://devfeed.tech/tags/language-models.md>), [large-language-model](<https://devfeed.tech/tags/large-language-model.md>), [large-language-models](<https://devfeed.tech/tags/large-language-models.md>), [llm](<https://devfeed.tech/tags/llm.md>), [llm-examples](<https://devfeed.tech/tags/llm-examples.md>), [llm-limitations](<https://devfeed.tech/tags/llm-limitations.md>), [llm-meaning](<https://devfeed.tech/tags/llm-meaning.md>), [llm-vs-ai-agent](<https://devfeed.tech/tags/llm-vs-ai-agent.md>), [neural-network](<https://devfeed.tech/tags/neural-network.md>)

### AI overview

LLMs generate text by predicting one token at a time, and their answers depend heavily on the context supplied at runtime. In production, fluent output can still be wrong because of hallucinations, stale or incomplete context, incorrect tool use, or provider model changes; failures may not show up in uptime or error rates. The excerpt recommends recording traces of prompts, retrieved data, tool calls, outputs, latency, token counts, and model versions; evaluating accuracy, relevance, and groundedness; checking context freshness and completeness; and tracking cost and performance. Comparing traces and data lineage can help distinguish a context problem from a model change. The excerpt does not cover full training stages in detail.

### Source excerpt

A large language model (LLM) is a neural network trained on large amounts of text to understand and generate human language. LLMs predict the most likely next piece of text, one token at a time, which lets them answer questions, write and summarize documents, generate code, and power AI agents that take actions.

## What Is an Embedding? How AI Turns Meaning Into Numbers

DevFeed: [What Is an Embedding? How AI Turns Meaning Into Numbers](<https://devfeed.tech/articles/what-is-an-embedding-how-ai-turns-meaning-into-numbers-141932.md>)

Original publisher: [Read original article](<https://montecarlo.ai/blog-embedding>)

Author: Ruslan Sultanov

Published: 2026-11-18T18:58:12Z

Content type: article

Language: en

Sources: [Monte Carlo](<https://devfeed.tech/sources/monte-carlo.md>)

Topics: [vector embeddings in AI](<https://devfeed.tech/topics/vector-embeddings-in-ai.md>), [Embeddings](<https://devfeed.tech/topics/embeddings.md>), [cosine similarity search](<https://devfeed.tech/topics/cosine-similarity-search.md>), [Data Quality](<https://devfeed.tech/topics/data-quality.md>), [AI Context Management](<https://devfeed.tech/topics/ai-context-management.md>), [clustering](<https://devfeed.tech/topics/clustering.md>)

Tags: [agent-trust](<https://devfeed.tech/tags/agent-trust.md>), [agent-trust-glossary](<https://devfeed.tech/tags/agent-trust-glossary.md>), [ai](<https://devfeed.tech/tags/ai.md>), [cosine-similarity](<https://devfeed.tech/tags/cosine-similarity.md>), [embedding](<https://devfeed.tech/tags/embedding.md>), [embedding-model](<https://devfeed.tech/tags/embedding-model.md>), [embeddings](<https://devfeed.tech/tags/embeddings.md>), [embeddings-in-machine-learning](<https://devfeed.tech/tags/embeddings-in-machine-learning.md>), [embeddings-rag](<https://devfeed.tech/tags/embeddings-rag.md>), [glossary](<https://devfeed.tech/tags/glossary.md>), [meaning](<https://devfeed.tech/tags/meaning.md>), [text-embeddings](<https://devfeed.tech/tags/text-embeddings.md>), [vector-database](<https://devfeed.tech/tags/vector-database.md>), [vector-embeddings](<https://devfeed.tech/tags/vector-embeddings.md>), [what-is-an-embedding](<https://devfeed.tech/tags/what-is-an-embedding.md>)

### AI overview

Embeddings represent text, images, or other data as numeric vectors, placing semantically similar items near each other so systems can retrieve by meaning rather than exact wording. In a typical RAG pipeline, documents are split into chunks, embedded and indexed with metadata; a query is embedded to retrieve nearby passages for an LLM. Vectors are meaningful only within the model that produced them, and they become outdated when source content changes. Production reliability therefore depends on versioning vectors, re-embedding changed documents, removing deleted content, monitoring index freshness and evaluating and tracing retrieval. Stale or missing passages can lead an agent to give confident but outdated answers.

### Source excerpt

A customer types "can I send this back?" Your help center never uses that phrase. It talks about the "return window" and the "refund policy." Keyword search finds nothing. Yet a modern AI agent pulls up the right policy in milliseconds, because it isn't matching words. It's matching meaning. That's what embeddings do. This guide ...

## What Is Context Engineering? Giving AI Agents the Right Information

DevFeed: [What Is Context Engineering? Giving AI Agents the Right Information](<https://devfeed.tech/articles/what-is-context-engineering-giving-ai-agents-the-right-information-141931.md>)

Original publisher: [Read original article](<https://montecarlo.ai/blog-context-engineering>)

Author: Ruslan Sultanov

Published: 2026-11-18T18:46:57Z

Content type: article

Language: en

Sources: [Monte Carlo](<https://devfeed.tech/sources/monte-carlo.md>)

Topics: [AI Context Management](<https://devfeed.tech/topics/ai-context-management.md>), [context window](<https://devfeed.tech/topics/context-window.md>), [Retrieval Augmented Generation (RAG)](<https://devfeed.tech/topics/retrieval-augmented-generation-rag.md>), [Prompt Engineering](<https://devfeed.tech/topics/prompt-engineering.md>), [AI-Ready Data](<https://devfeed.tech/topics/ai-ready-data.md>), [LLM observability](<https://devfeed.tech/topics/llm-observability.md>), [context](<https://devfeed.tech/topics/context.md>)

Tags: [agent-memory](<https://devfeed.tech/tags/agent-memory.md>), [agent-trust](<https://devfeed.tech/tags/agent-trust.md>), [agent-trust-glossary](<https://devfeed.tech/tags/agent-trust-glossary.md>), [agents](<https://devfeed.tech/tags/agents.md>), [ai](<https://devfeed.tech/tags/ai.md>), [context-engineering](<https://devfeed.tech/tags/context-engineering.md>), [context-engineering-for-ai-agents](<https://devfeed.tech/tags/context-engineering-for-ai-agents.md>), [context-engineering-vs-prompt-engineering](<https://devfeed.tech/tags/context-engineering-vs-prompt-engineering.md>), [context-engineering-vs-rag](<https://devfeed.tech/tags/context-engineering-vs-rag.md>), [context-window](<https://devfeed.tech/tags/context-window.md>), [context-window-management](<https://devfeed.tech/tags/context-window-management.md>), [data](<https://devfeed.tech/tags/data.md>), [data-pipeline](<https://devfeed.tech/tags/data-pipeline.md>), [data-quality](<https://devfeed.tech/tags/data-quality.md>), [glossary](<https://devfeed.tech/tags/glossary.md>), [llm-context](<https://devfeed.tech/tags/llm-context.md>), [prompts](<https://devfeed.tech/tags/prompts.md>), [what-is-context-engineering](<https://devfeed.tech/tags/what-is-context-engineering.md>)

### AI overview

Context engineering shapes everything an agent receives--instructions, retrieved data, tool results, and memory--so it has a small, relevant set of information at each step. Prompt engineering focuses on instruction wording; RAG fetches documents for the prompt; context engineering encompasses both, along with tool design, memory, and what to omit. Techniques include just-in-time retrieval, reranking and filtering, compressing long outputs, using sub-agents with separate context, storing notes externally, and structuring inputs. These cannot correct stale or faulty source data. The excerpt recommends monitoring freshness and volume, tracing outputs to source documents and versions, recording retrievals, and evaluating groundedness and correctness. It notes that larger context windows do not remove the need to select and manage information.

### Source excerpt

Two teams build the same support agent on the same model with nearly identical prompts. One answers correctly nine times out of ten. The other confidently quotes a shipping policy that changed in the spring. The difference isn't the model or the wording. It's what each agent was given to read before it answered. That's ...

## What Is an AI Agent? How Agents Work and Why They Fail

DevFeed: [What Is an AI Agent? How Agents Work and Why They Fail](<https://devfeed.tech/articles/what-is-an-ai-agent-how-agents-work-and-why-they-fail-141928.md>)

Original publisher: [Read original article](<https://montecarlo.ai/blog-ai-agent>)

Author: Ruslan Sultanov

Published: 2026-11-18T18:39:05Z

Content type: article

Language: en

Sources: [Monte Carlo](<https://devfeed.tech/sources/monte-carlo.md>)

Topics: [AI Agent](<https://devfeed.tech/topics/ai-agent.md>), [agent observability](<https://devfeed.tech/topics/agent-observability.md>), [AI Chat](<https://devfeed.tech/topics/ai-chat.md>), [Multi Agent Systems](<https://devfeed.tech/topics/multi-agent-systems.md>), [agent orchestration](<https://devfeed.tech/topics/agent-orchestration.md>), [AI for customer service automation](<https://devfeed.tech/topics/ai-for-customer-service-automation.md>), [supervisor agent pattern](<https://devfeed.tech/topics/supervisor-agent-pattern.md>), [human in the loop agent](<https://devfeed.tech/topics/human-in-the-loop-agent.md>)

Tags: [actions](<https://devfeed.tech/tags/actions.md>), [adoption](<https://devfeed.tech/tags/adoption.md>), [agent](<https://devfeed.tech/tags/agent.md>), [agent-trust](<https://devfeed.tech/tags/agent-trust.md>), [agent-trust-glossary](<https://devfeed.tech/tags/agent-trust-glossary.md>), [agentic-ai](<https://devfeed.tech/tags/agentic-ai.md>), [agents-in-production](<https://devfeed.tech/tags/agents-in-production.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-agent](<https://devfeed.tech/tags/ai-agent.md>), [ai-agent-examples](<https://devfeed.tech/tags/ai-agent-examples.md>), [ai-agent-vs-chatbot](<https://devfeed.tech/tags/ai-agent-vs-chatbot.md>), [ai-agent-vs-workflow](<https://devfeed.tech/tags/ai-agent-vs-workflow.md>), [ai-agents-explained](<https://devfeed.tech/tags/ai-agents-explained.md>), [chatbot](<https://devfeed.tech/tags/chatbot.md>), [customer](<https://devfeed.tech/tags/customer.md>), [days](<https://devfeed.tech/tags/days.md>), [fail](<https://devfeed.tech/tags/fail.md>), [glossary](<https://devfeed.tech/tags/glossary.md>), [how-do-ai-agents-work](<https://devfeed.tech/tags/how-do-ai-agents-work.md>), [jacket](<https://devfeed.tech/tags/jacket.md>), [loyalty](<https://devfeed.tech/tags/loyalty.md>), [order](<https://devfeed.tech/tags/order.md>), [policy](<https://devfeed.tech/tags/policy.md>), [return](<https://devfeed.tech/tags/return.md>), [table](<https://devfeed.tech/tags/table.md>), [types-of-ai-agents](<https://devfeed.tech/tags/types-of-ai-agents.md>), [what-is-an-ai-agent](<https://devfeed.tech/tags/what-is-an-ai-agent.md>), [window](<https://devfeed.tech/tags/window.md>)

### AI overview

AI agents use an LLM to choose steps, call tools, and adapt toward a goal, so they can act across systems rather than only reply. Compared with chatbots, they take actions; compared with code-defined LLM workflows, they are more flexible but harder to test and less predictable. The text recommends using the least autonomy needed, since fixed workflows can be cheaper and easier to debug. Production failures often stem from stale or irrelevant context, incorrect tool use, behavior drift, or excessive permissions. Suggested safeguards include tracing model and tool calls, monitoring source data, evaluating live outputs and tool paths, limiting permissions, requiring human approval for some actions, and assigning an owner. Multi-agent systems can divide larger tasks among specialists, but add failure points and require tracing across agents.

### Source excerpt

A customer asks whether they can still return a jacket. A chatbot would answer from what it was trained on. An AI agent looks up the order, checks the return policy table, sees the purchase was 34 days ago, notices the customer is a loyalty member with a 45-day window, and starts the return. Nobody ...

## What Is LLM Monitoring? What to Track Once Your Model Is Live

DevFeed: [What Is LLM Monitoring? What to Track Once Your Model Is Live](<https://devfeed.tech/articles/what-is-llm-monitoring-what-to-track-once-your-model-is-live-141933.md>)

Original publisher: [Read original article](<https://montecarlo.ai/blog-llm-monitoring>)

Author: Ruslan Sultanov

Published: 2026-11-18T18:07:29Z

Content type: article

Language: en

Sources: [Monte Carlo](<https://devfeed.tech/sources/monte-carlo.md>)

Topics: [LLM observability](<https://devfeed.tech/topics/llm-observability.md>), [Monitoring](<https://devfeed.tech/topics/monitoring.md>), [ai observability](<https://devfeed.tech/topics/ai-observability.md>), [Monitoring & Alerting](<https://devfeed.tech/topics/monitoring-alerting.md>), [data observability](<https://devfeed.tech/topics/data-observability.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>)

Tags: [agent-trust](<https://devfeed.tech/tags/agent-trust.md>), [agent-trust-glossary](<https://devfeed.tech/tags/agent-trust-glossary.md>), [glossary](<https://devfeed.tech/tags/glossary.md>), [llm](<https://devfeed.tech/tags/llm.md>), [llm-monitoring](<https://devfeed.tech/tags/llm-monitoring.md>), [llm-monitoring-metrics](<https://devfeed.tech/tags/llm-monitoring-metrics.md>), [llm-monitoring-vs-observability](<https://devfeed.tech/tags/llm-monitoring-vs-observability.md>), [llm-performance-monitoring](<https://devfeed.tech/tags/llm-performance-monitoring.md>), [llm-quality-monitoring](<https://devfeed.tech/tags/llm-quality-monitoring.md>), [model](<https://devfeed.tech/tags/model.md>), [monitoring](<https://devfeed.tech/tags/monitoring.md>), [monitoring-llms-in-production](<https://devfeed.tech/tags/monitoring-llms-in-production.md>), [what-is-llm-monitoring](<https://devfeed.tech/tags/what-is-llm-monitoring.md>)

### AI overview

LLM monitoring catches production failures that uptime, latency, and error dashboards miss: an app can be fast and available while returning incorrect or unsafe answers. It continuously tracks performance (latency, tokens, cost, errors), context (source freshness and retrieval), behavior (tool calls, retries, refusals), and output quality (groundedness, relevance, correctness, safety, and user feedback). A practical setup traces requests across retrieval, model, and tool calls; scores traffic with deterministic checks and LLM-as-judge evaluations; alerts on trends or anomalies; then uses trace context to investigate causes such as stale data, prompt changes, or model updates. Evals assess fixed examples before release, monitoring detects production changes, and observability supplies traces and lineage to explain them. The article recommends versioning traces, establishing baselines, checking judges against human review, and assigning alert owners.

### Source excerpt

Your LLM app has been live for a month. Latency is under a second, the error rate is zero, and uptime is perfect. Meanwhile, a product manager forwards a screenshot: the bot told a customer about a discount that ended in March. Nothing on the dashboard moved. That's the gap LLM monitoring fills. This guide ...

## What Is Agent Tracing? How to See Every Step an Agent Takes

DevFeed: [What Is Agent Tracing? How to See Every Step an Agent Takes](<https://devfeed.tech/articles/what-is-agent-tracing-how-to-see-every-step-an-agent-takes-141927.md>)

Original publisher: [Read original article](<https://montecarlo.ai/blog-agent-tracing>)

Author: Ruslan Sultanov

Published: 2026-11-18T17:20:39Z

Content type: article

Language: en

Sources: [Monte Carlo](<https://devfeed.tech/sources/monte-carlo.md>)

Topics: [ai observability](<https://devfeed.tech/topics/ai-observability.md>), [tracing](<https://devfeed.tech/topics/tracing.md>), [AI Agent](<https://devfeed.tech/topics/ai-agent.md>), [Multi-step AI processes](<https://devfeed.tech/topics/multi-step-ai-processes.md>), [Latency](<https://devfeed.tech/topics/latency.md>)

Tags: [accountability](<https://devfeed.tech/tags/accountability.md>), [agent](<https://devfeed.tech/tags/agent.md>), [agent-spans](<https://devfeed.tech/tags/agent-spans.md>), [agent-tracing](<https://devfeed.tech/tags/agent-tracing.md>), [agent-tracing-vs-logging](<https://devfeed.tech/tags/agent-tracing-vs-logging.md>), [agent-trust](<https://devfeed.tech/tags/agent-trust.md>), [agent-trust-glossary](<https://devfeed.tech/tags/agent-trust-glossary.md>), [ai-agent-tracing](<https://devfeed.tech/tags/ai-agent-tracing.md>), [black-box](<https://devfeed.tech/tags/black-box.md>), [customer](<https://devfeed.tech/tags/customer.md>), [glossary](<https://devfeed.tech/tags/glossary.md>), [how-to](<https://devfeed.tech/tags/how-to.md>), [latency](<https://devfeed.tech/tags/latency.md>), [llm-tracing](<https://devfeed.tech/tags/llm-tracing.md>), [logs](<https://devfeed.tech/tags/logs.md>), [model](<https://devfeed.tech/tags/model.md>), [opentelemetry-genai-semantic-conventions](<https://devfeed.tech/tags/opentelemetry-genai-semantic-conventions.md>), [order](<https://devfeed.tech/tags/order.md>), [request](<https://devfeed.tech/tags/request.md>), [response](<https://devfeed.tech/tags/response.md>), [service](<https://devfeed.tech/tags/service.md>), [tools](<https://devfeed.tech/tags/tools.md>), [trace-ai-agents](<https://devfeed.tech/tags/trace-ai-agents.md>), [tracing](<https://devfeed.tech/tags/tracing.md>), [what-is](<https://devfeed.tech/tags/what-is.md>), [what-is-agent-tracing](<https://devfeed.tech/tags/what-is-agent-tracing.md>)

### AI overview

Agent tracing records an AI agent's full run as linked spans, capturing steps such as planning, retrieval, model calls, and tool use alongside inputs, outputs, timing, and metadata. This makes silent failures, retries, quality issues, and cost or latency drivers easier to investigate than ordinary logs or service-focused distributed traces. The excerpt recommends OpenTelemetry GenAI conventions, trace IDs and parent-span IDs, versioning prompts and models, linking retrieved items to source data, and scoring traces. Capture sensitive prompts carefully: redact personal data and set retention policies. OpenTelemetry conventions remain in development, so pin versions. Traces also cannot reveal upstream data freshness problems without lineage or data monitoring. Teams can store traces in their own warehouse, but need tooling to query, score, and alert on them.

### Source excerpt

An agent tells a customer their order shipped. It didn't. Every service returned 200 OK, latency was normal, and the logs show a request in and a response out. What happened in between, which documents the agent read, which tools it called, what the model was told, is a black box. Agent tracing opens that ...

## What Is AI Accountability? Who Answers When Agents Fail

DevFeed: [What Is AI Accountability? Who Answers When Agents Fail](<https://devfeed.tech/articles/what-is-ai-accountability-who-answers-when-agents-fail-136089.md>)

Original publisher: [Read original article](<https://montecarlo.ai/blog-ai-accountability>)

Author: Ruslan Sultanov

Published: 2026-11-16T19:22:02Z

Content type: article

Language: en

Sources: [Monte Carlo](<https://devfeed.tech/sources/monte-carlo.md>)

Topics: [responsible-ai](<https://devfeed.tech/topics/responsible-ai.md>), [Post Mortem](<https://devfeed.tech/topics/post-mortem.md>), [human review](<https://devfeed.tech/topics/human-review.md>), [OpenTelemetry](<https://devfeed.tech/topics/opentelemetry.md>), [data ownership](<https://devfeed.tech/topics/data-ownership.md>), [human in the loop agent](<https://devfeed.tech/topics/human-in-the-loop-agent.md>)

Tags: [accountability](<https://devfeed.tech/tags/accountability.md>), [accountability-in-ai-agents](<https://devfeed.tech/tags/accountability-in-ai-agents.md>), [agent](<https://devfeed.tech/tags/agent.md>), [agent-trust](<https://devfeed.tech/tags/agent-trust.md>), [agent-trust-glossary](<https://devfeed.tech/tags/agent-trust-glossary.md>), [agents](<https://devfeed.tech/tags/agents.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-accountability](<https://devfeed.tech/tags/ai-accountability.md>), [ai-accountability-framework](<https://devfeed.tech/tags/ai-accountability-framework.md>), [ai-accountability-vs-ai-governance](<https://devfeed.tech/tags/ai-accountability-vs-ai-governance.md>), [ai-governance](<https://devfeed.tech/tags/ai-governance.md>), [business](<https://devfeed.tech/tags/business.md>), [customer](<https://devfeed.tech/tags/customer.md>), [data](<https://devfeed.tech/tags/data.md>), [data-team](<https://devfeed.tech/tags/data-team.md>), [don-t](<https://devfeed.tech/tags/don-t.md>), [engineers](<https://devfeed.tech/tags/engineers.md>), [eu-ai-act-record-keeping](<https://devfeed.tech/tags/eu-ai-act-record-keeping.md>), [exists](<https://devfeed.tech/tags/exists.md>), [fail](<https://devfeed.tech/tags/fail.md>), [glossary](<https://devfeed.tech/tags/glossary.md>), [incident](<https://devfeed.tech/tags/incident.md>), [nist-ai-rmf-accountability](<https://devfeed.tech/tags/nist-ai-rmf-accountability.md>), [other](<https://devfeed.tech/tags/other.md>), [point](<https://devfeed.tech/tags/point.md>), [policy](<https://devfeed.tech/tags/policy.md>), [post](<https://devfeed.tech/tags/post.md>), [post-incident-review](<https://devfeed.tech/tags/post-incident-review.md>), [what-is](<https://devfeed.tech/tags/what-is.md>), [who-is-responsible-when-ai-makes-a-mistake](<https://devfeed.tech/tags/who-is-responsible-when-ai-makes-a-mistake.md>)

### AI overview

AI accountability means naming who owns an agent's decisions, preserving evidence of how each run happened, and ensuring someone can correct the outcome. Agents make this harder because they take actions across systems, failures can originate with different teams, and technically successful runs may still be wrong. The document proposes four foundations: named agent and data owners; traces recording inputs, retrieved context, tool calls, versions, and outputs, linked to source data lineage; oversight through approval rules and least-privilege access; and recourse through stopping, rolling back, correcting records, and reviewing incidents. Its production steps include inventorying agents and owners, tracing runs with OpenTelemetry GenAI semantic conventions, connecting outputs to sources, evaluating production outcomes, setting approval and permission rules, and rehearsing rollback. It also notes Article 12 of the EU AI Act requires lifetime event logging for high-risk systems. The article reports survey figures on complaint-driven issue discovery and rollback limits, and presents Monte Carlo's platform and Bayes as a vendor-specific approach.

### Source excerpt

A support agent tells a customer they qualify for a refund they don't. The trace exists. The source table exists. In the post-incident review, three teams point at each other: the AI engineers who built the agent, the data team that owns the policy table, and the business unit that asked for the agent in ...

## How to Improve AI Security Posture Management (AI-SPM) Once Agents Go Live

DevFeed: [How to Improve AI Security Posture Management (AI-SPM) Once Agents Go Live](<https://devfeed.tech/articles/how-to-improve-ai-security-posture-management-ai-spm-once-agents-go-live-136090.md>)

Original publisher: [Read original article](<https://montecarlo.ai/blog-ai-security-posture-management>)

Author: Ruslan Sultanov

Published: 2026-11-16T19:04:54Z

Content type: article

Language: en

Sources: [Monte Carlo](<https://devfeed.tech/sources/monte-carlo.md>)

Topics: [ai security](<https://devfeed.tech/topics/ai-security.md>), [LLM security](<https://devfeed.tech/topics/llm-security.md>), [trust](<https://devfeed.tech/topics/trust.md>), [OpenTelemetry](<https://devfeed.tech/topics/opentelemetry.md>)

Tags: [agent](<https://devfeed.tech/tags/agent.md>), [agent-observability](<https://devfeed.tech/tags/agent-observability.md>), [agent-trust](<https://devfeed.tech/tags/agent-trust.md>), [agent-trust-glossary](<https://devfeed.tech/tags/agent-trust-glossary.md>), [agents](<https://devfeed.tech/tags/agents.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-agent](<https://devfeed.tech/tags/ai-agent.md>), [ai-agents](<https://devfeed.tech/tags/ai-agents.md>), [ai-security](<https://devfeed.tech/tags/ai-security.md>), [ai-security-posture](<https://devfeed.tech/tags/ai-security-posture.md>), [ai-security-posture-management](<https://devfeed.tech/tags/ai-security-posture-management.md>), [ai-spm](<https://devfeed.tech/tags/ai-spm.md>), [ai-standards](<https://devfeed.tech/tags/ai-standards.md>), [alert](<https://devfeed.tech/tags/alert.md>), [alerts](<https://devfeed.tech/tags/alerts.md>), [api](<https://devfeed.tech/tags/api.md>), [glossary](<https://devfeed.tech/tags/glossary.md>), [how-to](<https://devfeed.tech/tags/how-to.md>), [permissions](<https://devfeed.tech/tags/permissions.md>), [posture-management](<https://devfeed.tech/tags/posture-management.md>), [security](<https://devfeed.tech/tags/security.md>), [security-posture](<https://devfeed.tech/tags/security-posture.md>)

### AI overview

AI-SPM permission scans show what agents can access, but not what they actually do; a clean scan therefore cannot establish that an agent behaved safely. The excerpt recommends pairing access checks with agent run traces and data monitoring. Inventory models, agents, pipelines, connected services, and ranked risks, prioritizing what agents can change. Monitor agent-facing tables for volume, freshness, and schema shifts, using lineage to identify the tables agents really read and trace issues to their sources. To investigate a bad answer, inspect the full run trace, identify the retrieved table, follow its lineage, check monitors for alerts in the same period, and find other answers that used affected data. These signals cover different parts of agent activity: a table alert alone does not show what the agent did or which outputs were affected. The excerpt cites survey findings that many builders struggle to trace failures and often learn of problems from customer complaints. It describes combining posture scans, data observability, lineage, and traces as a way to assess both permitted access and production behavior.

### Source excerpt

Imagine you hire a bank teller. You check their references, confirm they have the right keys, and make sure they can't touch the safe-deposit boxes. Then you turn off the cameras and never count the cash. This is, roughly, how a lot of companies run their AI agents: the permissions have been audited very carefully, ...

## The Biggest Risk to Your AI Is Not in the Chat Box, and Your LLM Firewall Cannot See It

DevFeed: [The Biggest Risk to Your AI Is Not in the Chat Box, and Your LLM Firewall Cannot See It](<https://devfeed.tech/articles/the-biggest-risk-to-your-ai-is-not-in-the-chat-box-and-your-llm-firewall-cannot-see-it-136091.md>)

Original publisher: [Read original article](<https://montecarlo.ai/blog-llm-firewall>)

Author: Ruslan Sultanov

Published: 2026-11-16T18:12:53Z

Content type: article

Language: en

Sources: [Monte Carlo](<https://devfeed.tech/sources/monte-carlo.md>)

Topics: [LLM security](<https://devfeed.tech/topics/llm-security.md>), [ai observability](<https://devfeed.tech/topics/ai-observability.md>), [LLM observability](<https://devfeed.tech/topics/llm-observability.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>), [human review](<https://devfeed.tech/topics/human-review.md>)

Tags: [agent](<https://devfeed.tech/tags/agent.md>), [agent-observability](<https://devfeed.tech/tags/agent-observability.md>), [agent-trust](<https://devfeed.tech/tags/agent-trust.md>), [agent-trust-glossary](<https://devfeed.tech/tags/agent-trust-glossary.md>), [agents](<https://devfeed.tech/tags/agents.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-agent](<https://devfeed.tech/tags/ai-agent.md>), [api](<https://devfeed.tech/tags/api.md>), [api-keys](<https://devfeed.tech/tags/api-keys.md>), [background-jobs](<https://devfeed.tech/tags/background-jobs.md>), [bayes](<https://devfeed.tech/tags/bayes.md>), [chatbot](<https://devfeed.tech/tags/chatbot.md>), [cisa](<https://devfeed.tech/tags/cisa.md>), [firewall](<https://devfeed.tech/tags/firewall.md>), [glossary](<https://devfeed.tech/tags/glossary.md>), [language-model](<https://devfeed.tech/tags/language-model.md>), [large-language-model](<https://devfeed.tech/tags/large-language-model.md>), [llm](<https://devfeed.tech/tags/llm.md>), [llm-firewall](<https://devfeed.tech/tags/llm-firewall.md>)

### AI overview

An LLM firewall can inspect messages to and from a model, but it may miss stale or poisoned data retrieved by an agent and cannot stop risky actions the agent is authorized to take. The excerpt recommends routing every model call through the firewall, scanning inputs and outputs, testing policies in log-only mode before enforcement, and accounting for latency from output scanning. It also recommends limiting each agent's permissions, logging model inputs, outputs, and tool or API calls, monitoring tool-use patterns and false positives, and using run traces plus data lineage to investigate incidents. Sudden behavior changes may indicate tampering; gradual declines may reflect data drift. Monte Carlo's described approach combines data and agent observability with lineage to track sources, behavior, and outputs; the excerpt does not establish that these measures guarantee prevention.

### Source excerpt

An LLM firewall has a simple job. It sits between the people using your AI agent and the large language model (LLM) behind it, checking every message going in and every reply coming out so that it can block anything that looks like an attack, a leaked password, or an off-limits topic. If someone types ...

## The Other Kind of Context Rot: How Source Rot Corrupts LLM Performance Before Inference Even Begins

DevFeed: [The Other Kind of Context Rot: How Source Rot Corrupts LLM Performance Before Inference Even Begins](<https://devfeed.tech/articles/the-other-kind-of-context-rot-how-source-rot-corrupts-llm-performance-before-inference-even-begins-133883.md>)

Original publisher: [Read original article](<https://montecarlo.ai/blog-the-other-kind-of-context-rot-how-source-rot>)

Author: Barr Moses

Published: 2026-11-14T00:02:53Z

Content type: article

Language: en

Sources: [Monte Carlo](<https://devfeed.tech/sources/monte-carlo.md>)

Topics: [context window](<https://devfeed.tech/topics/context-window.md>), [AI Context Management](<https://devfeed.tech/topics/ai-context-management.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>), [LLM observability](<https://devfeed.tech/topics/llm-observability.md>), [LLM security](<https://devfeed.tech/topics/llm-security.md>), [LLM evaluation / benchmarking](<https://devfeed.tech/topics/llm-evaluation-benchmarking.md>), [llamaindex](<https://devfeed.tech/topics/llamaindex.md>)

Tags: [agent](<https://devfeed.tech/tags/agent.md>), [agent-memory](<https://devfeed.tech/tags/agent-memory.md>), [agent-trust](<https://devfeed.tech/tags/agent-trust.md>), [agentic](<https://devfeed.tech/tags/agentic.md>), [agents](<https://devfeed.tech/tags/agents.md>), [context-rot](<https://devfeed.tech/tags/context-rot.md>), [failure](<https://devfeed.tech/tags/failure.md>), [large-language-models](<https://devfeed.tech/tags/large-language-models.md>), [llm](<https://devfeed.tech/tags/llm.md>), [performance](<https://devfeed.tech/tags/performance.md>), [source](<https://devfeed.tech/tags/source.md>)

### AI overview

The article argues that agents can fail before inference when retrieved data is stale, corrupted, schema-drifted, or conflicting ("source rot"); prompt compaction and reranking cannot fix bad upstream inputs. It distinguishes this from context-window decay and describes how source errors can recur across retrievals and persist in long-term memory. It cites examples of conflicting airline policy sources, faulty inputs degrading Unity ad targeting, and an AI Overview repeating an old Reddit joke. Suggested safeguards are to attach freshness timestamps and validation status to retrieved data and enforce freshness SLAs; block downstream execution on schema drift; use TTLs and mark or prune superseded memory; and monitor source-data health and freshness alongside model traces. The article reports that the STALE benchmark's top model scored 55% overall, and presents a 2026 practitioner survey in which 64% named context and data quality as their biggest visibility blind spot.

### Source excerpt

Every developer working with large language models has learned to fear context degradation. This manifests when an agent, which started off sharp, gradually loses the thread as a conversation lengthens, forgetting instructions, fixating on irrelevant details, and eventually hallucinating altogether. The industry has given this failure mode a name many practitioners are now aware of: ...

## What it takes to ship a self-improving agent

DevFeed: [What it takes to ship a self-improving agent](<https://devfeed.tech/articles/what-it-takes-to-ship-a-self-improving-agent-133418.md>)

Original publisher: [Read original article](<https://montecarlo.ai/blog-ship-self-improving-agents>)

Author: Ruslan Sultanov

Published: 2026-11-13T19:54:30Z

Content type: article

Language: en

Sources: [Monte Carlo](<https://devfeed.tech/sources/monte-carlo.md>)

Topics: [AI Chat](<https://devfeed.tech/topics/ai-chat.md>), [ai observability](<https://devfeed.tech/topics/ai-observability.md>), [Monitoring](<https://devfeed.tech/topics/monitoring.md>), [Production Engineering](<https://devfeed.tech/topics/production-engineering.md>)

Tags: [agent](<https://devfeed.tech/tags/agent.md>), [agent-trust](<https://devfeed.tech/tags/agent-trust.md>), [bayes](<https://devfeed.tech/tags/bayes.md>), [eval](<https://devfeed.tech/tags/eval.md>), [guardrails](<https://devfeed.tech/tags/guardrails.md>), [monte-carlo](<https://devfeed.tech/tags/monte-carlo.md>), [production](<https://devfeed.tech/tags/production.md>), [self-improving-agent](<https://devfeed.tech/tags/self-improving-agent.md>), [self-improving-agents](<https://devfeed.tech/tags/self-improving-agents.md>)

### AI overview

Production checks and guardrails catch anticipated failures, but agents can fail silently as users, data, models, tools, rate limits, and team changes shift. The article argues for a recurring production learning loop: join traces, eval results, tickets, and incident context; detect issues in real traffic; triage findings; trace root causes across agents and their data; propose a fix in the team's workflow; and verify it against later runs. Monte Carlo's Bayes illustrates this loop, with human involvement ranging from manual work to approval of proposed fixes or autonomous handling with exception escalation. To begin, review a month of incidents for one established agent, noting how each was detected and whether the fix was verified.

### Source excerpt

Most teams learn their agent is broken the same way: a customer tells them. In Monte Carlo's 2026 survey of 260 agent builders and leaders, 52% of builders said they find out about production issues through customer or user complaints. Those teams didn't skip the work. Their agents passed an eval suite, sat behind guardrails, ...

## Fixing the AI Platform Team's Blind Spot: A Common-Thread Approach to Context Across Teams for Trusted AI Deployment

DevFeed: [Fixing the AI Platform Team's Blind Spot: A Common-Thread Approach to Context Across Teams for Trusted AI Deployment](<https://devfeed.tech/articles/fixing-the-ai-platform-team-s-blind-spot-a-common-thread-approach-to-context-across-teams-for-trusted-ai-deployment-133417.md>)

Original publisher: [Read original article](<https://montecarlo.ai/blog-fixing-the-ai-platform-teams-blind-spot-a-common-thread-approach-to-context-across-teams-for-trusted-ai-deployment>)

Author: Barr Moses

Published: 2026-11-13T15:20:06Z

Content type: opinion

Language: en

Sources: [Monte Carlo](<https://devfeed.tech/sources/monte-carlo.md>)

Topics: [AI Platform](<https://devfeed.tech/topics/ai-platform.md>), [AI Context Management](<https://devfeed.tech/topics/ai-context-management.md>), [AI-Ready Data](<https://devfeed.tech/topics/ai-ready-data.md>), [Cloud Native Operations](<https://devfeed.tech/topics/cloud-native-operations.md>)

Tags: [agent-trust](<https://devfeed.tech/tags/agent-trust.md>), [agents](<https://devfeed.tech/tags/agents.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-platform](<https://devfeed.tech/tags/ai-platform.md>), [approach](<https://devfeed.tech/tags/approach.md>), [context](<https://devfeed.tech/tags/context.md>), [deployment](<https://devfeed.tech/tags/deployment.md>), [enablement](<https://devfeed.tech/tags/enablement.md>), [enterprise](<https://devfeed.tech/tags/enterprise.md>), [evals](<https://devfeed.tech/tags/evals.md>), [evolution](<https://devfeed.tech/tags/evolution.md>), [form](<https://devfeed.tech/tags/form.md>), [gateway](<https://devfeed.tech/tags/gateway.md>), [groups](<https://devfeed.tech/tags/groups.md>), [industry](<https://devfeed.tech/tags/industry.md>), [infrastructure](<https://devfeed.tech/tags/infrastructure.md>), [organizations](<https://devfeed.tech/tags/organizations.md>), [platform](<https://devfeed.tech/tags/platform.md>), [team](<https://devfeed.tech/tags/team.md>), [teams](<https://devfeed.tech/tags/teams.md>), [thread](<https://devfeed.tech/tags/thread.md>)

### AI overview

The article argues that trusted AI deployment depends on managing context as an enterprise loop: data teams source it, domain owners define it, platform teams deliver it, product teams use it, and agent telemetry and write-backs feed later runs. Failures often occur at team interfaces, where separate agent tracing and data pipeline monitoring can each appear healthy while stale or transformed data causes incorrect outputs. The proposed "common thread" is a shared view linking agent outputs to delivery, meaning, and source, with end-to-end lineage. Suggested practices include assigning accountability for wrong production answers, setting data freshness safeguards and fallbacks, governing agent-generated memory and summaries like upstream data, and handling agent and data issues through a unified incident process. The excerpt notes that the weakest interface can depend on platform reporting structure; it does not establish that one organizational placement solves the problem.

### Source excerpt

Across the enterprise, organizations are standing up some form of an AI platform team--whether under the banner of "AI enablement" or an "AI Center of Excellence." As we've explored in previous posts on industry structures and team evolution, these groups tend to own the shared infrastructure path that agents run on: gateway, runtime, tools, evals, ...

## AWS re:Invent 2026: Everything You Need to Know Before You Go

DevFeed: [AWS re:Invent 2026: Everything You Need to Know Before You Go](<https://devfeed.tech/articles/aws-re-invent-2026-everything-you-need-to-know-before-you-go-131022.md>)

Original publisher: [Read original article](<https://montecarlo.ai/blog-aws-reinvent-2026-know-before-you-go>)

Author: Ruslan Sultanov

Published: 2026-11-12T14:20:50Z

Content type: article

Language: en

Sources: [Monte Carlo](<https://devfeed.tech/sources/monte-carlo.md>)

Topics: [Amazon Web Services](<https://devfeed.tech/topics/aws.md>), [Amazon Bedrock](<https://devfeed.tech/topics/amazon-bedrock.md>), [AI Chat](<https://devfeed.tech/topics/ai-chat.md>)

Tags: [2026](<https://devfeed.tech/tags/2026.md>), [agent](<https://devfeed.tech/tags/agent.md>), [agent-trust](<https://devfeed.tech/tags/agent-trust.md>), [agentic](<https://devfeed.tech/tags/agentic.md>), [agentic-ai](<https://devfeed.tech/tags/agentic-ai.md>), [agentic-workflows](<https://devfeed.tech/tags/agentic-workflows.md>), [agents](<https://devfeed.tech/tags/agents.md>), [agents-in-production](<https://devfeed.tech/tags/agents-in-production.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-cloud](<https://devfeed.tech/tags/ai-cloud.md>), [ai-coding-assistants](<https://devfeed.tech/tags/ai-coding-assistants.md>), [ai-cost](<https://devfeed.tech/tags/ai-cost.md>), [ai-engineer](<https://devfeed.tech/tags/ai-engineer.md>), [ai-leaders](<https://devfeed.tech/tags/ai-leaders.md>), [aws](<https://devfeed.tech/tags/aws.md>), [aws-re-invent](<https://devfeed.tech/tags/aws-re-invent.md>), [aws-re-invent-2026](<https://devfeed.tech/tags/aws-re-invent-2026.md>), [know-before-you-go](<https://devfeed.tech/tags/know-before-you-go.md>), [networking](<https://devfeed.tech/tags/networking.md>), [re-invent](<https://devfeed.tech/tags/re-invent.md>), [sessions](<https://devfeed.tech/tags/sessions.md>), [team](<https://devfeed.tech/tags/team.md>)

### AI overview

AWS re:Invent 2026 runs November 30-December 4 across several Las Vegas Strip venues, with 2,200+ sessions, hands-on labs, an Expo and social events. The article highlights production agent reliability, AI costs, security and governance, and developer productivity. Monte Carlo will demonstrate Bayes at Booth #1145 and co-present a GE Healthcare session on deploying trusted AI in high-stakes settings on December 3. It also promotes a December 2 Agent Trust Grand Prix happy hour. Practical tips include favoriting and reserving sessions in the AWS Events app, allowing time to travel between venues, capturing insights and following up with new contacts.

### Source excerpt

AWS re:Invent is right around the corner! If you're planning to attend, you're in exactly the right place. Our team has pulled together everything you need to know to make the most of the event: from key sessions and our booth, to networking opportunities and how you can connect with our team onsite. Date & ...

## What Is Agentic Observability? How It Works and Where It Stops

DevFeed: [What Is Agentic Observability? How It Works and Where It Stops](<https://devfeed.tech/articles/what-is-agentic-observability-how-it-works-and-where-it-stops-127373.md>)

Original publisher: [Read original article](<https://montecarlo.ai/blog-what-is-agentic-observability>)

Author: Ruslan Sultanov

Published: 2026-11-09T21:20:52Z

Content type: article

Language: en

Sources: [Monte Carlo](<https://devfeed.tech/sources/monte-carlo.md>)

Topics: [agent observability](<https://devfeed.tech/topics/agent-observability.md>), [observability](<https://devfeed.tech/topics/observability.md>), [LLM observability](<https://devfeed.tech/topics/llm-observability.md>), [OpenTelemetry](<https://devfeed.tech/topics/opentelemetry.md>), [Agentic AI](<https://devfeed.tech/topics/what-is-agentic-ai.md>)

Tags: [agent](<https://devfeed.tech/tags/agent.md>), [agent-framework](<https://devfeed.tech/tags/agent-framework.md>), [agent-tracing](<https://devfeed.tech/tags/agent-tracing.md>), [agent-trust](<https://devfeed.tech/tags/agent-trust.md>), [agent-trust-glossary](<https://devfeed.tech/tags/agent-trust-glossary.md>), [agentic](<https://devfeed.tech/tags/agentic.md>), [agentic-ai](<https://devfeed.tech/tags/agentic-ai.md>), [agentic-observability](<https://devfeed.tech/tags/agentic-observability.md>), [agentic-observability-vs-llm-observability](<https://devfeed.tech/tags/agentic-observability-vs-llm-observability.md>), [agents](<https://devfeed.tech/tags/agents.md>), [agents-in-production](<https://devfeed.tech/tags/agents-in-production.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-agent-observability](<https://devfeed.tech/tags/ai-agent-observability.md>), [ai-and-data](<https://devfeed.tech/tags/ai-and-data.md>), [ai-coding-assistants](<https://devfeed.tech/tags/ai-coding-assistants.md>), [ai-engineering](<https://devfeed.tech/tags/ai-engineering.md>), [ai-observability](<https://devfeed.tech/tags/ai-observability.md>), [customer](<https://devfeed.tech/tags/customer.md>), [error](<https://devfeed.tech/tags/error.md>), [glossary](<https://devfeed.tech/tags/glossary.md>), [latency](<https://devfeed.tech/tags/latency.md>), [observability](<https://devfeed.tech/tags/observability.md>), [observability-for-ai-agents](<https://devfeed.tech/tags/observability-for-ai-agents.md>), [request](<https://devfeed.tech/tags/request.md>), [tool](<https://devfeed.tech/tags/tool.md>)

### AI overview

Agentic observability tracks whether an AI agent made a sound decision, not just whether its services ran successfully. A useful setup captures context and data quality, performance, behavior such as tool calls, and output quality. The described workflow instruments runs with traces, evaluates production outputs, follows failures through tools and upstream data using lineage, then fixes and verifies the cause. The refund example shows how a correct tool call can still return a wrong amount because its source table is stale; agent tracing alone or pipeline monitoring alone may miss that connection. Observability provides evidence but does not improve agents by itself: teams also need owners, resolution measures, and decisions about human approval versus automation. OpenTelemetry-compatible traces can support portability, while data observability covers freshness and quality. The text distinguishes observing agents from using agents to investigate telemetry and act on findings.

### Source excerpt

What Is Agentic Observability? How It Works and Where It Stops A support agent tells a customer their refund is $40. The real number is $140. Every dashboard is green: the request returned 200, latency was normal, no tool threw an error. Nobody finds out until the customer writes back. That gap between "the system ...

## Is "outcome maxxing" the new "tokenmaxxing"?

DevFeed: [Is "outcome maxxing" the new "tokenmaxxing"?](<https://devfeed.tech/articles/is-outcome-maxxing-the-new-tokenmaxxing-121189.md>)

Original publisher: [Read original article](<https://montecarlo.ai/blog-outcome-maxxing-vs-tokenmaxxing>)

Author: Barr Moses

Published: 2026-11-05T22:03:29Z

Content type: article

Language: en

Sources: [Monte Carlo](<https://devfeed.tech/sources/monte-carlo.md>)

Topics: [ai observability](<https://devfeed.tech/topics/ai-observability.md>), [LLM observability](<https://devfeed.tech/topics/llm-observability.md>), [observability](<https://devfeed.tech/topics/observability.md>)

Tags: [agent](<https://devfeed.tech/tags/agent.md>), [agent-trust](<https://devfeed.tech/tags/agent-trust.md>), [agentic-ai-business-value](<https://devfeed.tech/tags/agentic-ai-business-value.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-kpis](<https://devfeed.tech/tags/ai-kpis.md>), [ai-roi](<https://devfeed.tech/tags/ai-roi.md>), [ai-token-leaderboards](<https://devfeed.tech/tags/ai-token-leaderboards.md>), [how-to-measure-ai-value](<https://devfeed.tech/tags/how-to-measure-ai-value.md>), [outcome-maxxing](<https://devfeed.tech/tags/outcome-maxxing.md>), [tokenmaxxing](<https://devfeed.tech/tags/tokenmaxxing.md>)

### AI overview

Outcome maxxing evaluates AI by business results rather than token use, but those results need trusted quality checks and a baseline. Token spending can indicate early experimentation, yet leaderboards at Meta, Uber, and Amazon showed how usage targets can drive waste without proving value. Organizations use different measures: Salesforce has agentic work units, while Medtronic's Rashmi Kumar points to SG&A growing more slowly than revenue. The article warns that outcome metrics can also be gamed, and cites a survey in which 62% of senior data and AI executives still check AI outputs by hand and 61% reported silent failures. Its practical guidance is to choose a metric already tracked by the business, verify answer quality alongside it, and treat an unhelpful AI task as a reason to reconsider the task. For production agents, it proposes monitoring context, performance, behavior, and output; token spend is one performance signal, not a measure of overall success.

### Source excerpt

Tokenmaxxing had a big year. Dictionary.com named "maxxing" its 2026 Word of the Year, and the token utilization trend wave was part of the reason. With its rise soon came its fall, however, as organizations matured in their approaches to AI. In an internal memo Amazon published last week, the company called tokenmaxxing a deeply ...

## AI's Full-Circle Moment: Are the Consultants Back?

DevFeed: [AI's Full-Circle Moment: Are the Consultants Back?](<https://devfeed.tech/articles/ai-s-full-circle-moment-are-the-consultants-back-121187.md>)

Original publisher: [Read original article](<https://montecarlo.ai/blog-ais-full-circle-moment>)

Author: Barr Moses

Published: 2026-11-05T17:21:23Z

Content type: article

Language: en

Sources: [Monte Carlo](<https://devfeed.tech/sources/monte-carlo.md>)

Topics: [AI Integration Strategies](<https://devfeed.tech/topics/ai-integration-strategies.md>), [implementation](<https://devfeed.tech/topics/implementation.md>), [The Three Main Cloud Service Models](<https://devfeed.tech/topics/the-three-main-cloud-service-models.md>), [ai-experiments](<https://devfeed.tech/topics/ai-experiments.md>), [human review](<https://devfeed.tech/topics/human-review.md>), [Bootcamp](<https://devfeed.tech/topics/bootcamp.md>)

Tags: [academy](<https://devfeed.tech/tags/academy.md>), [accenture](<https://devfeed.tech/tags/accenture.md>), [agent-trust](<https://devfeed.tech/tags/agent-trust.md>), [agentic](<https://devfeed.tech/tags/agentic.md>), [agentic-ai](<https://devfeed.tech/tags/agentic-ai.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-teammates](<https://devfeed.tech/tags/ai-teammates.md>), [ai-transformation](<https://devfeed.tech/tags/ai-transformation.md>), [anthropic](<https://devfeed.tech/tags/anthropic.md>), [forward-deployed-engineer](<https://devfeed.tech/tags/forward-deployed-engineer.md>), [software](<https://devfeed.tech/tags/software.md>), [tech-industry](<https://devfeed.tech/tags/tech-industry.md>)

### AI overview

Enterprise AI currently relies on human deployment specialists because models lack local business context, agents can take consequential actions that need ownership, and productivity gains have not yet translated into comparable organization-level returns. The excerpt describes this as a recurring cycle: services bridge the gap between software demos and enterprise-scale operation until products improve. It cites Microsoft's embedded expert investment, rising U.S. postings for forward-deployed engineers, and a McKinsey survey in which eight in ten respondents reported personal productivity gains but 37% reported an EBIT contribution. The article argues services cannot scale as the end state: agents need ongoing maintenance, deployment knowledge can leave with consultants, and Gartner predicts many enterprises will abandon vendor-built agentic AI by 2028. It favors using field deployments as product R&D and building customer capability, then proposes AI systems that learn company context, monitor and repair deployments continuously, adapt autonomy to task risk, and retain knowledge in-house.

### Source excerpt

The tech industry is having a bit of a deja vu moment right now. As AI transformation spreads rapidly, large organizations are increasingly leaning on human expertise to drive it forward. While the title of "Forward Deployed Engineer" might be new, this is a cycle we have seen play out with software before. AI imperatives, ...

## Beyond the "Agent Boss": Why Human-AI Teams Need a Middle Layer

DevFeed: [Beyond the "Agent Boss": Why Human-AI Teams Need a Middle Layer](<https://devfeed.tech/articles/beyond-the-agent-boss-why-human-ai-teams-need-a-middle-layer-121188.md>)

Original publisher: [Read original article](<https://montecarlo.ai/blog-beyond-the-agent-boss-why-human-ai-teams-need-a-middle-layer>)

Author: Barr Moses

Published: 2026-11-05T15:40:51Z

Content type: opinion

Language: en

Sources: [Monte Carlo](<https://devfeed.tech/sources/monte-carlo.md>)

Topics: [agentic workflows](<https://devfeed.tech/topics/agentic-workflows.md>), [human review](<https://devfeed.tech/topics/human-review.md>), [human in the loop agent](<https://devfeed.tech/topics/human-in-the-loop-agent.md>), [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [Multi Agent Systems](<https://devfeed.tech/topics/multi-agent-systems.md>), [LLM security](<https://devfeed.tech/topics/llm-security.md>), [engineering-leadership](<https://devfeed.tech/topics/engineering-leadership.md>)

Tags: [agent](<https://devfeed.tech/tags/agent.md>), [agent-teams](<https://devfeed.tech/tags/agent-teams.md>), [agent-trust](<https://devfeed.tech/tags/agent-trust.md>), [agent-trust-ai-observability](<https://devfeed.tech/tags/agent-trust-ai-observability.md>), [agents](<https://devfeed.tech/tags/agents.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-observability](<https://devfeed.tech/tags/ai-observability.md>), [enterprises](<https://devfeed.tech/tags/enterprises.md>), [envision](<https://devfeed.tech/tags/envision.md>), [example](<https://devfeed.tech/tags/example.md>), [industry](<https://devfeed.tech/tags/industry.md>), [layer](<https://devfeed.tech/tags/layer.md>), [layers](<https://devfeed.tech/tags/layers.md>), [mckinsey](<https://devfeed.tech/tags/mckinsey.md>), [need](<https://devfeed.tech/tags/need.md>), [people](<https://devfeed.tech/tags/people.md>), [planning](<https://devfeed.tech/tags/planning.md>), [small](<https://devfeed.tech/tags/small.md>), [structure](<https://devfeed.tech/tags/structure.md>), [teams](<https://devfeed.tech/tags/teams.md>), [this](<https://devfeed.tech/tags/this.md>), [top](<https://devfeed.tech/tags/top.md>), [up](<https://devfeed.tech/tags/up.md>)

### AI overview

The article argues that human review cannot scale with machine-speed agent fleets and proposes an independent AI "teammate" between working agents and people. This middle layer would continuously inspect traces, evaluations, logs, and operational data; diagnose failures; take only policy-authorized actions; and escalate consequential issues for human approval. In the currency-feed example, it detects a renamed field causing null conversions and default-rate invoices, holds the batch, and submits a fix with impact details for review. The author distinguishes this operational role from security-focused "guardian agents," since authorized agents can still fail because of deeper data problems. To keep oversight trustworthy, the teammate should have separate runtime and credentials, ignore instructions from agents it monitors, use evidence they cannot alter, and leave immutable logs for human audit. Autonomy should be set per task according to risk and reversibility, with people setting policy and owning outcomes. The article closes with questions about oversight capacity, independence, and traceability.

### Source excerpt

Most enterprises envision their human-agent teams will be made up of two layers. On top, there is a small group of people planning, orchestrating, and managing. Underneath, there is a growing workforce of agents executing the work. This structure is well documented across the industry. McKinsey, for example, describes teams of two to five people ...

## What Do Your Snowflake Cortex Agents Actually Cost? Now You Know

DevFeed: [What Do Your Snowflake Cortex Agents Actually Cost? Now You Know](<https://devfeed.tech/articles/what-do-your-snowflake-cortex-agents-actually-cost-now-you-know-112988.md>)

Original publisher: [Read original article](<https://montecarlo.ai/blog-what-do-your-snowflake-cortex-agents-actually-cost-now-you-know>)

Author: Virna Sekuj

Published: 2026-11-02T17:12:56Z

Content type: release

Language: en

Sources: [Monte Carlo](<https://devfeed.tech/sources/monte-carlo.md>)

Topics: [LLM observability](<https://devfeed.tech/topics/llm-observability.md>)

Tags: [agent-costs](<https://devfeed.tech/tags/agent-costs.md>), [agent-observability](<https://devfeed.tech/tags/agent-observability.md>), [agent-trust](<https://devfeed.tech/tags/agent-trust.md>), [agent-trust-ai-observability](<https://devfeed.tech/tags/agent-trust-ai-observability.md>), [agents](<https://devfeed.tech/tags/agents.md>), [ai-cost](<https://devfeed.tech/tags/ai-cost.md>), [ai-observability](<https://devfeed.tech/tags/ai-observability.md>), [alert](<https://devfeed.tech/tags/alert.md>), [better-together](<https://devfeed.tech/tags/better-together.md>), [caching](<https://devfeed.tech/tags/caching.md>), [compute](<https://devfeed.tech/tags/compute.md>), [confidence](<https://devfeed.tech/tags/confidence.md>), [conversations](<https://devfeed.tech/tags/conversations.md>), [cortex](<https://devfeed.tech/tags/cortex.md>), [cost](<https://devfeed.tech/tags/cost.md>), [costs](<https://devfeed.tech/tags/costs.md>), [customers](<https://devfeed.tech/tags/customers.md>), [demo](<https://devfeed.tech/tags/demo.md>), [deployments](<https://devfeed.tech/tags/deployments.md>), [efficiency](<https://devfeed.tech/tags/efficiency.md>), [errors](<https://devfeed.tech/tags/errors.md>), [snowflake](<https://devfeed.tech/tags/snowflake.md>), [snowflake-cortex](<https://devfeed.tech/tags/snowflake-cortex.md>)

### AI overview

Monte Carlo's Agent Observability now estimates Snowflake Cortex agent costs in credits by combining LLM token usage with warehouse credits attributed to each query. Cost appears in summaries, traces, conversations and turns, with breakdowns for token and query credits; teams can filter and rank costly traces, monitor spend, view a 30-day projection based on the trailing 7-day average, and see prompt-caching savings. Rates can be overridden with negotiated per-model prices, while models without overrides use defaults; views update without backfill. The source says ingestion for Monte Carlo-hosted deployments is enabled per account and must be turned on through the Monte Carlo team. These are described as costs calculated from agent spans and Snowflake's published credit table or configured rates.

### Source excerpt

Snowflake Cortex has made it easy to put an AI agent right next to the data that feeds it. Teams are shipping Cortex agents that answer questions, write SQL, and run workflows, often across several business units at once. Now the bill is arriving, and people want to know two things: what are these agents ...

## See How Your Genie Agent Got to the Answer

DevFeed: [See How Your Genie Agent Got to the Answer](<https://devfeed.tech/articles/see-how-your-genie-agent-got-to-the-answer-112987.md>)

Original publisher: [Read original article](<https://montecarlo.ai/blog-see-how-your-genie-agent-got-to-the-answer>)

Author: Virna Sekuj

Published: 2026-11-02T17:11:35Z

Content type: release

Language: en

Sources: [Monte Carlo](<https://devfeed.tech/sources/monte-carlo.md>)

Topics: [agent observability](<https://devfeed.tech/topics/agent-observability.md>), [ai observability](<https://devfeed.tech/topics/ai-observability.md>), [observability ai agents](<https://devfeed.tech/topics/observability-ai-agents.md>), [observability](<https://devfeed.tech/topics/observability.md>), [OpenTelemetry](<https://devfeed.tech/topics/opentelemetry.md>)

Tags: [agent](<https://devfeed.tech/tags/agent.md>), [agent-observability](<https://devfeed.tech/tags/agent-observability.md>), [agent-trust](<https://devfeed.tech/tags/agent-trust.md>), [agent-trust-ai-observability](<https://devfeed.tech/tags/agent-trust-ai-observability.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-observability](<https://devfeed.tech/tags/ai-observability.md>), [bi](<https://devfeed.tech/tags/bi.md>), [business](<https://devfeed.tech/tags/business.md>), [can](<https://devfeed.tech/tags/can.md>), [data](<https://devfeed.tech/tags/data.md>), [databricks](<https://devfeed.tech/tags/databricks.md>), [databricks-ai](<https://devfeed.tech/tags/databricks-ai.md>), [english](<https://devfeed.tech/tags/english.md>), [enterprises](<https://devfeed.tech/tags/enterprises.md>), [genie](<https://devfeed.tech/tags/genie.md>), [monte-carlo](<https://devfeed.tech/tags/monte-carlo.md>), [observability](<https://devfeed.tech/tags/observability.md>), [question](<https://devfeed.tech/tags/question.md>), [research](<https://devfeed.tech/tags/research.md>), [show](<https://devfeed.tech/tags/show.md>), [sql](<https://devfeed.tech/tags/sql.md>), [traces](<https://devfeed.tech/tags/traces.md>), [users](<https://devfeed.tech/tags/users.md>)

### AI overview

Monte Carlo's updated Genie traces expose the research trail behind each answer, making it easier to locate where an agent went off course. In agent mode, each research step appears as a node in the trace tree with its SQL queries nested beneath it; spans identify the turn mode, and turns show the agent's restated intent, plan, and narration. Standard-mode turns also surface selected tables, applied instructions, and matched example questions with confidence scores. Existing traces gain the new research-step structure automatically, while standard-mode traces remain unchanged. Monte Carlo-hosted Agent Observability can also ingest traces from Databricks MLflow SDK agents, Knowledge Assistants, and Genie: workspace traces are exported on a schedule, converted to OpenTelemetry, and stored for viewing without a live warehouse query. Hosted ingestion is enabled per account.

### Source excerpt

Genie traces now show every research step behind an answer, and Databricks agent traces can be ingested into Monte Carlo-hosted Agent Observability. Databricks AI/BI Genie has become the front door to data for many enterprises. Business users ask a question in plain English, and Genie writes the SQL, runs it, and hands back an answer. ...

## Your Agents Will Ship Before Your Homegrown Monitoring Does

DevFeed: [Your Agents Will Ship Before Your Homegrown Monitoring Does](<https://devfeed.tech/articles/your-agents-will-ship-before-your-homegrown-monitoring-does-109692.md>)

Original publisher: [Read original article](<https://montecarlo.ai/blog-homegrown-monitoring>)

Author: Ruslan Sultanov

Published: 2026-10-30T16:35:13Z

Content type: opinion

Language: en

Sources: [Monte Carlo](<https://devfeed.tech/sources/monte-carlo.md>)

Topics: [ai observability](<https://devfeed.tech/topics/ai-observability.md>), [Monitoring](<https://devfeed.tech/topics/monitoring.md>), [data observability](<https://devfeed.tech/topics/data-observability.md>), [cloud cost management](<https://devfeed.tech/topics/cloud-cost-management.md>)

Tags: [agent-trust](<https://devfeed.tech/tags/agent-trust.md>), [agent-trust-ai-observability](<https://devfeed.tech/tags/agent-trust-ai-observability.md>), [agentic-operations](<https://devfeed.tech/tags/agentic-operations.md>), [agents](<https://devfeed.tech/tags/agents.md>), [agents-in-production](<https://devfeed.tech/tags/agents-in-production.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-coding-tools](<https://devfeed.tech/tags/ai-coding-tools.md>), [ai-observability](<https://devfeed.tech/tags/ai-observability.md>), [budget](<https://devfeed.tech/tags/budget.md>), [build](<https://devfeed.tech/tags/build.md>), [build-vs-buy](<https://devfeed.tech/tags/build-vs-buy.md>), [building](<https://devfeed.tech/tags/building.md>), [challenges](<https://devfeed.tech/tags/challenges.md>), [code](<https://devfeed.tech/tags/code.md>), [coding](<https://devfeed.tech/tags/coding.md>), [coverage](<https://devfeed.tech/tags/coverage.md>), [cto](<https://devfeed.tech/tags/cto.md>), [enterprise](<https://devfeed.tech/tags/enterprise.md>), [homegrown-monitoring](<https://devfeed.tech/tags/homegrown-monitoring.md>), [monitoring](<https://devfeed.tech/tags/monitoring.md>), [production](<https://devfeed.tech/tags/production.md>), [ship](<https://devfeed.tech/tags/ship.md>)

### AI overview

The article argues that homegrown agent monitoring can suit teams with one or two agents, but often falls behind as deployments grow. Around a fifth agent, it says, token costs, tooling maintenance, incident investigation gaps, and higher-impact failures tend to converge. A trust platform involves monitoring data quality and lineage, agent context, tool use, cost and answer quality, plus agentic operations to profile data, set checks, triage alerts and fix issues. Coding tools may speed initial implementation, but tuning baselines and reducing alert noise depend on accumulated response history. In the authors' mid-sized-enterprise model, matching a bought platform takes about 24 months, with agentic operations beginning in month 19; a three-year build for a composite company with 30,000 tables is estimated at $5.14M versus $0.98M to buy, peaking at 7-10 engineers. The article advises estimating agent growth, assigning ongoing maintenance ownership, modeling evaluation costs, and deciding how to manage coverage before agentic operations are available. Estimates vary by organization.

### Source excerpt

Your first agent went to production in a few weeks. The plan for monitoring it was reasonable: build it ourselves. You know your stack, you can start today, and there's no procurement cycle in the way. Surely, homegrown monitoring makes sense, right? Then the second agent shipped, and the fifth. The monitoring is still on ...

## What is an AI Bill of Materials and Why Your Agents Need One

DevFeed: [What is an AI Bill of Materials and Why Your Agents Need One](<https://devfeed.tech/articles/what-is-an-ai-bill-of-materials-and-why-your-agents-need-one-109691.md>)

Original publisher: [Read original article](<https://montecarlo.ai/blog-ai-bill-of-materials>)

Author: Ruslan Sultanov

Published: 2026-10-30T15:25:54Z

Content type: article

Language: en

Sources: [Monte Carlo](<https://devfeed.tech/sources/monte-carlo.md>)

Topics: [spdx](<https://devfeed.tech/topics/spdx.md>), [AI Chat](<https://devfeed.tech/topics/ai-chat.md>), [OpenTelemetry](<https://devfeed.tech/topics/opentelemetry.md>), [Post Mortem](<https://devfeed.tech/topics/post-mortem.md>)

Tags: [account](<https://devfeed.tech/tags/account.md>), [agent](<https://devfeed.tech/tags/agent.md>), [agent-trust](<https://devfeed.tech/tags/agent-trust.md>), [agents](<https://devfeed.tech/tags/agents.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-agent](<https://devfeed.tech/tags/ai-agent.md>), [ai-bill-of-materials](<https://devfeed.tech/tags/ai-bill-of-materials.md>), [bill](<https://devfeed.tech/tags/bill.md>), [bill-of-materials](<https://devfeed.tech/tags/bill-of-materials.md>), [finance](<https://devfeed.tech/tags/finance.md>), [materials](<https://devfeed.tech/tags/materials.md>), [need](<https://devfeed.tech/tags/need.md>), [one](<https://devfeed.tech/tags/one.md>), [payment](<https://devfeed.tech/tags/payment.md>), [picture](<https://devfeed.tech/tags/picture.md>), [question](<https://devfeed.tech/tags/question.md>), [security](<https://devfeed.tech/tags/security.md>), [suppliers](<https://devfeed.tech/tags/suppliers.md>), [team](<https://devfeed.tech/tags/team.md>), [this](<https://devfeed.tech/tags/this.md>), [what-is](<https://devfeed.tech/tags/what-is.md>)

### AI overview

An AI bill of materials (AIBOM) is a machine-readable inventory of an agent's models, prompts, tools, libraries, and runtime data. The excerpt says the G7's 2026 minimum elements add models, datasets, and AI infrastructure to SBOMs, but may omit prompt content, table contents, and who can change them. Build-time inventories can go stale when prompts, model configurations, APIs, or data change in production. The proposed approach is to record runtime tables and their underlying views and sources, ownership, and change frequency; compare agent traces with the inventory; and monitor data freshness, volume, schema, quality, and lineage alongside agent behavior. During an incident, traces identify what the agent read and called, while upstream and downstream lineage can reveal the source of bad data and other affected consumers.

### Source excerpt

Picture an AI agent on a finance team that pays suppliers, with a track record so clean that approvals have become a formality. Then one morning, a payment meant for a longtime supplier lands in an account someone slipped into the vendor record the night before. Security asks the obvious question: what is this agent ...

[Next page](<https://devfeed.tech/sources/monte-carlo.md?cursor=WyIyMDI2LTEwLTMwVDE1OjI1OjU0KzAwOjAwIiwgIjU2MDhjNTliLTIxOGUtNDE0MS1hNTkyLWQ5ZWYzNTRmZGVlYyJd>)