# When the algorithm hallucinates in the ICU, who's watching?

DevFeed: [When the algorithm hallucinates in the ICU, who's watching?](<https://devfeed.tech/articles/when-the-algorithm-hallucinates-in-the-icu-who-s-watching-124335.md>)

Original publisher: [Read original article](<https://www.dynatrace.com/news/blog/when-the-algorithm-hallucinates-in-the-icu-whos-watching/>)

Author: Tom Andriola

Published: 2026-11-09T20:23:16Z

Content type: article

Language: en

Sources: [Dynatrace news](<https://devfeed.tech/sources/dynatrace-news.md>)

Topics: [observability](<https://devfeed.tech/topics/observability.md>), [ai-governance](<https://devfeed.tech/topics/ai-governance.md>), [LLM observability](<https://devfeed.tech/topics/llm-observability.md>), [responsible-ai](<https://devfeed.tech/topics/responsible-ai.md>), [human review](<https://devfeed.tech/topics/human-review.md>), [dynatrace](<https://devfeed.tech/topics/dynatrace.md>), [AI Infrastructure](<https://devfeed.tech/topics/ai-infrastructure.md>), [control-plane](<https://devfeed.tech/topics/control-plane.md>), [Access Control](<https://devfeed.tech/topics/access-control.md>)

Tags: [agent](<https://devfeed.tech/tags/agent.md>), [agentic](<https://devfeed.tech/tags/agentic.md>), [agentic-ai](<https://devfeed.tech/tags/agentic-ai.md>), [agents-in-production](<https://devfeed.tech/tags/agents-in-production.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-agent](<https://devfeed.tech/tags/ai-agent.md>), [ai-agents](<https://devfeed.tech/tags/ai-agents.md>), [ai-assistants](<https://devfeed.tech/tags/ai-assistants.md>), [ai-control-plane](<https://devfeed.tech/tags/ai-control-plane.md>), [ai-governance](<https://devfeed.tech/tags/ai-governance.md>), [ai-in-healthcare](<https://devfeed.tech/tags/ai-in-healthcare.md>), [ai-observability](<https://devfeed.tech/tags/ai-observability.md>), [ai-oversight](<https://devfeed.tech/tags/ai-oversight.md>), [ai-risk](<https://devfeed.tech/tags/ai-risk.md>), [ai-risk-management](<https://devfeed.tech/tags/ai-risk-management.md>), [api](<https://devfeed.tech/tags/api.md>), [apis](<https://devfeed.tech/tags/apis.md>), [audit](<https://devfeed.tech/tags/audit.md>), [audit-logs](<https://devfeed.tech/tags/audit-logs.md>), [clinical-ai](<https://devfeed.tech/tags/clinical-ai.md>), [ehr-observability](<https://devfeed.tech/tags/ehr-observability.md>), [engineering](<https://devfeed.tech/tags/engineering.md>), [enterprise-ai](<https://devfeed.tech/tags/enterprise-ai.md>), [enterprise-ai-control-plane](<https://devfeed.tech/tags/enterprise-ai-control-plane.md>), [healthcare](<https://devfeed.tech/tags/healthcare.md>), [healthcare-ai](<https://devfeed.tech/tags/healthcare-ai.md>), [healthcare-compliance](<https://devfeed.tech/tags/healthcare-compliance.md>), [healthcare-technology](<https://devfeed.tech/tags/healthcare-technology.md>), [observability](<https://devfeed.tech/tags/observability.md>), [responsible-ai](<https://devfeed.tech/tags/responsible-ai.md>)

## AI overview

The article argues that observability supplies evidence healthcare AI governance depends on, but does not itself enforce policies or provide accountability and human judgment. As agents spread across EHRs and other platforms, it recommends correlating signals across intent, model behavior, infrastructure, and outcomes, including agent actions, workflow results, system changes, overrides, and compliance events. In deployment, relevant signals may include EHR telemetry, audit logs, clinical activity, FHIR and HL7 interfaces, patient-facing systems, and ETL jobs; visibility is limited by what closed platforms expose and must respect HIPAA, privacy policies, business associate agreements, and minimum-necessary access. Infrastructure monitoring can help investigate AI workflows but does not by itself reveal model reasoning, prompt behavior, or drift. The article presents timestamped audit trails, human oversight, and outcome-linked evidence as useful for operational review and regulatory defensibility.

## Source excerpt

Why observability is no longer a "nice-to-have" and is the foundation for responsible AI in healthcare. The post When the algorithm hallucinates in the ICU, who's watching? appeared first on Dynatrace news.