# audit

A systematic, independent, and documented process for obtaining and objectively evaluating evidence against audit criteria, including assessing system controls for compliance with policies and procedures.

This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles.

## How Standard Bank Group automates AI risk assessment for generative AI

DevFeed: [How Standard Bank Group automates AI risk assessment for generative AI](<https://devfeed.tech/articles/how-standard-bank-group-automates-ai-risk-assessment-for-generative-ai-146575.md>)

Original publisher: [Read original article](<https://aws.amazon.com/blogs/machine-learning/how-standard-bank-group-automates-ai-risk-assessment-for-generative-ai/>)

Author: Tala Qraitem

Published: 2026-11-20T19:28:56Z

Content type: article

Language: en

Sources: [Artificial Intelligence](<https://devfeed.tech/sources/artificial-intelligence.md>)

Topics: [ai-governance](<https://devfeed.tech/topics/ai-governance.md>), [Generative AI](<https://devfeed.tech/topics/generative-ai.md>), [Amazon Bedrock](<https://devfeed.tech/topics/amazon-bedrock.md>), [Amazon Bedrock Guardrails](<https://devfeed.tech/topics/amazon-bedrock-guardrails.md>), [audit](<https://devfeed.tech/topics/audit.md>), [test](<https://devfeed.tech/topics/test.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [ai-governance](<https://devfeed.tech/tags/ai-governance.md>), [ai-risk](<https://devfeed.tech/tags/ai-risk.md>), [amazon-bedrock](<https://devfeed.tech/tags/amazon-bedrock.md>), [amazon-bedrock-guardrails](<https://devfeed.tech/tags/amazon-bedrock-guardrails.md>), [amazon-cloudwatch](<https://devfeed.tech/tags/amazon-cloudwatch.md>), [approval-gates](<https://devfeed.tech/tags/approval-gates.md>), [assessment](<https://devfeed.tech/tags/assessment.md>), [automated](<https://devfeed.tech/tags/automated.md>), [aws](<https://devfeed.tech/tags/aws.md>), [aws-professional-services](<https://devfeed.tech/tags/aws-professional-services.md>), [bank](<https://devfeed.tech/tags/bank.md>), [bedrock](<https://devfeed.tech/tags/bedrock.md>), [generative-ai](<https://devfeed.tech/tags/generative-ai.md>), [intermediate-200](<https://devfeed.tech/tags/intermediate-200.md>), [llm-as-a-judge](<https://devfeed.tech/tags/llm-as-a-judge.md>), [policies](<https://devfeed.tech/tags/policies.md>), [risk](<https://devfeed.tech/tags/risk.md>), [risk-assessment](<https://devfeed.tech/tags/risk-assessment.md>), [standards](<https://devfeed.tech/tags/standards.md>), [thought-leadership](<https://devfeed.tech/tags/thought-leadership.md>), [use-case](<https://devfeed.tech/tags/use-case.md>)

### AI overview

Standard Bank Group automated evidence gathering for generative AI approval using an assessment deployed in its own AWS account. A pre-assessment agent maps repository files and documentation to about 40 intake questions; a pipeline checks documents and code, tests live applications, and uses Amazon Bedrock LLM-as-a-judge to evaluate more than 300 checks against international standards and bank policies. Narrow yes-or-no judgments feed inspectable scores, with repeated evaluations flagging inconsistencies; findings link to evidence, policy clauses, and mitigations. Reports and a dashboard support reviewers, while risk committees retain approval and regulatory decisions. The assessment is optional during proof of concept and mandatory at build-stage approval. Standard Bank Group reports that six assessments reduced a manual two-week review to about two hours and that remediation improved use-case risk scores by 14%. The assessment provides evidence; it does not certify regulatory compliance.

### Source excerpt

Standard Bank Group automated the evidence behind its generative AI approval gates. Working with AWS Professional Services, the bank built an assessment in its own AWS account that scores each use case against international AI standards and its own policies using LLM-as-a-judge on Amazon Bedrock, giving risk committees consistent, traceable evidence.

## Best Agent Frameworks for Production Use

DevFeed: [Best Agent Frameworks for Production Use](<https://devfeed.tech/articles/best-agent-frameworks-for-production-use-146945.md>)

Original publisher: [Read original article](<https://www.swytchcode.com/content/best-agent-frameworks-production>)

Author: Swytchcode Team

Published: 2026-11-20T09:00:00Z

Content type: comparison

Language: en

Sources: [Swytchcode](<https://devfeed.tech/sources/swytchcode.md>)

Topics: [Microsoft Agent Framework](<https://devfeed.tech/topics/microsoft-agent-framework.md>), [Multi Agent Systems](<https://devfeed.tech/topics/multi-agent-systems.md>), [Agent Harness](<https://devfeed.tech/topics/agent-harness.md>), [ai-governance](<https://devfeed.tech/topics/ai-governance.md>), [audit](<https://devfeed.tech/topics/audit.md>), [agent observability](<https://devfeed.tech/topics/agent-observability.md>), [Production Engineering](<https://devfeed.tech/topics/production-engineering.md>), [SDK](<https://devfeed.tech/topics/sdk.md>), [llm-reasoning](<https://devfeed.tech/topics/llm-reasoning.md>)

Tags: [2026](<https://devfeed.tech/tags/2026.md>), [agent-frameworks](<https://devfeed.tech/tags/agent-frameworks.md>), [ai-agents](<https://devfeed.tech/tags/ai-agents.md>), [audit](<https://devfeed.tech/tags/audit.md>), [best](<https://devfeed.tech/tags/best.md>), [permissions](<https://devfeed.tech/tags/permissions.md>), [production](<https://devfeed.tech/tags/production.md>), [retries](<https://devfeed.tech/tags/retries.md>)

### AI overview

The excerpt's central finding is that agent frameworks manage reasoning and tool orchestration, while API execution controls such as credentials, policy checks, idempotent retries, and independent audit generally require another layer. LangGraph is suited to branching, stateful workflows with checkpointing and human review; OpenAI Agents SDK offers native handoffs and scoped guardrails, with gaps around handoffs, hosted tools, retries, and audit; Claude Agent SDK's local tool approvals fit supervised coding and internal tools. CrewAI adds runtime controls and credential hygiene, while Vercel AI SDK offers provider and runtime flexibility but leaves credentials, sandboxing, approvals, and idempotency to the caller. Microsoft Agent Framework provides broad orchestration, with governance positioned in Azure AI Foundry. Temporal adds durable state and retries after failures, but does not decide whether a call is authorized. The excerpt describes a separate execution layer as handling allowlists, live schema validation, policy, credential injection, safe retries, response checks, and audit.

### Source excerpt

We rank six agent frameworks used in production as of November 2026 and show why none of them handle API permissions, retries, or audit by design.

## Replication of Agentic Security Events: Audit and Investigation (ROGUE-AI)

DevFeed: [Replication of Agentic Security Events: Audit and Investigation (ROGUE-AI)](<https://devfeed.tech/articles/replication-of-agentic-security-events-audit-and-investigation-rogue-ai-145684.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.26553>)

Author: Mengxiao Wang, Mahbub Alam, Kalyan Nakka, Nitesh Saxena

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Cryptography and Security](<https://devfeed.tech/sources/arxiv-cryptography-and-security.md>)

Topics: [LLM security](<https://devfeed.tech/topics/llm-security.md>), [audit](<https://devfeed.tech/topics/audit.md>), [ai security](<https://devfeed.tech/topics/ai-security.md>), [Security](<https://devfeed.tech/topics/security.md>)

Tags: [agent-framework](<https://devfeed.tech/tags/agent-framework.md>), [agentic](<https://devfeed.tech/tags/agentic.md>), [agentic-security](<https://devfeed.tech/tags/agentic-security.md>), [agents](<https://devfeed.tech/tags/agents.md>), [ai](<https://devfeed.tech/tags/ai.md>), [audit](<https://devfeed.tech/tags/audit.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cr](<https://devfeed.tech/tags/cs-cr.md>), [cs-ma](<https://devfeed.tech/tags/cs-ma.md>), [database](<https://devfeed.tech/tags/database.md>), [deepseek](<https://devfeed.tech/tags/deepseek.md>), [delete](<https://devfeed.tech/tags/delete.md>), [events](<https://devfeed.tech/tags/events.md>), [failures](<https://devfeed.tech/tags/failures.md>), [files](<https://devfeed.tech/tags/files.md>), [framework](<https://devfeed.tech/tags/framework.md>), [gemini](<https://devfeed.tech/tags/gemini.md>), [investigation](<https://devfeed.tech/tags/investigation.md>), [multi-agent](<https://devfeed.tech/tags/multi-agent.md>), [replication](<https://devfeed.tech/tags/replication.md>), [rogue-ai](<https://devfeed.tech/tags/rogue-ai.md>), [security](<https://devfeed.tech/tags/security.md>), [security-events](<https://devfeed.tech/tags/security-events.md>)

### AI overview

ROGUE-AI audits reported agent failures by reconstructing incidents offline and comparing agent runs with matched controls that omit only the suspected cause. In 108 attempted replications, it reproduced reported harm in 89%; across five GPT, Claude, Gemini, and DeepSeek models, the reported mistake occurred in 18-33% of runs versus at most 1% in controls. Among 72 incidents involving real-world harm, 62 had no attacker: agents followed their tasks but missed ordinary hazards. These replications caused more harm than prompt-injection cases for all five models, exposing a gap for defenses focused on injected instructions. A warning after each tool output that named the hazard and its consequence reduced harm for all five models; an equally long generic note left most harm in place. The abstract presents the replications as a reusable audit framework vendors can rerun before releases.

### Source excerpt

arXiv:2611.26553v1 Announce Type: cross Abstract: Agents based on large language models (LLMs) can now delete files and send payments on behalf of their users. News media reports routinely describe failures of these agents (e.g., Replit's coding agent deleting a production database), but a report often only says what went wrong and rarely (in)validates the incident or finds its root cause. Incident catalogs collect these reports, and benchmarks record whether agents fail, but neither replicates an incident nor tests the cause its report might identify. In this paper, we present ROGUE-AI (Replication of Agentic Security Events: Audit and Investigation), a multi-agent framework that closes the above gap and rebuilds each reported incident offline together with a matched control that lacks only the blamed cause, and runs agents on both. Our results suggest that ROGUE-AI is able to reproduce the reported harm in 89% of its 108 attempted replications. Agents built on five current GPT, Claude, Gemini and DeepSeek models made the reported mistake in 18% to 33% of runs, against at most 1% without the blamed cause. Of the 72 incidents with real-world harm, 62 had no underlying attacker, and in these, agents did what their task asked for but missed an ordinary hazard, such as a database shared with the live site. Replications without an attacker caused more harm than replications of prompt injection for all five models. Defenses against prompt injection look for injected instructions, but these replications contain none. Further, a warning after each tool output that names the hazard and its consequence lowered harm for all five models, whereas a note of the same length that does not mention the hazard left most of the harm in place. Built from real incidents, the ROGUE-AI replications form a reusable audit framework that vendors can rerun before each new release.

## Does AI Assistance Preserve or Collapse Disagreement? A Study of Pre-Annotations in Ambiguous Video Labeling

DevFeed: [Does AI Assistance Preserve or Collapse Disagreement? A Study of Pre-Annotations in Ambiguous Video Labeling](<https://devfeed.tech/articles/does-ai-assistance-preserve-or-collapse-disagreement-a-study-of-pre-annotations-in-ambiguous-video-labeling-144862.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2510.21798>)

Author: Juan Guti\\'errez, Victor Guti\\'errez-Garc\\'ia, Jos\\'e Luis Blanco-Murillo

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [human feedback](<https://devfeed.tech/topics/human-feedback.md>), [ai alignment](<https://devfeed.tech/topics/ai-alignment.md>), [audit](<https://devfeed.tech/topics/audit.md>), [human computer interaction](<https://devfeed.tech/topics/human-computer-interaction.md>)

Tags: [accelerate](<https://devfeed.tech/tags/accelerate.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-assistance](<https://devfeed.tech/tags/ai-assistance.md>), [alignment](<https://devfeed.tech/tags/alignment.md>), [anchor](<https://devfeed.tech/tags/anchor.md>), [annotation](<https://devfeed.tech/tags/annotation.md>), [annotations](<https://devfeed.tech/tags/annotations.md>), [audit](<https://devfeed.tech/tags/audit.md>), [behavior](<https://devfeed.tech/tags/behavior.md>), [boundary](<https://devfeed.tech/tags/boundary.md>), [clip](<https://devfeed.tech/tags/clip.md>), [consensus](<https://devfeed.tech/tags/consensus.md>), [consistency](<https://devfeed.tech/tags/consistency.md>), [context](<https://devfeed.tech/tags/context.md>), [cost](<https://devfeed.tech/tags/cost.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [cs-hc](<https://devfeed.tech/tags/cs-hc.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [dataset](<https://devfeed.tech/tags/dataset.md>), [diagnostic](<https://devfeed.tech/tags/diagnostic.md>), [labeling](<https://devfeed.tech/tags/labeling.md>), [study](<https://devfeed.tech/tags/study.md>), [video](<https://devfeed.tech/tags/video.md>)

### AI overview

In a counterbalanced pilot of 18 annotators across 180 ambiguous video-labeling sessions, a fixed CLIP-based pre-annotation engine reduced mean annotation time by 23.11%; 72% of annotators were faster, with a median individual gain of 35%. Assistance increased inter-annotator consistency and CLIP-space standardization, while semantic-label entropy changed only slightly. Alignment with a six-annotator human consensus was similar across conditions (AMI ~= 0.64), but the diagnostic included those annotators and is descriptive, not independent evidence that model influence was absent. The results suggest pre-annotations mainly served as editable temporal scaffolds in this pilot, while subtler semantic anchoring remains possible. The study also presents an audit protocol separating cost, agreement, variation, standardization, and editing.

### Source excerpt

arXiv:2510.21798v3 Announce Type: replace-cross Abstract: AI-generated Pre-Annotations can accelerate video labeling, but they may also anchor annotators to model priors and suppress disagreement that is valuable for pluralistic dataset construction. We study this tradeoff in ambiguous temporal video annotation, where annotators choose event boundaries and assign context-dependent labels such as "normal" or "abnormal." We introduce a controlled audit protocol that separates annotation cost, consensus alignment, inter-annotator consistency, temporal-boundary variation, semantic-label variation, latent-space standardization, and edit behavior. In a counterbalanced pilot study with 18 annotators and 180 annotation sessions, a fixed CLIP-based Pre-Annotation engine reduced mean annotation time by 23.11%; 72% of annotators were faster with assistance, with a median per-annotator gain of 35%. Assistance increased inter-annotator consistency and CLIP-space standardization, while semantic-label entropy changed only slightly in aggregate. A six-annotator human consensus diagnostic showed comparable alignment across conditions (AMI $\approx 0.64$ in both conditions), but this diagnostic is self-including and should be interpreted as descriptive rather than as independent evidence that model influence is absent. Overall, the results suggest that Pre-Annotations acted mainly as editable temporal scaffolds in this pilot setting, while leaving open the possibility of subtler semantic anchoring effects. We contribute an audit framework and anonymized interaction traces for studying when AI-assisted annotation preserves, reshapes, or collapses human disagreement.

## Release-Bound Withdrawal Authorization for Bitcoin Custody

DevFeed: [Release-Bound Withdrawal Authorization for Bitcoin Custody](<https://devfeed.tech/articles/release-bound-withdrawal-authorization-for-bitcoin-custody-146025.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.25231>)

Author: Jian Cui (University of Illinois Urbana-Champaign), Rui Zhu (Yale University), Haixu Tang (Indiana University Bloomington)

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Cryptography and Security](<https://devfeed.tech/sources/arxiv-cryptography-and-security.md>), [arXiv: Distributed Computing](<https://devfeed.tech/sources/arxiv-distributed-computing.md>)

Topics: [Access Control](<https://devfeed.tech/topics/access-control.md>), [Bitcoin](<https://devfeed.tech/topics/bitcoin.md>), [audit](<https://devfeed.tech/topics/audit.md>), [Threat monitoring](<https://devfeed.tech/topics/threat-monitoring.md>), [Authorization](<https://devfeed.tech/topics/authorization.md>), [Authentication](<https://devfeed.tech/topics/authentication.md>)

Tags: [audit](<https://devfeed.tech/tags/audit.md>), [authentication](<https://devfeed.tech/tags/authentication.md>), [authorization](<https://devfeed.tech/tags/authorization.md>), [bitcoin](<https://devfeed.tech/tags/bitcoin.md>), [cs-cr](<https://devfeed.tech/tags/cs-cr.md>), [cs-dc](<https://devfeed.tech/tags/cs-dc.md>), [custody](<https://devfeed.tech/tags/custody.md>), [http](<https://devfeed.tech/tags/http.md>), [lease](<https://devfeed.tech/tags/lease.md>), [probing](<https://devfeed.tech/tags/probing.md>), [recovery](<https://devfeed.tech/tags/recovery.md>), [rollback](<https://devfeed.tech/tags/rollback.md>), [rpc](<https://devfeed.tech/tags/rpc.md>), [synthetic](<https://devfeed.tech/tags/synthetic.md>), [tests](<https://devfeed.tech/tags/tests.md>)

### AI overview

Release-Bound Withdrawal Authorization (RBWA) orders Bitcoin withdrawal permits against pauses in a replicated journal and restores exposure if closing evidence leaves the active chain. An optional route delays large payouts and allows independent cold recovery; after a rollback, replayed recovery can prevail or lose to an already exported mature payout. On regtest, direct payments confirmed in one block, while staged payments confirmed in three or seven blocks with delays of two or six. Authenticated receipts can revoke one signing lease in a shared wallet. In matched synthetic schedules, probes reduced permits through compromised leases from 20 to 9 of 33, with one false fence. Two held-out HTTP workloads showed equal fixed-threshold precision for boosted audit and probe-augmented detectors. Live tests found a file marker could fence a compromised lease missed by RPC-only audit; audit using the same file-read evidence also caught a reader bypassing the probe. The benefit of probing therefore depends on evidence available to the operator.

### Source excerpt

arXiv:2611.25231v1 Announce Type: new Abstract: A compromised cryptocurrency withdrawal backend can request signatures on unauthorized payments. Previously signed transactions may remain spendable after pause or become spendable again after a reorganization. Release-Bound Withdrawal Authorization (RBWA) orders permits against pause in a replicated journal and restores exposure when its closing evidence disappears from Bitcoin's active chain. An optional route delays large payouts and permits independent cold recovery. Direct payments confirm in one block on regtest, and staged payments confirm in three or seven at delays of two or six. After rollback, a replayed recovery can win or lose to an already exported mature payout. Authenticated receipts can revoke one signing lease in a shared wallet. In matched synthetic schedules, probes reduce permits through compromised leases from 20 to 9 out of 33, with one false fence. Two held-out HTTP workloads find equal fixed-threshold precision for boosted audit and probe-augmented detectors. The audit fields already expose the probe authentication outcome. Live tests verify an exact maintenance exception and add a file marker as a distinct source. The marker fences a compromised lease that RPC-only audit misses. Audit with the same file-read evidence also fences a reader who bypasses the probe. The results tie the benefit of active probing to which evidence the operator can obtain.

## On the Reliability of CyberGym Evaluation: Task Material Consistency and Reference-Target Identity

DevFeed: [On the Reliability of CyberGym Evaluation: Task Material Consistency and Reference-Target Identity](<https://devfeed.tech/articles/on-the-reliability-of-cybergym-evaluation-task-material-consistency-and-reference-target-identity-145781.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.25790>)

Author: Yule Ding, Hongzhen Wu

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Cryptography and Security](<https://devfeed.tech/sources/arxiv-cryptography-and-security.md>), [arXiv: Software Engineering](<https://devfeed.tech/sources/arxiv-software-engineering.md>)

Topics: [Ground truth / benchmark quality](<https://devfeed.tech/topics/ground-truth-benchmark-quality.md>), [audit](<https://devfeed.tech/topics/audit.md>)

Tags: [assessment](<https://devfeed.tech/tags/assessment.md>), [audit](<https://devfeed.tech/tags/audit.md>), [consistency](<https://devfeed.tech/tags/consistency.md>), [coverage](<https://devfeed.tech/tags/coverage.md>), [cs-cr](<https://devfeed.tech/tags/cs-cr.md>), [cs-se](<https://devfeed.tech/tags/cs-se.md>), [cybergym](<https://devfeed.tech/tags/cybergym.md>), [evaluation](<https://devfeed.tech/tags/evaluation.md>), [identity](<https://devfeed.tech/tags/identity.md>), [material](<https://devfeed.tech/tags/material.md>), [proof-of-concept](<https://devfeed.tech/tags/proof-of-concept.md>), [reference](<https://devfeed.tech/tags/reference.md>), [reliability](<https://devfeed.tech/tags/reliability.md>), [reproduction](<https://devfeed.tech/tags/reproduction.md>), [target](<https://devfeed.tech/tags/target.md>), [vulnerability](<https://devfeed.tech/tags/vulnerability.md>)

### AI overview

CyberGym's differential crash check alone may overstate successful vulnerability reproduction: a PoC must trigger the defect associated with the reference crash report. Auditing all 1,507 tasks, the authors label 338 (22.4%) as materially inconsistent, most often for coverage inconsistency (205) or descriptions and patches that jointly indicate another defect (110). In one complete Level-1 GLM-5.3 run, their identity procedure--execution checks, crash-signature matching, and LLM-assisted root-cause review--classifies 89 of 1,286 differential passes as non-target and 10 as insufficient evidence. Requiring a target hit reduces the pass rate from 85.3% to 78.8%; non-target PoCs also appear among tasks with no detected material inconsistency (39 of 1,052 passes, 3.7%). The authors caution that identity verdicts need independent validation and recommend assessing task consistency and submission identity separately.

### Source excerpt

arXiv:2611.25790v1 Announce Type: cross Abstract: CyberGym evaluates vulnerability reproduction by checking whether a proof-of-concept input (PoC) crashes a vulnerable program version but not a fixed version. Interpreting this differential pass as successful reproduction requires consistent task materials and evidence that the PoC triggers the intended defect. We examine these requirements through a material audit of all 1,507 CyberGym tasks and an identity assessment of the final PoCs from one complete Level-1 GLM-5.3 run. Taking the defect associated with each reference crash report as the target, the audit assigns inconsistency labels to 338 tasks (22.4%) after correcting two labels identified by stratified blind review. The largest categories are coverage inconsistency (205 tasks) and descriptions and patches that jointly point to another defect (110 tasks). We assess PoC identity using an execution check, crash-signature matching, and LLM-assisted root-cause review. Of the 1,286 differential passes, the procedure assigns 89 non-target and 10 insufficient-evidence verdicts; requiring a target hit lowers the pass rate from 85.3% to 78.8%. Non-target verdicts also occur in tasks with no material inconsistency found: 39 of 1,052 differential passes (3.7%). Although the identity verdicts require independent validation, these findings motivate separate checks of task-material consistency and submission identity when interpreting vulnerability-reproduction scores.

## LLM GerryMandering: Separating Interactions from Additive Effects in LLM Audits

DevFeed: [LLM GerryMandering: Separating Interactions from Additive Effects in LLM Audits](<https://devfeed.tech/articles/llm-gerrymandering-separating-interactions-from-additive-effects-in-llm-audits-145681.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.26538>)

Author: George Rahul, Ronit Jahilal, Bhaskara Hanuma Vedula, Ponnurangam Kumaraguru, Pushkar Shukla

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [Large Language Model](<https://devfeed.tech/topics/llm.md>), [audit](<https://devfeed.tech/topics/audit.md>), [AI Chat](<https://devfeed.tech/topics/ai-chat.md>)

Tags: [arxiv](<https://devfeed.tech/tags/arxiv.md>), [audit](<https://devfeed.tech/tags/audit.md>), [auditing](<https://devfeed.tech/tags/auditing.md>), [audits](<https://devfeed.tech/tags/audits.md>), [benchmark](<https://devfeed.tech/tags/benchmark.md>), [bias](<https://devfeed.tech/tags/bias.md>), [challenge](<https://devfeed.tech/tags/challenge.md>), [class](<https://devfeed.tech/tags/class.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [data](<https://devfeed.tech/tags/data.md>), [dataset](<https://devfeed.tech/tags/dataset.md>), [datasets](<https://devfeed.tech/tags/datasets.md>), [effects](<https://devfeed.tech/tags/effects.md>), [gender](<https://devfeed.tech/tags/gender.md>), [gerrymandering](<https://devfeed.tech/tags/gerrymandering.md>), [identity](<https://devfeed.tech/tags/identity.md>), [independent](<https://devfeed.tech/tags/independent.md>), [indian](<https://devfeed.tech/tags/indian.md>), [interaction](<https://devfeed.tech/tags/interaction.md>), [language](<https://devfeed.tech/tags/language.md>), [llm](<https://devfeed.tech/tags/llm.md>)

### AI overview

The standard subgroup criterion can label independent attribute effects as intersectional bias. Across 150 synthetic datasets with no interactions, it produced 66.1 false subgroup flags per dataset on average; a repaired version recovered 32% of planted interactions. ADDIS fits an additive model across attributes, measures each subgroup's Additive Residual Distance from its prediction, and tests it with a Freedman-Lane null. On the same benchmark it averaged 0.09 false flags per dataset and recovered 84% of planted interactions. In audits of 13 open language models using 10,000 synthetic Indian personas, the standard criterion flagged 7.1% of 738 tested subgroups on average, compared with 3.0% for ADDIS. Gender identity crossed with sexual orientation was the largest pairing in every model. Counterfactual personas also revealed a caste-class interaction in all but one model, which the observational test lacked data to detect.

### Source excerpt

arXiv:2611.26538v1 Announce Type: new Abstract: Intersectional bias arises when a model treats people who hold several marginalised identities at once differently from what their separate identities would predict. Auditing such intersectional subgroups remains a challenge: their number grows quadratically, and there is no agreed baseline for what independent attributes would predict. Without such a baseline, many existing audits flag a subgroup when it exceeds each attribute alone, a condition that attributes acting independently already satisfy. We test this standard criterion on 150 synthetic datasets built with no interaction. With the shuffle null, it flags 66.1 false subgroups per dataset on average, and at least one in every dataset. When we plant interactions of known size into these datasets, a repaired null lets the criterion recover only 32% of them. We introduce ADDIS (Additive Intersectional Screen), which fits an additive model over all attributes, scores each subgroup by its Additive Residual Distance (ARD) from that model's prediction, and tests it against a Freedman-Lane null. On our synthetic benchmark, ADDIS flags 0.09 false subgroups per dataset and recovers 84% of the same planted interactions. We then build IndiPersona, 10,000 synthetic Indian personas with six identity attributes, and audit 13 open language models. The standard criterion flags 7.1% of the 738 tested subgroups on average; ADDIS flags 3.0%. Gender identity crossed with sexual orientation is the largest pairing in every model, and counterfactual personas confirm it and additionally reveal a caste x class interaction in all but one model that the observational test lacked the data to detect. Much of what audits report as intersectional bias can thus arise from independent single-attribute effects alone, yet a small core of interactions survives.

## Vercel completes ISO/IEC 42001:2023 certification

DevFeed: [Vercel completes ISO/IEC 42001:2023 certification](<https://devfeed.tech/articles/vercel-completes-iso-iec-42001-2023-certification-146540.md>)

Original publisher: [Read original article](<https://vercel.com/blog/vercel-completes-iso-iec-42001-2023-certification>)

Author: Kacee Taylor

Published: 2026-11-20T05:00:00Z

Content type: release

Language: en

Sources: [Vercel News](<https://devfeed.tech/sources/vercel-news.md>)

Topics: [ai-governance](<https://devfeed.tech/topics/ai-governance.md>), [responsible-ai](<https://devfeed.tech/topics/responsible-ai.md>), [audit](<https://devfeed.tech/topics/audit.md>)

Tags: [accountability](<https://devfeed.tech/tags/accountability.md>), [agent](<https://devfeed.tech/tags/agent.md>), [agents](<https://devfeed.tech/tags/agents.md>), [agents-in-production](<https://devfeed.tech/tags/agents-in-production.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-gateway](<https://devfeed.tech/tags/ai-gateway.md>), [ai-governance](<https://devfeed.tech/tags/ai-governance.md>), [ai-products](<https://devfeed.tech/tags/ai-products.md>), [ai-risk](<https://devfeed.tech/tags/ai-risk.md>), [assessment](<https://devfeed.tech/tags/assessment.md>), [audit](<https://devfeed.tech/tags/audit.md>), [audits](<https://devfeed.tech/tags/audits.md>), [benefits](<https://devfeed.tech/tags/benefits.md>), [certification](<https://devfeed.tech/tags/certification.md>), [customers](<https://devfeed.tech/tags/customers.md>), [data](<https://devfeed.tech/tags/data.md>), [independent](<https://devfeed.tech/tags/independent.md>), [iso](<https://devfeed.tech/tags/iso.md>), [iso-42001](<https://devfeed.tech/tags/iso-42001.md>), [vercel](<https://devfeed.tech/tags/vercel.md>)

### AI overview

Vercel reports completing ISO/IEC 42001:2023 certification for an AI management system covering its infrastructure, AI Gateway, Vercel Agent, v0, AI-based open source software and frameworks, and the people and processes behind them. The standard requires governance across leadership accountability, risk assessment, lifecycle management, data governance, transparency, and continual improvement; unlike security standards focused on information protection, it also assesses AI's impacts on people, including potential benefits and harms. An independent accredited body conducted a two-stage audit of the system's design and operation. The certificate lasts three years, with annual surveillance audits. Vercel says it conducts impact assessments and reassesses them when systems materially change, evaluates performance, safety, and security before release with human review of material changes, and offers data handling preferences from zero data retention to training opt-out where the pricing plan does not already determine the default. The certification may support customer and partner AI governance diligence; Vercel notes that its documentation specifies responsibilities shared between V0?

### Source excerpt

Vercel has completed its ISO/IEC 42001:2023 certification, meeting the international standard for AI management systems. This certification gives our customers confidence to ship fast with agents on Vercel, demonstrating how Vercel designs, builds, operates, and governs our AI products. Our AI management system covers Vercel's underlying infrastructure and core AI products, including AI Gateway, Vercel Agent, v0, AI-based open source software and frameworks, as well as the people and processes behind them. Vercel's certificate is available here, and additional audit documentation is available through our Trust Center. What is ISO 42001? ISO/IEC 42001:2023 is the international standard for AI management systems and applies to any organization that develops, provides, or uses AI. An AI management system is the set of policies, roles, processes, and controls through which an organization consistently governs AI activities. The standard includes requirements for leadership accountability, AI risk assessment, lifecycle management, data governance, transparency, and continual improvement. The requirement that makes ISO 42001 distinct from security standards is the AI system impact assessment. Security frameworks cover how an organization protects information, but ISO 42001 covers how it builds and uses AI, including the assessment of how an AI system impacts the people it touches, and the benefits and harms that would result when that system materially changes. To obtain the certification, we underwent a two-stage audit performed by an independent, accredited certification body, which evaluated whether our AI management system is designed correctly and operates effectively. The certificate is valid for three years, with annual surveillance audits in between. "Enterprises want agents in production, but no one should have to choose between moving fast with AI and governing it well," said Talha Tariq, CTO for Security. "We built our AI governance to run as continuously as we

## ClickHouse Security, Compliance and Governance in Polyglot Environments

DevFeed: [ClickHouse Security, Compliance and Governance in Polyglot Environments](<https://devfeed.tech/articles/clickhouse-security-compliance-and-governance-in-polyglot-environments-146542.md>)

Original publisher: [Read original article](<https://severalnines.com/blog/clickhouse-security-compliance-and-governance-in-polyglot-environments/>)

Author: Paul Namuag

Published: 2026-11-19T22:47:51Z

Content type: article

Language: en

Sources: [SeveralNines](<https://devfeed.tech/sources/severalnines.md>)

Topics: [Security](<https://devfeed.tech/topics/security.md>), [audit](<https://devfeed.tech/topics/audit.md>), [observability](<https://devfeed.tech/topics/observability.md>)

Tags: [access-control](<https://devfeed.tech/tags/access-control.md>), [access-controls](<https://devfeed.tech/tags/access-controls.md>), [analytics](<https://devfeed.tech/tags/analytics.md>), [architecture](<https://devfeed.tech/tags/architecture.md>), [audit](<https://devfeed.tech/tags/audit.md>), [audit-log](<https://devfeed.tech/tags/audit-log.md>), [audit-logging](<https://devfeed.tech/tags/audit-logging.md>), [audit-logs](<https://devfeed.tech/tags/audit-logs.md>), [authentication](<https://devfeed.tech/tags/authentication.md>), [authorization](<https://devfeed.tech/tags/authorization.md>), [clickhouse](<https://devfeed.tech/tags/clickhouse.md>), [compliance](<https://devfeed.tech/tags/compliance.md>), [data-governance](<https://devfeed.tech/tags/data-governance.md>), [database-security](<https://devfeed.tech/tags/database-security.md>), [environments](<https://devfeed.tech/tags/environments.md>), [governance](<https://devfeed.tech/tags/governance.md>), [hybrid](<https://devfeed.tech/tags/hybrid.md>), [hybrid-cloud](<https://devfeed.tech/tags/hybrid-cloud.md>), [hybrid-operations](<https://devfeed.tech/tags/hybrid-operations.md>), [mysql](<https://devfeed.tech/tags/mysql.md>), [oltp](<https://devfeed.tech/tags/oltp.md>), [rbac](<https://devfeed.tech/tags/rbac.md>), [security](<https://devfeed.tech/tags/security.md>), [security-compliance](<https://devfeed.tech/tags/security-compliance.md>)

### AI overview

For production ClickHouse, lock down the default account, use SQL-driven RBAC with least-privilege roles, configure TLS for client and interserver traffic, and retain and centralize audit logs. The article contrasts RBAC, available since version 20.1, with legacy XML user configuration, which lacks true roles and is harder to audit and maintain. It recommends a canonical identity source, automated access updates across database engines, periodic drift checks, and lineage tracking for ingested data. Hybrid deployments also need private network access, a consistent key-management strategy, and explicit log forwarding. A support-engineer checklist covers deployment and ongoing reviews, credential rotation, monitoring, and patching. ClusterControl is described as providing multi-engine visibility and ClickHouse cluster operations; identity policy reconciliation and audit collection still require complementary controls and procedures.

### Source excerpt

In our previous blog, we tackled monitoring with observability for ClickHouse. We also addressed hybrid OLTP/analytics database architecture, where data originates from MySQL and flows into ClickHouse. However, running ClickHouse in a hybrid environment also means you need to think about security and compliance. This blog covers the key practices you should have in place, [...] The post ClickHouse Security, Compliance and Governance in Polyglot Environments appeared first on Severalnines.

## Inside Agentwhy, Part 1: A Step-by-Step Audit Trail for AI Coding Agents

DevFeed: [Inside Agentwhy, Part 1: A Step-by-Step Audit Trail for AI Coding Agents](<https://devfeed.tech/articles/inside-agentwhy-part-1-a-step-by-step-audit-trail-for-ai-coding-agents-142899.md>)

Original publisher: [Read original article](<https://hackernoon.com/inside-agentwhy-part-1-a-step-by-step-audit-trail-for-ai-coding-agents?source=rss>)

Author: Karol

Published: 2026-11-19T08:32:55Z

Content type: article

Language: en

Sources: [HackerNoon](<https://devfeed.tech/sources/hackernoon.md>)

Topics: [audit](<https://devfeed.tech/topics/audit.md>), [agentic-coding](<https://devfeed.tech/topics/agentic-coding.md>), [Claude Code](<https://devfeed.tech/topics/claude-code.md>), [json lines format](<https://devfeed.tech/topics/json-lines-format.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [ai-agent-audit](<https://devfeed.tech/tags/ai-agent-audit.md>), [ai-agents](<https://devfeed.tech/tags/ai-agents.md>), [ai-coding-agents](<https://devfeed.tech/tags/ai-coding-agents.md>), [audit-trail](<https://devfeed.tech/tags/audit-trail.md>), [claude-code](<https://devfeed.tech/tags/claude-code.md>), [claude-code-jsonl](<https://devfeed.tech/tags/claude-code-jsonl.md>), [coding](<https://devfeed.tech/tags/coding.md>), [cybersecurity](<https://devfeed.tech/tags/cybersecurity.md>), [delegation](<https://devfeed.tech/tags/delegation.md>), [demo](<https://devfeed.tech/tags/demo.md>), [open-source](<https://devfeed.tech/tags/open-source.md>), [sha-256-fingerprints](<https://devfeed.tech/tags/sha-256-fingerprints.md>)

### AI overview

agentwhy turns saved coding-agent conversations into an ordered audit trail showing reads, commands, changes, blocked attempts, and helper handoffs with each helper's task and model. For Claude Code, it matches helper files to Agent calls by call ID; ambiguous or missing links are marked unresolved rather than guessed, since timestamps can be unreliable. It also checks command output against salted hashes of distinctive lines from private project files to detect reads that omit filenames. This can add evidence of a read but does not remove one, and has limits: it excludes files outside the project, skips large files or projects beyond stated thresholds, and misses partial, transformed, or changed content. The excerpt says Codex and Copilot reports are less complete, and MCP tools' file access is not visible. agentwhy runs locally; the post says secret values are replaced before the page is built.

### Source excerpt

One part of agentwhy each week. This one shows every step your AI and its helpers took, and where a rule stopped them.

## S3 audit log streaming with OpenID Connect expands to GHE.com

DevFeed: [S3 audit log streaming with OpenID Connect expands to GHE.com](<https://devfeed.tech/articles/s3-audit-log-streaming-with-openid-connect-expands-to-ghe-com-142929.md>)

Original publisher: [Read original article](<https://github.blog/changelog/2026-11-19-s3-audit-log-streaming-with-openid-connect-expands-to-ghe-com>)

Author: Allison

Published: 2026-11-19T08:27:16Z

Content type: news

Language: en

Sources: [GitHub Changelog](<https://devfeed.tech/sources/github-changelog.md>)

Topics: [OpenID connect (OIDC)](<https://devfeed.tech/topics/oidc.md>), [Amazon S3](<https://devfeed.tech/topics/amazon-s3.md>), [audit](<https://devfeed.tech/topics/audit.md>)

Tags: [amazon](<https://devfeed.tech/tags/amazon.md>), [amazon-s3](<https://devfeed.tech/tags/amazon-s3.md>), [audit](<https://devfeed.tech/tags/audit.md>), [audit-log](<https://devfeed.tech/tags/audit-log.md>), [audit-logs](<https://devfeed.tech/tags/audit-logs.md>), [aws](<https://devfeed.tech/tags/aws.md>), [cloud](<https://devfeed.tech/tags/cloud.md>), [credentials](<https://devfeed.tech/tags/credentials.md>), [data](<https://devfeed.tech/tags/data.md>), [data-residency](<https://devfeed.tech/tags/data-residency.md>), [enterprise](<https://devfeed.tech/tags/enterprise.md>), [enterprise-cloud](<https://devfeed.tech/tags/enterprise-cloud.md>), [enterprise-management-tools](<https://devfeed.tech/tags/enterprise-management-tools.md>), [github](<https://devfeed.tech/tags/github.md>), [github-enterprise-cloud](<https://devfeed.tech/tags/github-enterprise-cloud.md>), [improvement](<https://devfeed.tech/tags/improvement.md>), [oidc](<https://devfeed.tech/tags/oidc.md>), [openid](<https://devfeed.tech/tags/openid.md>), [openid-connect](<https://devfeed.tech/tags/openid-connect.md>), [platform-governance](<https://devfeed.tech/tags/platform-governance.md>), [s3](<https://devfeed.tech/tags/s3.md>), [streaming](<https://devfeed.tech/tags/streaming.md>)

### AI overview

GitHub Enterprise Cloud with data residency (GHE.com) now supports streaming enterprise audit logs to Amazon S3 using OpenID Connect (OIDC), extending the existing github.com capability. This enables authentication without storing long-lived AWS credentials on GitHub. The excerpt does not describe setup steps or other requirements.

### Source excerpt

You can now stream enterprise audit logs to Amazon S3 using OpenID Connect (OIDC) on GitHub Enterprise Cloud with data residency (GHE.com). This extends the existing github.com capability, letting you... The post S3 audit log streaming with OpenID Connect expands to GHE.com appeared first on The GitHub Blog.

## A level is not a rank: an empirical audit of quantile-to-index conversion in 10 released conformal-prediction libraries

DevFeed: [A level is not a rank: an empirical audit of quantile-to-index conversion in 10 released conformal-prediction libraries](<https://devfeed.tech/articles/a-level-is-not-a-rank-an-empirical-audit-of-quantile-to-index-conversion-in-10-released-conformal-prediction-libraries-142176.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.23329>)

Author: Shivam Lalakiya

Published: 2026-11-19T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Software Engineering](<https://devfeed.tech/sources/arxiv-software-engineering.md>)

Topics: [audit](<https://devfeed.tech/topics/audit.md>)

Tags: [audit](<https://devfeed.tech/tags/audit.md>), [conformal-prediction](<https://devfeed.tech/tags/conformal-prediction.md>), [conversion](<https://devfeed.tech/tags/conversion.md>), [coverage](<https://devfeed.tech/tags/coverage.md>), [cs-se](<https://devfeed.tech/tags/cs-se.md>), [index](<https://devfeed.tech/tags/index.md>), [libraries](<https://devfeed.tech/tags/libraries.md>), [quantile](<https://devfeed.tech/tags/quantile.md>), [rank](<https://devfeed.tech/tags/rank.md>)

### AI overview

The audit finds that quantile-to-index conversion can miss the rank needed for conformal coverage, but results vary by implementation. Across 39 sites in 10 pinned packages--a named set, not a systematic search--all seven tabular implementations reach the required rank. In darts, coverage is unchanged where ranks agree and 0.0280-0.0560 lower on one code path where they differ; a second archive shows the same split without that range. At n = 20, statsforecast's interval endpoint falls at position 18.05 rather than required rank 19. Below the feasibility floor, 5 of 14 public call paths silently return a finite value, while the others raise or warn. The audit proposes a conformance suite classifying helpers by reached rank and a review checklist. It reports that 6 of 19 upstream filings have merged, including one that fixed a coverage shortfall.

### Source excerpt

arXiv:2611.23329v1 Announce Type: new Abstract: Context. A conformal interval at 0.90 keeps that guarantee only at the right rank: a fresh score falls at or below the rth smallest of n with probability r/(n + 1). The libraries we study instead hand a level to a quantile function, which can land elsewhere. At n = 20, the upper end of statsforecast's interval sits at position 18.05 where rank 19 is required. Objective. Where does installed code resolve the wrong rank, at what cost in coverage, and what does it return when no rank reaches the level? Method. We audit 39 sites where a level becomes an index in 10 pinned packages, a named set rather than a systematic search. 12 are classified by running them, 11 through their shared numpy method, the rest from source. Coverage is paired against the required rank on the same scores. Results. All 7 tabular implementations reach the required rank. In darts, on one code path, coverage is unchanged (0.0000) where the ranks agree and 0.0280 to 0.0560 lower where they differ; a second archive repeats that split but not its range. Below the feasibility floor, $n < 1/\alpha - 1$, of 14 public call paths 5 silently return a finite number; the rest raise or warn. Conclusions. Quantile interfaces take only a level; statsforecast's default calibrates below its floor. 6 of 19 upstream filings have merged, one fixing a coverage shortfall. A conformance suite classifies helpers by the rank they reach, and a checklist guides review.

## What Moves the Ceiling on Automated Audit: from General Code to Scientific Code, Results and Data

DevFeed: [What Moves the Ceiling on Automated Audit: from General Code to Scientific Code, Results and Data](<https://devfeed.tech/articles/what-moves-the-ceiling-on-automated-audit-from-general-code-to-scientific-code-results-and-data-142127.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.23250>)

Author: Zhaohe Dong

Published: 2026-11-19T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>), [arXiv: Software Engineering](<https://devfeed.tech/sources/arxiv-software-engineering.md>)

Topics: [audit](<https://devfeed.tech/topics/audit.md>), [Test automation](<https://devfeed.tech/topics/test-automation.md>), [reproducibility](<https://devfeed.tech/topics/reproducibility.md>)

Tags: [abstract](<https://devfeed.tech/tags/abstract.md>), [all](<https://devfeed.tech/tags/all.md>), [arxiv](<https://devfeed.tech/tags/arxiv.md>), [audit](<https://devfeed.tech/tags/audit.md>), [automated](<https://devfeed.tech/tags/automated.md>), [behaviour](<https://devfeed.tech/tags/behaviour.md>), [but](<https://devfeed.tech/tags/but.md>), [can](<https://devfeed.tech/tags/can.md>), [code](<https://devfeed.tech/tags/code.md>), [cross](<https://devfeed.tech/tags/cross.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [cs-se](<https://devfeed.tech/tags/cs-se.md>), [data](<https://devfeed.tech/tags/data.md>), [decision](<https://devfeed.tech/tags/decision.md>), [documentation](<https://devfeed.tech/tags/documentation.md>), [error](<https://devfeed.tech/tags/error.md>), [figure](<https://devfeed.tech/tags/figure.md>), [flag](<https://devfeed.tech/tags/flag.md>), [general](<https://devfeed.tech/tags/general.md>), [hidden](<https://devfeed.tech/tags/hidden.md>), [independent](<https://devfeed.tech/tags/independent.md>), [results](<https://devfeed.tech/tags/results.md>)

### AI overview

The study finds that automated audits have different ceilings across general code, scientific code, reports and data. For general-purpose code, eight auditor readings achieved 30.0% recall while flagging 16.0% of correct code, with no plateau; results also shifted with the reading model and sampling, a rulebook sentence, and the decision rule. Many misses were later judged to concern behavior the specification left undefined. In three preregistered science studies, source-tracing checks caught all 49 inconsistent reports and none of 25 consistent fabrications; the model auditor flagged 23 fabrications, 21 with a sound finding. On data, a documentation-derived validator and model auditor caught different faulty items, together flagging 112 of 140 while flagging 2.9% of clean data. Reframing scientific-code auditing around one step reduced flags on correct code from 68.7% to 36.0%. Deterministic checks avoided clean-item flags but could not detect consistent errors; the model auditor caught some such errors but also made mistakes.

### Source excerpt

arXiv:2611.23250v1 Announce Type: new Abstract: Models increasingly check what other models write. We study one shipped arrangement, in which an auditor from a different vendor reads the work and deterministic checks verify what code can verify, and measure its limits against ground truth no model supplies. On general-purpose code with hidden tests, eight readings of the auditor reach 30.0% recall [20.0, 40.7] at 16.0% [10.1, 22.3] flags on correct code, with diminishing returns and no plateau. The reading model (confounded with its sampling), one rulebook sentence and the decision rule each move that figure, and much of what is missed is, on a post hoc rating, behaviour the specification never determined. In three preregistered science studies, checking that reported numbers trace to their sources flagged all 49 inconsistent reports and none of 25 consistent fabrications, of which the model auditor flagged 23 (21 with a sound finding, post hoc); on data, a documentation-derived validator and the auditor flagged overlapping but different faulty items, together 112 of 140 at a 2.9% [0.7, 5.7] flag rate on clean data; and on scientific code, reframing the task around one step cut flags on correct code from 68.7% to 36.0%. Where both tiers ran, the deterministic one flagged no clean item and could not see consistent error; the model saw some and erred itself. Studies making model calls were preregistered, the two unregistered analyses are named, and every quoted report passed independent cross-vendor review.

## Hearsay: Who Can Audit the Record a Deployed Agent Harness Writes?

DevFeed: [Hearsay: Who Can Audit the Record a Deployed Agent Harness Writes?](<https://devfeed.tech/articles/hearsay-who-can-audit-the-record-a-deployed-agent-harness-writes-142117.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2609.32495>)

Author: Jiahong Dai, Zhuochen Yang, Pengyang Shao, Kelvin Ng, Zhongyi Liu, Chengquan Ju, Yuting He, Bo Hu

Published: 2026-11-19T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Cryptography and Security](<https://devfeed.tech/sources/arxiv-cryptography-and-security.md>), [arXiv: Software Engineering](<https://devfeed.tech/sources/arxiv-software-engineering.md>)

Topics: [audit](<https://devfeed.tech/topics/audit.md>), [Agent Harness](<https://devfeed.tech/topics/agent-harness.md>), [Audit-then-score](<https://devfeed.tech/topics/audit-then-score.md>)

Tags: [agent](<https://devfeed.tech/tags/agent.md>), [agent-harness](<https://devfeed.tech/tags/agent-harness.md>), [audit](<https://devfeed.tech/tags/audit.md>), [benchmark](<https://devfeed.tech/tags/benchmark.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-ce](<https://devfeed.tech/tags/cs-ce.md>), [cs-cr](<https://devfeed.tech/tags/cs-cr.md>), [cs-se](<https://devfeed.tech/tags/cs-se.md>), [frameworks](<https://devfeed.tech/tags/frameworks.md>), [harness](<https://devfeed.tech/tags/harness.md>), [ledger](<https://devfeed.tech/tags/ledger.md>), [model](<https://devfeed.tech/tags/model.md>), [records](<https://devfeed.tech/tags/records.md>), [sandbox](<https://devfeed.tech/tags/sandbox.md>), [tasks](<https://devfeed.tech/tags/tasks.md>), [writer](<https://devfeed.tech/tags/writer.md>)

### AI overview

The paper argues that an agent run record cannot independently verify itself when the harness is its sole writer. In a benchmark spanning five harnesses, fourteen tasks, and examiners from three vendors plus a human panel, readers often identified faults but could not check the record; model examiners overstated corroboration, and even the strongest detected only half of inserted alterations. The authors report that all sixteen surveyed frameworks had the harness as the only writer. Their proposed sandbox adds a ledger of exchanges between harness and model and compares it with the record in both directions to detect omissions and additions. On unseen records, it caught 64 of 66 alterations without flagging 84 clean records, and caught 109 of 115 held-out records across five formats.

### Source excerpt

arXiv:2609.32495v2 Announce Type: replace-cross Abstract: The record of an agent's run is written by the harness that drives the agent. When the run is later disputed, the auditor was not there and has only this record. We ask whether someone who was not there can check the record, and what decides the answer. It depends on who wrote what a check can rely on: if the harness wrote all of it, the record is only the harness's own word, and we call it hearsay. We build Hearsay, a benchmark that examines the record and not the task: five harnesses run fourteen tasks, model examiners from three vendors and a human panel read the records and answer questions about the runs, and every passage they cite is labelled mechanically by who wrote it. We also audit what is deployed: in all sixteen frameworks, the harness is the only writer. With one writer, readers can name the fault from the record, in up to nine runs in ten, but they cannot check the record. The model examiners claim corroboration for 18 to 55 points more of their evidence than the labels support, a gap we then predicted for runs driven by another vendor's model and found again. And the best of them notices only half of the alterations we make to a record, while the two that name the fault most often cannot tell what a harness left out. With a second writer, the record can be checked. Our sandbox keeps a ledger of what passes between harness and model, and reading the ledger against the record in both directions shows what the record leaves out and what it adds. We test this as an auditor would use it, on one unseen record at a time. On runs driven by a third vendor's model, with every rule committed before the records existed, it reports 64 of 66 alterations and flags none of 84 clean records. On held-out records in five formats it reports 109 of 115. What makes a record evidence is a second writer, not a better reader.

## Evaluating Governed LLM-Driven ML Experimentation: Lifecycle, Provenance, and Failure Metrics

DevFeed: [Evaluating Governed LLM-Driven ML Experimentation: Lifecycle, Provenance, and Failure Metrics](<https://devfeed.tech/articles/evaluating-governed-llm-driven-ml-experimentation-lifecycle-provenance-and-failure-metrics-142192.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.23678>)

Author: Dowling Wong

Published: 2026-11-19T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>), [arXiv: Software Engineering](<https://devfeed.tech/sources/arxiv-software-engineering.md>)

Topics: [ai observability](<https://devfeed.tech/topics/ai-observability.md>), [agentic workflows](<https://devfeed.tech/topics/agentic-workflows.md>), [audit](<https://devfeed.tech/topics/audit.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>), [Open Data](<https://devfeed.tech/topics/open-data.md>)

Tags: [agent](<https://devfeed.tech/tags/agent.md>), [agents](<https://devfeed.tech/tags/agents.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [cs-se](<https://devfeed.tech/tags/cs-se.md>), [experimentation](<https://devfeed.tech/tags/experimentation.md>), [failure](<https://devfeed.tech/tags/failure.md>), [governance](<https://devfeed.tech/tags/governance.md>), [harness](<https://devfeed.tech/tags/harness.md>), [ledger](<https://devfeed.tech/tags/ledger.md>), [lifecycle](<https://devfeed.tech/tags/lifecycle.md>), [llm](<https://devfeed.tech/tags/llm.md>), [memory](<https://devfeed.tech/tags/memory.md>), [metrics](<https://devfeed.tech/tags/metrics.md>), [ml](<https://devfeed.tech/tags/ml.md>), [process](<https://devfeed.tech/tags/process.md>), [provenance](<https://devfeed.tech/tags/provenance.md>)

### AI overview

The study evaluates LLM-driven ML experimentation by process as well as best score. Its governed harness assigns trial lifecycle, edit-scope checks, keep-or-discard decisions, and append-only logging to a control plane rather than the proposing LLM; governance metrics are computed from that ledger. Across six tasks and 1,445 trials, the same contract exposed failures hidden by task scores, including 120 failed trials in a no-memory arm and a reset failure detected by reproducibility hashes. In a matched comparison, success-only logging omitted 43 of 50 trials, while the governed ledger recorded all 50. Memory's effect on repeated bad proposals varied by task. The authors say they release all trial ledgers so reported numbers can be recomputed.

### Source excerpt

arXiv:2611.23678v1 Announce Type: cross Abstract: Autonomous agents that run ML experiments are usually judged by the best score they reach. That score says nothing about how it was reached: whether the agent stayed within the files it was allowed to edit, whether failed trials were kept or silently dropped, and whether each keep or discard decision can be traced to its evidence. We propose evaluating such systems on their experimental process as well as their outcome, and we build a governed harness that makes this possible. A control plane, not the LLM that proposes changes, owns each trial's lifecycle, checks its edit scope, decides keep or discard, and appends one record per trial to an append-only ledger. Governance metrics such as the invalid-action rate, the repeated-bad rate and provenance completeness are computed from that ledger. Across six ML tasks, from synthetic logistic regression to language-model training, and 1,445 trials, the same contract applied unchanged. The ledger exposed failures that a task score hides: a no-memory arm in which all 120 trials failed, a benchmark adapter whose bounded proposals ran out, and a reset failure that reproducibility hashes revealed, after which the affected campaigns were rerun. In a matched counterfactual, a loop that logs only successful trials left no record of 43 of 50 trials, while the governed ledger recorded all 50. Memory ablations show that memory's effect on repeated bad proposals depends on the task. We release the ledgers of all 1,445 trials, so that every number reported for them can be recomputed.

## Grading Executable Legal Rules Against Independent Records: Transitional Provisions and Korean SME Status

DevFeed: [Grading Executable Legal Rules Against Independent Records: Transitional Provisions and Korean SME Status](<https://devfeed.tech/articles/grading-executable-legal-rules-against-independent-records-transitional-provisions-and-korean-sme-status-142748.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.24055>)

Author: Jinhyung Bae

Published: 2026-11-19T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [rule-engine](<https://devfeed.tech/topics/rule-engine.md>), [llm-reasoning](<https://devfeed.tech/topics/llm-reasoning.md>), [audit](<https://devfeed.tech/topics/audit.md>)

Tags: [abstract](<https://devfeed.tech/tags/abstract.md>), [all](<https://devfeed.tech/tags/all.md>), [also](<https://devfeed.tech/tags/also.md>), [arxiv](<https://devfeed.tech/tags/arxiv.md>), [better](<https://devfeed.tech/tags/better.md>), [can](<https://devfeed.tech/tags/can.md>), [cases](<https://devfeed.tech/tags/cases.md>), [changes](<https://devfeed.tech/tags/changes.md>), [comparison](<https://devfeed.tech/tags/comparison.md>), [condition](<https://devfeed.tech/tags/condition.md>), [construction](<https://devfeed.tech/tags/construction.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cl](<https://devfeed.tech/tags/cs-cl.md>), [disclosures](<https://devfeed.tech/tags/disclosures.md>), [enterprise](<https://devfeed.tech/tags/enterprise.md>), [independent](<https://devfeed.tech/tags/independent.md>), [korean](<https://devfeed.tech/tags/korean.md>), [legal](<https://devfeed.tech/tags/legal.md>), [records](<https://devfeed.tech/tags/records.md>), [rules](<https://devfeed.tech/tags/rules.md>), [sme](<https://devfeed.tech/tags/sme.md>), [status](<https://devfeed.tech/tags/status.md>)

### AI overview

LexTemporis evaluates executable legal rules against 10,901 firm-years of SME status disclosed by Korean listed companies, rather than answers based on a grader's interpretation of the law. One transitional provision raises agreement with disclosures from 83-86% to 89-90%, showing that provisions outside the main statutory text can affect date-specific outcomes. The disclosures support the author's interpretation of a contested provision in 130 of 189 firm-years where readings diverge. Four LLMs generated rules from different selections of the law; a pre-registered comparison found no condition best across all four. In exploratory analyses, the apparent benefit of supplying relevant provisions was smaller against actual records than against the author's reading for three models, underscoring how grading against a shared interpretation can overstate performance.

### Source excerpt

arXiv:2611.24055v1 Announce Type: cross Abstract: Laws are increasingly turned into programs that decide cases, such as whether a firm was a small or medium enterprise (SME) on a given date, and large language models (LLMs) now write them. The programs are usually graded against answers written by someone who read the same law, so a program can score well by sharing the grader's reading rather than by getting the law right. LexTemporis grades them instead against records of what actually happened: 10,901 firm-years of SME status disclosed by Korean listed companies, with every version of the governing rules. An appendix adds comparison cases from US income tax and German hospital minimum volumes. The date changes outcomes mainly through provisions outside the main statutory text: one transitional provision raises agreement with the disclosures from 83-86% to 89-90%. The disclosures also test the author's reading of a contested provision, favouring it in 130 of 189 firm-years where readings diverge. Four LLMs wrote the rules, given different selections of the law. A pre-registered comparison finds no condition better across all four models. In exploratory analyses, supplying the right provisions looks better graded against the author's reading than against the records: the advantage shrinks for three models (for one, to between a quarter and three-fifths of its size), and per firm-year it is several times larger where the author's reading is wrong, partly by construction, since there reproducing that reading more faithfully must lose. Grading legal programs against the grader's own reading can overstate how well they work.

## Evidence-Carrying Contracts for Safe Claim Composition in Data Marketplace Workflows

DevFeed: [Evidence-Carrying Contracts for Safe Claim Composition in Data Marketplace Workflows](<https://devfeed.tech/articles/evidence-carrying-contracts-for-safe-claim-composition-in-data-marketplace-workflows-142148.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.23276>)

Author: Duc-Trung Bui, Phuong-Chi Le, Quyet-Thang Huynh

Published: 2026-11-19T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Software Engineering](<https://devfeed.tech/sources/arxiv-software-engineering.md>)

Topics: [audit](<https://devfeed.tech/topics/audit.md>), [Finite-state machine](<https://devfeed.tech/topics/finite-state-machine.md>), [Test-driven development](<https://devfeed.tech/topics/tdd.md>)

Tags: [aggregation](<https://devfeed.tech/tags/aggregation.md>), [api](<https://devfeed.tech/tags/api.md>), [blocking](<https://devfeed.tech/tags/blocking.md>), [completeness](<https://devfeed.tech/tags/completeness.md>), [composition](<https://devfeed.tech/tags/composition.md>), [contracts](<https://devfeed.tech/tags/contracts.md>), [covers](<https://devfeed.tech/tags/covers.md>), [critical](<https://devfeed.tech/tags/critical.md>), [cs-se](<https://devfeed.tech/tags/cs-se.md>), [data](<https://devfeed.tech/tags/data.md>), [data-marketplace](<https://devfeed.tech/tags/data-marketplace.md>), [declarative](<https://devfeed.tech/tags/declarative.md>), [defects](<https://devfeed.tech/tags/defects.md>), [delivery](<https://devfeed.tech/tags/delivery.md>), [dependencies](<https://devfeed.tech/tags/dependencies.md>), [diagnostic](<https://devfeed.tech/tags/diagnostic.md>), [domain](<https://devfeed.tech/tags/domain.md>), [guarantees](<https://devfeed.tech/tags/guarantees.md>), [harness](<https://devfeed.tech/tags/harness.md>), [independent](<https://devfeed.tech/tags/independent.md>), [workflows](<https://devfeed.tech/tags/workflows.md>)

### AI overview

The paper proposes evidence-carrying claim contracts to prevent data marketplace workflows from promoting limited evidence--such as payment, policy presence, or API delivery--into unsupported transaction claims. Contracts declare assumptions, guarantees, blocking obligations, and evidence dependencies; a four-state model tracks requirements as supported, refuted, unresolved, or conflicting. Composition bounds claims by local limits and blocks full promotion while a critical obligation remains open. The authors report executable conformance checks across workflow configurations, revocations, alternative support, and aggregation schedules, plus an independent evaluator that agrees with the declarative oracle. The system preserves a full claim when valid alternative support remains and blocks it when support is absent. The evidence demonstrates conformance to declared contracts, not that the specifications are complete, evidence is authentic, or claims are legally valid.

### Source excerpt

arXiv:2611.23276v1 Announce Type: new Abstract: Data marketplace workflows cross organizational boundaries, where payment, policy presence, or API delivery may be promoted into transaction claims that the evidence does not support. We introduce evidence-carrying claim contracts: a lightweight assume-guarantee semantics declaring assumptions, guarantees, blocking obligations, and evidence dependencies. A conflict-sensitive four-state domain distinguishes supported, refuted, unresolved, and conflicting requirements. Safe composition bounds the transaction claim by the meet of local bounds and blocks full promotion while a critical obligation remains open. We establish local soundness, compositional boundedness, revocation safety, termination, and aggregation-order invariance for finite acyclic workflows. An executable conformance harness covers all 4,096 configurations (four states for each of six stages), 18 mutations, six basic revocations, 12 alternative-support scenarios, an eight-transition evidence trace, 2,880 aggregation schedules, 96 requirement-level and six interface checks. An independent operational evaluator agrees with the declarative oracle; mutation-sensitive campaigns detect all eight seeded defects, though no single campaign detects all. It issues no unsupported full claim, preserves a full claim under valid alternative support, and blocks it when none remains. Four diagnostic negative controls, not competitive baselines, issue 63-1,023 unsupported full claims. These results establish executable conformance to declared contracts, not specification completeness, evidence authenticity, or legal validity.

## When Figures Disagree with Text: Evaluating Autonomous Visual Evidence Verification in Scientific Papers

DevFeed: [When Figures Disagree with Text: Evaluating Autonomous Visual Evidence Verification in Scientific Papers](<https://devfeed.tech/articles/when-figures-disagree-with-text-evaluating-autonomous-visual-evidence-verification-in-scientific-papers-142783.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.24353>)

Author: Guijia Zhang, Hongbo Qiu, Jiajun Zha, Guohao Cai, Hao Zheng, Huihao Jing, Kexu Cheng, Shipeng Luo, JinYang Huang, Shing-Chi Cheung, Changsheng Lu, Haoxuan Che, Harry Yang

Published: 2026-11-19T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [test](<https://devfeed.tech/topics/test.md>), [Image](<https://devfeed.tech/topics/image.md>), [audit](<https://devfeed.tech/topics/audit.md>)

Tags: [artificial-intelligence](<https://devfeed.tech/tags/artificial-intelligence.md>), [autonomous](<https://devfeed.tech/tags/autonomous.md>), [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [multimodal](<https://devfeed.tech/tags/multimodal.md>), [paper](<https://devfeed.tech/tags/paper.md>), [papers](<https://devfeed.tech/tags/papers.md>), [pdf](<https://devfeed.tech/tags/pdf.md>), [reviews](<https://devfeed.tech/tags/reviews.md>), [study](<https://devfeed.tech/tags/study.md>), [verification](<https://devfeed.tech/tags/verification.md>)

### AI overview

In a study of 31 manuscripts with hidden, reviewer-documented figure-text inconsistencies, six multimodal models often identified the flawed figure but rarely diagnosed its contradiction: 60.6% of 1,445 open-ended reviews mentioned it, while 3.3% met the rubric. The authors caution that a blind audit found the rubric lenient, so they treat accepted rates as comparative rather than absolute. In a controlled run, naming the figure raised accepted diagnosis from 4.3% to 20.4%; a consistency instruction and coarse visual cues did not. For contradictions visible only in rendered figures, text-only review had no accepted diagnoses, versus 15.8% when rendered pages were reviewed and the figure named. A centrality-based figure-selection baseline reached 4.8%. The findings suggest that deciding what evidence to check remains a major challenge; the tested selection rule did not solve it.

### Source excerpt

arXiv:2611.24353v1 Announce Type: new Abstract: Scientific figures carry evidence for the claims that cite them, so reviewing a paper means checking that a figure agrees with the text, and not only describing what the figure shows. Benchmarks of figure-text verification typically hand the model the figure, the claim, or the knowledge that an inconsistency exists, which leaves untested whether a model can find the figure worth checking in a complete paper, relate a visual element to the claim it conflicts with, and recognize the conflict. We study this autonomous setting on 31 full manuscripts, each carrying a reviewer-documented figure-text inconsistency from PRISMM-Bench that is hidden from the model. Six multimodal models review each PDF under a neutral prompt, and we then add one input at a time to the same manuscripts. Of 1,445 open-ended reviews, 60.6% mention the flawed figure, yet a grading rubric accepts only 3.3% as diagnosing its contradiction; a blind audit finds the rubric lenient, so we read these as rubric-accepted rates and compare conditions rather than absolute values. In a separate controlled run, naming the figure, without stating that it is flawed, raises rubric-accepted diagnosis from 4.3% to 20.4%, while a consistency instruction and coarse visual cues do not. When the contradiction is visible only in the rendered figure, text-only review yields no accepted diagnosis, while review of the rendered pages with the figure named reaches 15.8%. A baseline that lets the model choose which figures to check, ranked by centrality, stays at 4.8%. Deciding what to verify is thus a large part of the difficulty, and the selection rule we tested does not resolve it. Multimodal document understanding should be evaluated on whether a model can propose what to verify, not only answer questions about a figure it is handed.

## Retain more logs for longer with Flex Logs BYOBucket

DevFeed: [Retain more logs for longer with Flex Logs BYOBucket](<https://devfeed.tech/articles/retain-more-logs-for-longer-with-flex-logs-byobucket-145232.md>)

Original publisher: [Read original article](<https://www.datadoghq.com/blog/flex-logs-bring-your-own-bucket/>)

Author: Corey Brincks; Bobby Vellanki

Published: 2026-11-19T00:00:00Z

Content type: article

Language: en

Sources: [Datadog](<https://devfeed.tech/sources/datadog-the-monitor-blog.md>)

Topics: [datadog](<https://devfeed.tech/topics/datadog.md>), [telemetry](<https://devfeed.tech/topics/telemetry.md>), [Logging and metrics](<https://devfeed.tech/topics/logging-and-metrics.md>), [audit](<https://devfeed.tech/topics/audit.md>)

Tags: [audit-logs](<https://devfeed.tech/tags/audit-logs.md>), [bucket](<https://devfeed.tech/tags/bucket.md>), [cloud](<https://devfeed.tech/tags/cloud.md>), [costs](<https://devfeed.tech/tags/costs.md>), [datadog](<https://devfeed.tech/tags/datadog.md>), [flex-logs](<https://devfeed.tech/tags/flex-logs.md>), [investigations](<https://devfeed.tech/tags/investigations.md>), [log-management](<https://devfeed.tech/tags/log-management.md>), [logs](<https://devfeed.tech/tags/logs.md>), [retention](<https://devfeed.tech/tags/retention.md>), [storage](<https://devfeed.tech/tags/storage.md>)

### AI overview

Datadog Flex Logs BYOBucket, available in Preview, lets teams store indexed logs in their own cloud bucket while keeping them searchable in Datadog. Customers pay their cloud provider for storage at their negotiated rates; Datadog charges a fixed monthly fee for compute capacity, and writes logs in compressed form. Teams control bucket retention and can keep historical logs available for investigations and audits without Datadog storage fees for that retention. Flex Logs supports Log Explorer, dashboards, log-based monitoring, and querying with Bits Chat and MCP-compatible tools; Standard Indexing is not required.

### Source excerpt

Store Flex Logs in your cloud bucket with unlimited retention to control storage costs and keep logs searchable in Datadog for investigations and audits.

## Design responsible AI controls for Amazon Aurora PostgreSQL

DevFeed: [Design responsible AI controls for Amazon Aurora PostgreSQL](<https://devfeed.tech/articles/design-responsible-ai-controls-for-amazon-aurora-postgresql-141279.md>)

Original publisher: [Read original article](<https://aws.amazon.com/blogs/machine-learning/design-responsible-ai-controls-for-amazon-aurora-postgresql/>)

Author: InduTeja Aligeti

Published: 2026-11-18T21:38:22Z

Content type: article

Language: en

Sources: [Artificial Intelligence](<https://devfeed.tech/sources/artificial-intelligence.md>)

Topics: [Amazon Aurora](<https://devfeed.tech/topics/amazon-aurora.md>), [responsible-ai](<https://devfeed.tech/topics/responsible-ai.md>), [AI Chat](<https://devfeed.tech/topics/ai-chat.md>), [ai-governance](<https://devfeed.tech/topics/ai-governance.md>), [Amazon Bedrock Knowledge Bases](<https://devfeed.tech/topics/amazon-bedrock-knowledge-bases.md>), [audit](<https://devfeed.tech/topics/audit.md>), [API Logging](<https://devfeed.tech/topics/api-logging.md>), [Amazon CloudWatch](<https://devfeed.tech/topics/amazon-cloudwatch.md>), [Amazon S3](<https://devfeed.tech/topics/amazon-s3.md>)

Tags: [advanced-300](<https://devfeed.tech/tags/advanced-300.md>), [ai](<https://devfeed.tech/tags/ai.md>), [amazon](<https://devfeed.tech/tags/amazon.md>), [amazon-aurora](<https://devfeed.tech/tags/amazon-aurora.md>), [amazon-bedrock](<https://devfeed.tech/tags/amazon-bedrock.md>), [amazon-bedrock-guardrails](<https://devfeed.tech/tags/amazon-bedrock-guardrails.md>), [chatbot](<https://devfeed.tech/tags/chatbot.md>), [controls](<https://devfeed.tech/tags/controls.md>), [design](<https://devfeed.tech/tags/design.md>), [guardrails](<https://devfeed.tech/tags/guardrails.md>), [logging](<https://devfeed.tech/tags/logging.md>), [pgvector](<https://devfeed.tech/tags/pgvector.md>), [postgresql](<https://devfeed.tech/tags/postgresql.md>), [rag](<https://devfeed.tech/tags/rag.md>), [responsible-ai](<https://devfeed.tech/tags/responsible-ai.md>), [sources](<https://devfeed.tech/tags/sources.md>), [sql](<https://devfeed.tech/tags/sql.md>), [technical-how-to](<https://devfeed.tech/tags/technical-how-to.md>), [traces](<https://devfeed.tech/tags/traces.md>)

### AI overview

The article details an Explainability design for a customer-support RAG chatbot using Aurora PostgreSQL, pgvector, Aurora ML, and Amazon Bedrock. It embeds documents and queries with the same Titan Text Embeddings V2 model, retrieves tenant-scoped chunks by cosine similarity, applies Bedrock Guardrails for privacy and contextual grounding, and logs retrieved chunk IDs, scores, actually cited sources, guardrail traces, and model ID in SQL. It stresses protecting interaction logs with PII redaction or encryption and row-level security, and archiving records to S3 before purging. Amazon Bedrock Knowledge Bases is presented as a simpler option when its default retrieval and citations suffice; custom logging suits cases needing provenance in a user-managed schema. Limitations include guardrail latency and charges, approximate HNSW search that may return fewer results after filtering, grounding that does not verify source-document accuracy, and synchronous embedding generation for large ingestions.

### Source excerpt

Learn how to make RAG chatbot answers traceable with Amazon Aurora PostgreSQL, pgvector, and Amazon Bedrock Guardrails, logging sources, scores, and guardrail traces in SQL.

## How to use Amazon Aurora DSQL audit logs

DevFeed: [How to use Amazon Aurora DSQL audit logs](<https://devfeed.tech/articles/how-to-use-amazon-aurora-dsql-audit-logs-141385.md>)

Original publisher: [Read original article](<https://aws.amazon.com/blogs/database/how-to-use-amazon-aurora-dsql-audit-logs/>)

Author: Koji Shinkubo

Published: 2026-11-18T21:33:01Z

Content type: tutorial

Language: en

Sources: [AWS Database Blog](<https://devfeed.tech/sources/aws-database-blog.md>)

Topics: [DSQL](<https://devfeed.tech/topics/dsql.md>), [audit](<https://devfeed.tech/topics/audit.md>), [Logging](<https://devfeed.tech/topics/logging.md>), [Amazon S3](<https://devfeed.tech/topics/amazon-s3.md>), [Amazon CloudWatch](<https://devfeed.tech/topics/amazon-cloudwatch.md>), [Tool](<https://devfeed.tech/topics/tool.md>), [Query (disambiguation)](<https://devfeed.tech/topics/query.md>)

Tags: [advanced-300](<https://devfeed.tech/tags/advanced-300.md>), [amazon-athena](<https://devfeed.tech/tags/amazon-athena.md>), [amazon-aurora](<https://devfeed.tech/tags/amazon-aurora.md>), [amazon-s3](<https://devfeed.tech/tags/amazon-s3.md>), [athena](<https://devfeed.tech/tags/athena.md>), [audit](<https://devfeed.tech/tags/audit.md>), [audit-logging](<https://devfeed.tech/tags/audit-logging.md>), [audit-logs](<https://devfeed.tech/tags/audit-logs.md>), [configure](<https://devfeed.tech/tags/configure.md>), [database](<https://devfeed.tech/tags/database.md>), [dsql](<https://devfeed.tech/tags/dsql.md>), [field](<https://devfeed.tech/tags/field.md>), [how-to](<https://devfeed.tech/tags/how-to.md>), [logging](<https://devfeed.tech/tags/logging.md>), [logs](<https://devfeed.tech/tags/logs.md>), [post](<https://devfeed.tech/tags/post.md>), [query](<https://devfeed.tech/tags/query.md>), [records](<https://devfeed.tech/tags/records.md>), [requirements](<https://devfeed.tech/tags/requirements.md>), [s3](<https://devfeed.tech/tags/s3.md>), [sql](<https://devfeed.tech/tags/sql.md>), [technical-how-to](<https://devfeed.tech/tags/technical-how-to.md>)

### AI overview

Aurora DSQL audit logging records configured SQL statement classes and delivers records to S3; it is off until both a destination and recording settings are configured. The article describes role-based scope, including READ/SELECT, and a setup that separates audit administration from DBA permissions. Records can identify the IAM role and database role, transaction, statement, and optionally bound values and row counts. Athena can query the S3 logs, while CloudTrail connection events can be joined by session ID to add client IP information; transaction IDs can also link audit records to CDC changes. Limits and tradeoffs include potentially high log volume and charges, no table- or column-level scope, no client IP or success/failure field in audit records, and recording statements from rolled-back transactions. The article recommends testing scope and volume before production and protecting sensitive log content through access controls, encryption, and retention settings.

### Source excerpt

Amazon Aurora DSQL audit logging records the SQL statements that run in your database and delivers them to Amazon S3. This post shows how to configure audit logging to meet regulatory requirements, explains what each recorded field proves in an audit, and shows how to query the logs in Amazon S3 with Amazon Athena.

## The Trust Layer: Securing Agentic AI with Docker AI Governance

DevFeed: [The Trust Layer: Securing Agentic AI with Docker AI Governance](<https://devfeed.tech/articles/the-trust-layer-securing-agentic-ai-with-docker-ai-governance-141263.md>)

Original publisher: [Read original article](<https://www.docker.com/blog/agentic-ai-security-docker-governance/>)

Author: Jennifer Kohl

Published: 2026-11-18T14:00:00Z

Content type: article

Language: en

Sources: [Docker](<https://devfeed.tech/sources/docker.md>)

Topics: [Docker AI Governance](<https://devfeed.tech/topics/docker-ai-governance.md>), [ai-governance](<https://devfeed.tech/topics/ai-governance.md>), [Agentic AI](<https://devfeed.tech/topics/what-is-agentic-ai.md>), [Docker Hardened Images](<https://devfeed.tech/topics/docker-hardened-images.md>), [audit](<https://devfeed.tech/topics/audit.md>), [trust](<https://devfeed.tech/topics/trust.md>)

Tags: [agent](<https://devfeed.tech/tags/agent.md>), [agent-behavior](<https://devfeed.tech/tags/agent-behavior.md>), [agent-trust](<https://devfeed.tech/tags/agent-trust.md>), [agent-workflows](<https://devfeed.tech/tags/agent-workflows.md>), [agentic](<https://devfeed.tech/tags/agentic.md>), [agentic-ai](<https://devfeed.tech/tags/agentic-ai.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-governance](<https://devfeed.tech/tags/ai-governance.md>), [audit](<https://devfeed.tech/tags/audit.md>), [community](<https://devfeed.tech/tags/community.md>), [docker](<https://devfeed.tech/tags/docker.md>), [docker-ai-governance](<https://devfeed.tech/tags/docker-ai-governance.md>), [governance](<https://devfeed.tech/tags/governance.md>), [mcp](<https://devfeed.tech/tags/mcp.md>), [mcp-tool](<https://devfeed.tech/tags/mcp-tool.md>), [policies](<https://devfeed.tech/tags/policies.md>), [sandbox](<https://devfeed.tech/tags/sandbox.md>), [sandboxes](<https://devfeed.tech/tags/sandboxes.md>), [tool](<https://devfeed.tech/tags/tool.md>), [trust](<https://devfeed.tech/tags/trust.md>), [visibility](<https://devfeed.tech/tags/visibility.md>), [workflows](<https://devfeed.tech/tags/workflows.md>)

### AI overview

Docker AI Governance places enforceable controls around agent actions through three layers: sandbox policies restrict network and filesystem access; MCP Tool Governance controls servers and actions routed through Docker's MCP Gateway, with optional confirmation requirements; and audit records capture covered policy decisions and session events for local or cloud review. MCP policies do not govern direct connections to remote MCP servers, which instead fall under sandbox network policy. Audit coverage described is limited to Docker Sandboxes policy decisions and session events, and records contain governance metadata rather than prompts or responses. Docker Hardened Images address a separate supply-chain concern with signed SBOMs, VEX, signatures, and SLSA Build Level 3 provenance. Together, the controls bound runtime authority and provide visibility while agents work.

### Source excerpt

Learn how Docker AI Governance secures agentic AI with sandbox policies, MCP tool governance, and audit visibility to support controlled AI agent workflows.

## Compliance Memory: Keep Rule-to-Control Links When Staff Leave

DevFeed: [Compliance Memory: Keep Rule-to-Control Links When Staff Leave](<https://devfeed.tech/articles/compliance-memory-keep-rule-to-control-links-when-staff-leave-141052.md>)

Original publisher: [Read original article](<https://www.lyzr.ai/blog/compliance-memory-in-banks/>)

Author: Lyzr Team

Published: 2026-11-18T11:45:51Z

Content type: article

Language: en

Sources: [Lyzr.ai](<https://devfeed.tech/sources/lyzr-ai.md>)

Topics: [ai-governance](<https://devfeed.tech/topics/ai-governance.md>), [human review](<https://devfeed.tech/topics/human-review.md>), [audit](<https://devfeed.tech/topics/audit.md>), [control-plane](<https://devfeed.tech/topics/control-plane.md>)

Tags: [accountability](<https://devfeed.tech/tags/accountability.md>), [ai-agents](<https://devfeed.tech/tags/ai-agents.md>), [bank](<https://devfeed.tech/tags/bank.md>), [compliance](<https://devfeed.tech/tags/compliance.md>), [control](<https://devfeed.tech/tags/control.md>), [controls](<https://devfeed.tech/tags/controls.md>), [leave](<https://devfeed.tech/tags/leave.md>), [links](<https://devfeed.tech/tags/links.md>), [memory](<https://devfeed.tech/tags/memory.md>), [policies](<https://devfeed.tech/tags/policies.md>), [rules](<https://devfeed.tech/tags/rules.md>)

### AI overview

"Compliance memory" is the article's nonstandard term for preserving links among bank rules, policies, controls, evidence, and the reasons for those links so staff turnover does not erase institutional knowledge. It recommends recording rationale and signed approvals before departures, assigning ownership to persistent roles with fallbacks, and using agents to propose and re-check mappings, flag stale evidence or gaps, and route work for human approval. Agents cannot recover undocumented reasons and should not present guesses as fact or remediate automatically. Madison is described as storing this record for US banks and credit unions; the excerpt says an HR offboarding integration is unconfirmed and long-term tamper-resistant retention depends on the customer's environment. Opencontroller is described as a separate runtime control layer. The term and product claims are presented by a vendor, and cited regulatory expectations were not verified against primary sources.

### Source excerpt

Compliance memory is our shorthand for a record that keeps the links between a bank's rules, policies and controls, and the reasons behind them, so...

## Auditing AI Agents in Banks: What Changes for Internal Audit

DevFeed: [Auditing AI Agents in Banks: What Changes for Internal Audit](<https://devfeed.tech/articles/auditing-ai-agents-in-banks-what-changes-for-internal-audit-141048.md>)

Original publisher: [Read original article](<https://www.lyzr.ai/blog/auditing-ai-agents-in-banks/>)

Author: Lyzr Team

Published: 2026-11-18T10:17:35Z

Content type: article

Language: en

Sources: [Lyzr.ai](<https://devfeed.tech/sources/lyzr-ai.md>)

Topics: [audit](<https://devfeed.tech/topics/audit.md>)

Tags: [accountability](<https://devfeed.tech/tags/accountability.md>), [actions](<https://devfeed.tech/tags/actions.md>), [agent](<https://devfeed.tech/tags/agent.md>), [agent-behavior](<https://devfeed.tech/tags/agent-behavior.md>), [agent-control](<https://devfeed.tech/tags/agent-control.md>), [agent-identity](<https://devfeed.tech/tags/agent-identity.md>), [agent-observability](<https://devfeed.tech/tags/agent-observability.md>), [agent-safety](<https://devfeed.tech/tags/agent-safety.md>), [agentic](<https://devfeed.tech/tags/agentic.md>), [agentic-ai](<https://devfeed.tech/tags/agentic-ai.md>), [agents](<https://devfeed.tech/tags/agents.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-agents](<https://devfeed.tech/tags/ai-agents.md>), [audit](<https://devfeed.tech/tags/audit.md>), [auditing](<https://devfeed.tech/tags/auditing.md>), [banks](<https://devfeed.tech/tags/banks.md>), [changes](<https://devfeed.tech/tags/changes.md>), [continuous](<https://devfeed.tech/tags/continuous.md>), [defense](<https://devfeed.tech/tags/defense.md>), [testing](<https://devfeed.tech/tags/testing.md>)

### AI overview

Bank internal audit of agentic AI depends on an action trail captured when decisions occur: it should link agent identity and permissions to decisions, approvals, escalations, overrides, tests, and changes. KPMG and IIA Singapore favor real-time monitoring over retrospective sampling, while Sphere notes auditors may still sample when the full population is available. The key change is that sampling becomes a choice, provided the trail is complete. Auditors should verify the agent inventory against live systems, prioritize work by risk, replay real decisions from records, test approval gates for meaningful review, and retest after model, tool, data, or rule changes. Agents can retrieve evidence and run defined tests, but auditors retain judgment over materiality, context, and scope. The excerpt says US supervisors had not issued audit-specific agent rules among those reviewed, and the sources did not address auditor independence when using agents. Madison is described as a first and second line system whose records auditors should test independently.

### Source excerpt

Agentic AI changes the third line of defense in three ways: audit moves from periodic samples toward continuous testing, evidence has to come from a...

[Next page](<https://devfeed.tech/topics/audit.md?cursor=WyIyMDI2LTExLTE4VDEwOjE3OjM1KzAwOjAwIiwgIjJkY2NmZTYxLTIyYmYtNGJhNS04ZmMyLTg3Yzg1NzI4MGM2YiJd>)