# Agent Harness

An agent harness is software runtime scaffolding that connects a language model to context and tools, coordinates tool calls and approvals, and maintains session state.

This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles.

## Best Agent Frameworks for Production Use

DevFeed: [Best Agent Frameworks for Production Use](<https://devfeed.tech/articles/best-agent-frameworks-for-production-use-146945.md>)

Original publisher: [Read original article](<https://www.swytchcode.com/content/best-agent-frameworks-production>)

Author: Swytchcode Team

Published: 2026-11-20T09:00:00Z

Content type: comparison

Language: en

Sources: [Swytchcode](<https://devfeed.tech/sources/swytchcode.md>)

Topics: [Microsoft Agent Framework](<https://devfeed.tech/topics/microsoft-agent-framework.md>), [Multi Agent Systems](<https://devfeed.tech/topics/multi-agent-systems.md>), [Agent Harness](<https://devfeed.tech/topics/agent-harness.md>), [ai-governance](<https://devfeed.tech/topics/ai-governance.md>), [audit](<https://devfeed.tech/topics/audit.md>), [agent observability](<https://devfeed.tech/topics/agent-observability.md>), [Production Engineering](<https://devfeed.tech/topics/production-engineering.md>), [SDK](<https://devfeed.tech/topics/sdk.md>), [llm-reasoning](<https://devfeed.tech/topics/llm-reasoning.md>)

Tags: [2026](<https://devfeed.tech/tags/2026.md>), [agent-frameworks](<https://devfeed.tech/tags/agent-frameworks.md>), [ai-agents](<https://devfeed.tech/tags/ai-agents.md>), [audit](<https://devfeed.tech/tags/audit.md>), [best](<https://devfeed.tech/tags/best.md>), [permissions](<https://devfeed.tech/tags/permissions.md>), [production](<https://devfeed.tech/tags/production.md>), [retries](<https://devfeed.tech/tags/retries.md>)

### AI overview

The excerpt's central finding is that agent frameworks manage reasoning and tool orchestration, while API execution controls such as credentials, policy checks, idempotent retries, and independent audit generally require another layer. LangGraph is suited to branching, stateful workflows with checkpointing and human review; OpenAI Agents SDK offers native handoffs and scoped guardrails, with gaps around handoffs, hosted tools, retries, and audit; Claude Agent SDK's local tool approvals fit supervised coding and internal tools. CrewAI adds runtime controls and credential hygiene, while Vercel AI SDK offers provider and runtime flexibility but leaves credentials, sandboxing, approvals, and idempotency to the caller. Microsoft Agent Framework provides broad orchestration, with governance positioned in Azure AI Foundry. Temporal adds durable state and retries after failures, but does not decide whether a call is authorized. The excerpt describes a separate execution layer as handling allowlists, live schema validation, policy, credential injection, safe retries, response checks, and audit.

### Source excerpt

We rank six agent frameworks used in production as of November 2026 and show why none of them handle API permissions, retries, or audit by design.

## Meta-RTL: Evolving Agent Harnesses for RTL Optimization

DevFeed: [Meta-RTL: Evolving Agent Harnesses for RTL Optimization](<https://devfeed.tech/articles/meta-rtl-evolving-agent-harnesses-for-rtl-optimization-145602.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.25891>)

Author: Xinzhuang Xiong, Yonghao Wang, Shengyao Sun, Chengxi Li, Evangeline Young

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [Agent Harness](<https://devfeed.tech/topics/agent-harness.md>), [llm-reasoning](<https://devfeed.tech/topics/llm-reasoning.md>), [Combinatorial optimization](<https://devfeed.tech/topics/combinatorial-optimization.md>)

Tags: [agent](<https://devfeed.tech/tags/agent.md>), [agent-harnesses](<https://devfeed.tech/tags/agent-harnesses.md>), [agents](<https://devfeed.tech/tags/agents.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-ar](<https://devfeed.tech/tags/cs-ar.md>), [eda](<https://devfeed.tech/tags/eda.md>), [evolution](<https://devfeed.tech/tags/evolution.md>), [framework](<https://devfeed.tech/tags/framework.md>), [optimization](<https://devfeed.tech/tags/optimization.md>), [rtl](<https://devfeed.tech/tags/rtl.md>), [search](<https://devfeed.tech/tags/search.md>)

### AI overview

Meta-RTL evolves RTL optimization harnesses from a minimal loop, using EDA feedback while keeping the optimization model, EDA flow, and per-design budgets fixed. Compared with Dr. RTL, Meta-RTL* reduced aggregate WNS/TNS violations by 7.8%/11.0% across 20 development designs, with area reductions on 70%. Without further evolution, it transferred to 10 unseen IC-RTL designs, reducing aggregate WNS/TNS violations by 20.5%/14.0% and total synthesized area by 12.6%. The abstract also says it characterizes three search mechanisms and evaluates their contributions through ablations and a case study.

### Source excerpt

arXiv:2611.25891v1 Announce Type: new Abstract: Large language models enable agents to optimize RTL using EDA feedback, yet their executable search procedures remain largely hand- engineered. We present Meta-RTL, an EDA-grounded framework that evolves RTL optimization harnesses from a minimal loop while keeping the optimization model, EDA flow, and per-design budgets fixed. Compared with Dr. RTL, Meta-RTL* reduces aggregate WNS/TNS violations by 7.8%/11.0% across 20 development designs, with area reductions on 70% of them. Without further evolution, Meta-RTL* transfers to 10 unseen IC-RTL designs, reducing aggregate WNS/TNS violations by 20.5%/14.0% and total synthesized area by 12.6%. We further characterize three search mechanisms in the evolved harness and assess their contributions through ablations and a case study.

## MATE: Mobile Agent Trustworthy Execution for Safe and Efficient Mobile Control

DevFeed: [MATE: Mobile Agent Trustworthy Execution for Safe and Efficient Mobile Control](<https://devfeed.tech/articles/mate-mobile-agent-trustworthy-execution-for-safe-and-efficient-mobile-control-145527.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.25526>)

Author: Minsuk Jang, Hee-Seon Kim, Hui-Cheol Yu, Changick Kim

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [computer-use](<https://devfeed.tech/topics/computer-use.md>), [Agent Harness](<https://devfeed.tech/topics/agent-harness.md>), [human-in-the-loop UX](<https://devfeed.tech/topics/human-in-the-loop-ux.md>)

Tags: [agent](<https://devfeed.tech/tags/agent.md>), [block](<https://devfeed.tech/tags/block.md>), [blocking](<https://devfeed.tech/tags/blocking.md>), [boundary](<https://devfeed.tech/tags/boundary.md>), [control](<https://devfeed.tech/tags/control.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cl](<https://devfeed.tech/tags/cs-cl.md>), [data](<https://devfeed.tech/tags/data.md>), [decision](<https://devfeed.tech/tags/decision.md>), [device](<https://devfeed.tech/tags/device.md>), [effects](<https://devfeed.tech/tags/effects.md>), [execution](<https://devfeed.tech/tags/execution.md>), [fields](<https://devfeed.tech/tags/fields.md>), [gpt](<https://devfeed.tech/tags/gpt.md>), [guardrail](<https://devfeed.tech/tags/guardrail.md>), [interface](<https://devfeed.tech/tags/interface.md>), [layer](<https://devfeed.tech/tags/layer.md>), [llm](<https://devfeed.tech/tags/llm.md>), [mate](<https://devfeed.tech/tags/mate.md>), [mobile](<https://devfeed.tech/tags/mobile.md>), [model](<https://devfeed.tech/tags/model.md>), [safe](<https://devfeed.tech/tags/safe.md>)

### AI overview

MATE adds an execution layer between a mobile agent's proposed actions and their effects on the device, without modifying the underlying model. It records each UI action with its instruction, target app, screen context, primitive, and action fields. Runtime Guardrail uses that record to allow, block, or request user approval; Predefined Task Routing handles recurring requests through verified routes and checks the screen afterward. A compact floating overlay hosts execution. In reported evaluations, MATE increased high-risk blocking from 29.7% to 97.0% on MobileSafetyBench with GPT-4o; on AndroidWorld, it cut LLM calls by up to 6x and achieved 95.7% success on recurring-use tasks.

### Source excerpt

arXiv:2611.25526v1 Announce Type: cross Abstract: Mobile agents on personal smartphones can affect private data, payments, settings, and the foreground screen. We present MATE (Mobile Agent Trustworthy Execution), an execution-layer system that controls the boundary between agent action proposals and device effects without modifying the underlying model. MATE represents each UI primitive as a structured action record that combines the instruction, target app, screen context, primitive, and action fields. Runtime Guardrail (RG) maps this record to an allow, block, or user-approval decision; Predefined Task Routing (PTR) serves recurring requests through verified routes with post-execution screen checks; and a floating interface keeps execution in a compact overlay. On MobileSafetyBench, MATE raises high-risk blocking from 29.7% to 97.0% under GPT-4o. On AndroidWorld, it reduces LLM calls by up to 6x and raises recurring-use task success to 95.7%.

## Beyond Weight Updates: Evolving Agent Harnesses for Domain Specialization

DevFeed: [Beyond Weight Updates: Evolving Agent Harnesses for Domain Specialization](<https://devfeed.tech/articles/beyond-weight-updates-evolving-agent-harnesses-for-domain-specialization-145504.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.25380>)

Author: BoHao Chen

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [Agent Harness](<https://devfeed.tech/topics/agent-harness.md>), [AI Agent](<https://devfeed.tech/topics/ai-agent.md>), [llm-reasoning](<https://devfeed.tech/topics/llm-reasoning.md>), [Fine-tuning](<https://devfeed.tech/topics/fine-tuning.md>)

Tags: [adapters](<https://devfeed.tech/tags/adapters.md>), [agent](<https://devfeed.tech/tags/agent.md>), [agent-harness](<https://devfeed.tech/tags/agent-harness.md>), [agent-harnesses](<https://devfeed.tech/tags/agent-harnesses.md>), [coding-agent](<https://devfeed.tech/tags/coding-agent.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [domain](<https://devfeed.tech/tags/domain.md>), [gepa](<https://devfeed.tech/tags/gepa.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [llm](<https://devfeed.tech/tags/llm.md>), [llm-agents](<https://devfeed.tech/tags/llm-agents.md>), [planning](<https://devfeed.tech/tags/planning.md>), [policy](<https://devfeed.tech/tags/policy.md>), [prompt](<https://devfeed.tech/tags/prompt.md>), [reasoning](<https://devfeed.tech/tags/reasoning.md>), [reinforcement-learning](<https://devfeed.tech/tags/reinforcement-learning.md>), [specialization](<https://devfeed.tech/tags/specialization.md>), [swe-bench](<https://devfeed.tech/tags/swe-bench.md>), [terminal-bench](<https://devfeed.tech/tags/terminal-bench.md>), [weight](<https://devfeed.tech/tags/weight.md>)

### AI overview

The study evolves an agent's system prompt, tool schemas, skills, and planning scaffolding while keeping its policy weights frozen. It frames this as an RL-style outer loop and uses GEPA, a reflective evolutionary optimizer, to produce domain-specific harness adapters that require no GPU training and can be combined with weight-based RL. In experiments with Codev on SWE-bench Verified, OfficeBench, and Terminal-Bench, weight updates gave the strongest gains when used alone; harness evolution recovered most of those gains at a small fraction of the compute, and combining the approaches performed best. The authors also analyze SFT specialization's shortened reasoning chains and increased truncation, arguing that harness adapters avoid this failure mode by construction. These findings are limited to the reported agent and benchmarks.

### Source excerpt

arXiv:2611.25380v1 Announce Type: new Abstract: Domain specialization of LLM agents is usually framed as a weight-update problem: collect trajectories, run reinforcement learning (RL) or supervised fine-tuning (SFT), and deploy a new checkpoint. We study a complementary axis: evolving the \textbf{agent harness}---the system prompt, tool schemas, skill library, and planning scaffolding---while keeping the policy frozen. We cast harness evolution as an RL-style outer loop and instantiate it with GEPA, a reflective evolutionary optimizer. The resulting harness adapters require no GPU training, are loadable per vertical domain, and compose with weight-based RL. Experiments with Codev, a Claude Code-style coding agent, on SWE-bench Verified, OfficeBench, and Terminal-Bench show a consistent pattern: weight updates give the strongest single-path gains, harness evolution recovers most of those gains at a small fraction of the compute, and combining both paths is best overall. We also analyze a failure mode of SFT-based specialization---shortened reasoning chains and increased truncation---and argue that harness adapters avoid it by construction. Our results position harness evolution as a practical, non-destructive complement to weight updates for vertical agents.

## Spring AI 2.1.0 GA and 2.0.3 GA Available Now

DevFeed: [Spring AI 2.1.0 GA and 2.0.3 GA Available Now](<https://devfeed.tech/articles/spring-ai-2-1-0-ga-and-2-0-3-ga-available-now-146701.md>)

Original publisher: [Read original article](<https://spring.io/blog/2026/11/20/spring-ai-2-1-0-GA-available-now>)

Author: tzolov

Published: 2026-11-20T00:00:00Z

Content type: release

Language: en

Sources: [Spring](<https://devfeed.tech/sources/spring.md>)

Topics: [Spring AI](<https://devfeed.tech/topics/spring-ai.md>), [AI Chat](<https://devfeed.tech/topics/ai-chat.md>), [Agent Harness](<https://devfeed.tech/topics/agent-harness.md>), [AI Context Management](<https://devfeed.tech/topics/ai-context-management.md>), [Embeddings](<https://devfeed.tech/topics/embeddings.md>), [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [Multi-step AI processes](<https://devfeed.tech/topics/multi-step-ai-processes.md>)

Tags: [agents](<https://devfeed.tech/tags/agents.md>), [ai](<https://devfeed.tech/tags/ai.md>), [api](<https://devfeed.tech/tags/api.md>), [batch](<https://devfeed.tech/tags/batch.md>), [boot](<https://devfeed.tech/tags/boot.md>), [cloud](<https://devfeed.tech/tags/cloud.md>), [decision](<https://devfeed.tech/tags/decision.md>), [decision-models](<https://devfeed.tech/tags/decision-models.md>), [embeddings](<https://devfeed.tech/tags/embeddings.md>), [event-driven](<https://devfeed.tech/tags/event-driven.md>), [guardrail](<https://devfeed.tech/tags/guardrail.md>), [harness](<https://devfeed.tech/tags/harness.md>), [java](<https://devfeed.tech/tags/java.md>), [maven](<https://devfeed.tech/tags/maven.md>), [mcp](<https://devfeed.tech/tags/mcp.md>), [memory](<https://devfeed.tech/tags/memory.md>), [microservices](<https://devfeed.tech/tags/microservices.md>), [model-context-protocol](<https://devfeed.tech/tags/model-context-protocol.md>), [openai](<https://devfeed.tech/tags/openai.md>), [parts](<https://devfeed.tech/tags/parts.md>), [reactive](<https://devfeed.tech/tags/reactive.md>), [recall](<https://devfeed.tech/tags/recall.md>), [release](<https://devfeed.tech/tags/release.md>), [responses-api](<https://devfeed.tech/tags/responses-api.md>), [security](<https://devfeed.tech/tags/security.md>), [serverless](<https://devfeed.tech/tags/serverless.md>), [session](<https://devfeed.tech/tags/session.md>), [spring](<https://devfeed.tech/tags/spring.md>), [spring-ai](<https://devfeed.tech/tags/spring-ai.md>), [web-applications](<https://devfeed.tech/tags/web-applications.md>)

### AI overview

Spring AI 2.1.0 GA, built on Spring Boot 4.2, adds a stateless MCP 2026-07-28 protocol, typed Decision Models, ordered message parts that preserve reasoning and provider-specific payloads, and an OpenAI Responses API chat model. Its agent building blocks include a Session API with compaction and event recall (ChatMemory is deprecated for removal in 3.0), per-tool approval pauses, configurable per-turn call limits, and marked guardrail refusals. Eleven vector stores can now write and search vectors computed elsewhere, with width checks and stable-ID replacement. The release also includes reference-documentation changes and security fixes; Spring AI 2.0.3 carries the fixes on the 2.0.x line. Upgrading to 2.1 generally involves updating Spring AI and Spring Boot together, but some security fixes change behavior, and applications upgrading from 2.0.2 or earlier are directed to review the 2.0.2 and 2.0.3 notes.

### Source excerpt

On behalf of the Spring AI team and everyone who has contributed, I'm happy to announce that Spring AI 2.1.0 has been released and is now available from Maven Central! Release notes | Upgrade notes | Reference documentation Spring AI 2.1 arrives five months after 2.0.0 GA. It is built on Spring Boot 4.2 and contains more than 430 commits from 86 contributors. This post covers everything that changed since 2.0.0: the features of the 2.1 milestones and release candidates, what was added after RC2, and the additions that already shipped in 2.0.1 and 2.0.2. The short version: MCP 2026-07-28. Servers and clients for the new, stateless revision of the Model Context Protocol, next to the existing ones. Decision Models. A new model API for typed yes/no, choice and score decisions, with three providers. A richer message model. Messages hold ordered parts, so reasoning, tool calls and provider-specific blocks survive a round trip. The OpenAI Responses API is supported next to Chat Completions. The building blocks of an agent harness. The Session API, with session memory, compaction and recall, replaces ChatMemory. Tool calls can wait for a person's approval and are limited per turn, and guardrail refusals are marked as such. Your own embeddings. Eleven vector stores can write and search with vectors you computed yourself. New reference documentation, organized around tasks, with a quickstart and tutorials. Security. Fixes for 13 CVEs, which Spring AI 2.0.3 has as well. Next: Spring AI Agent. A new project for composing and orchestrating agents, built on Spring AI 2.1, starts incubating in early December. See What's next. The following map shows the building blocks of Spring AI 2.1, with the community projects in blue. It comes from the new reference documentation, where the blocks link to their pages. MCP 2026-07-28 The 2026-07-28 revision is a redesign of the Model Context Protocol. It is stateless: there is no initialize handshake and no session, every request carries the p

## Introducing harness adapters: adapt your favorite model to your harness to improve performance and cost

DevFeed: [Introducing harness adapters: adapt your favorite model to your harness to improve performance and cost](<https://devfeed.tech/articles/introducing-harness-adapters-adapt-your-favorite-model-to-your-harness-to-improve-performance-and-cost-146558.md>)

Original publisher: [Read original article](<https://huggingface.co/blog/harness-adapters-at-scale>)

Author: Younes B; ben burtenshaw; Nathan Habib; Lysandre; Amine Dirhoussi; Adithya S K; shaun smith; Sergio Paniego; Joel Niklaus; Lewis Tunstall

Published: 2026-11-20T00:00:00Z

Content type: tutorial

Language: en

Sources: [Hugging Face](<https://devfeed.tech/sources/hugging-face-blog.md>)

Topics: [Agent Harness](<https://devfeed.tech/topics/agent-harness.md>), [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [post-training](<https://devfeed.tech/topics/post-training.md>), [LLM evaluation / benchmarking](<https://devfeed.tech/topics/llm-evaluation-benchmarking.md>), [ai evaluation](<https://devfeed.tech/topics/ai-evaluation.md>), [Post-training optimization](<https://devfeed.tech/topics/post-training-optimization.md>)

Tags: [adapters](<https://devfeed.tech/tags/adapters.md>), [agent-harness](<https://devfeed.tech/tags/agent-harness.md>), [agentic](<https://devfeed.tech/tags/agentic.md>), [agentic-coding](<https://devfeed.tech/tags/agentic-coding.md>), [agents](<https://devfeed.tech/tags/agents.md>), [artificial-intelligence](<https://devfeed.tech/tags/artificial-intelligence.md>), [cost](<https://devfeed.tech/tags/cost.md>), [harness](<https://devfeed.tech/tags/harness.md>), [lora](<https://devfeed.tech/tags/lora.md>), [model](<https://devfeed.tech/tags/model.md>), [open](<https://devfeed.tech/tags/open.md>), [open-source](<https://devfeed.tech/tags/open-source.md>), [openenv](<https://devfeed.tech/tags/openenv.md>), [performance](<https://devfeed.tech/tags/performance.md>), [rl](<https://devfeed.tech/tags/rl.md>), [source](<https://devfeed.tech/tags/source.md>), [trl](<https://devfeed.tech/tags/trl.md>)

### AI overview

The article reports that reinforcement learning inside each target agent harness improved Qwen3.8-27B's coding benchmark results and reduced tool use. Separate LoRA adapters trained for Pi, Hermes Agent, and OpenCode each beat the base model under their own harness on Terminal-Bench 2.1 and hard SWE-bench Pro tasks. On tasks both models solved, specialists used 53-72% fewer tool calls. A merged adapter also beat the base under all three harnesses; with thinking off it averaged 39.5% to 49.0% pass@1, while cutting tool calls by 53-73%. The recipe uses validated Harbor tasks, difficulty filtering so GRPO receives tasks the base model solves inconsistently, and TRL's AsyncGRPOTrainer with the harness running as shipped. Results are not directly comparable across harnesses because their defaults differ. On 300 AutomationBench tasks outside the training domain and harnesses, adapters stayed near the base model, so transfer to that setting was not clear. The authors also note possible reasoning leakage in non-thinking outputs and call for further study.

### Source excerpt

We're on a journey to advance and democratize artificial intelligence through open source and open science.

## Agentic Harness Engineering Beta: Requirement-Centered Evolution of Agent Harnesses

DevFeed: [Agentic Harness Engineering Beta: Requirement-Centered Evolution of Agent Harnesses](<https://devfeed.tech/articles/agentic-harness-engineering-beta-requirement-centered-evolution-of-agent-harnesses-142220.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.24924>)

Author: Jiahang Lin, Jiahao Chen, Yiding Wang, Shichun Liu, Chengjun Pan, Shihan Dou, Honglin Guo, Yuyang Hu, Changhao Jiang, Zhenhua Han, Hang Yan, Qi Zhang, Tao Gui, Xuanjing Huang

Published: 2026-11-19T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Software Engineering](<https://devfeed.tech/sources/arxiv-software-engineering.md>)

Topics: [Agent Harness](<https://devfeed.tech/topics/agent-harness.md>), [Loop Engineering](<https://devfeed.tech/topics/loop-engineering.md>), [agentic-coding](<https://devfeed.tech/topics/agentic-coding.md>), [Spec Driven Development](<https://devfeed.tech/topics/spec-driven-development.md>), [AI Engineering](<https://devfeed.tech/topics/ai-engineering.md>)

Tags: [2](<https://devfeed.tech/tags/2.md>), [3](<https://devfeed.tech/tags/3.md>), [abstract](<https://devfeed.tech/tags/abstract.md>), [agent](<https://devfeed.tech/tags/agent.md>), [agent-harnesses](<https://devfeed.tech/tags/agent-harnesses.md>), [agentic](<https://devfeed.tech/tags/agentic.md>), [all](<https://devfeed.tech/tags/all.md>), [arxiv](<https://devfeed.tech/tags/arxiv.md>), [base](<https://devfeed.tech/tags/base.md>), [based](<https://devfeed.tech/tags/based.md>), [benchmark](<https://devfeed.tech/tags/benchmark.md>), [beta](<https://devfeed.tech/tags/beta.md>), [capabilities](<https://devfeed.tech/tags/capabilities.md>), [change](<https://devfeed.tech/tags/change.md>), [changes](<https://devfeed.tech/tags/changes.md>), [claude](<https://devfeed.tech/tags/claude.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-se](<https://devfeed.tech/tags/cs-se.md>), [engineering](<https://devfeed.tech/tags/engineering.md>), [evolution](<https://devfeed.tech/tags/evolution.md>), [harness](<https://devfeed.tech/tags/harness.md>), [harness-engineering](<https://devfeed.tech/tags/harness-engineering.md>)

### AI overview

AHE-Beta evolves agent harnesses around verified user requirements: simulated users raise issues over multiple turns, a team implements generalizable changes, and changes are merged only after resolving a verified requirement and passing review. On held-out tasks, it raises pass@1 from 21.9% to 33.3% while using 40.8% fewer tokens than Base Harness, and the abstract reports higher pass@1 than Claude Code, Meta-Harness, and AHE. In a blind human A/B test, users preferred it in 31 of 48 valid pairs and rated it higher on all five dimensions. Without further evolution, reported pass@1 gains over Base Harness transferred to GLM-5.3 (15.8%) and DeepSeek-V4.1-Flash (5.3%).

### Source excerpt

arXiv:2611.24924v1 Announce Type: new Abstract: Real-world harness engineering is driven by explicit and implicit user requirements, which guide development and validation. However, existing harness evolution methods often overlook implicit requirements and interaction needs, while using a single evolution agent to modify the harness and benchmark scores to select changes. This process often produces narrow patches and leads to benchmark-specific local optima. We introduce Agentic Harness Engineering Beta (AHE-Beta), a requirement-centered loop that brings real-world engineering practices into harness evolution. The loop consists of three steps: (1) User-driven evaluation lets simulated users raise their requirements as issues while they work with the harness over multiple turns. (2) Team-based modification implements those requirements as generalizable changes rather than task-specific patches. (3) Requirement-driven selection merges a change only when it resolves a verified requirement and passes review. Empirically, on held-out tasks, AHE-Beta raises pass@1 from 21.9% to 33.3% while using 40.8% fewer tokens than Base Harness, and achieves higher pass@1 than Claude Code, Meta-Harness, and AHE. In a blind human A/B test, real users prefer AHE-Beta in 31 of 48 valid pairs and rate it higher on all five dimensions, matching the directional preference of the user simulator. The human A/B results show that AHE-Beta achieves its goal of aligning harness evolution with real-world user requirements. Without further evolution, the gains transfer to GLM-5.3 and DeepSeek-V4.1-Flash with pass@1 improvements of 15.8% and 5.3% over Base Harness. Together, these results show that requirement-centered harness evolution is a more general method for improving agent capabilities.

## GRAFT: Execution-Guided Genetic Recombination for Agent Design

DevFeed: [GRAFT: Execution-Guided Genetic Recombination for Agent Design](<https://devfeed.tech/articles/graft-execution-guided-genetic-recombination-for-agent-design-142666.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.23509>)

Author: Sevda Ebrahimi, Mohammed Saidul Islam, Xiaodan Zhu, Elham Dolatabadi

Published: 2026-11-19T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [Agent Harness](<https://devfeed.tech/topics/agent-harness.md>), [llm-reasoning](<https://devfeed.tech/topics/llm-reasoning.md>)

Tags: [agent](<https://devfeed.tech/tags/agent.md>), [aggregate](<https://devfeed.tech/tags/aggregate.md>), [architectures](<https://devfeed.tech/tags/architectures.md>), [arxiv](<https://devfeed.tech/tags/arxiv.md>), [automated](<https://devfeed.tech/tags/automated.md>), [based](<https://devfeed.tech/tags/based.md>), [bounded](<https://devfeed.tech/tags/bounded.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cl](<https://devfeed.tech/tags/cs-cl.md>), [cs-ne](<https://devfeed.tech/tags/cs-ne.md>), [design](<https://devfeed.tech/tags/design.md>), [evaluation](<https://devfeed.tech/tags/evaluation.md>), [execution](<https://devfeed.tech/tags/execution.md>), [executor](<https://devfeed.tech/tags/executor.md>), [failed](<https://devfeed.tech/tags/failed.md>), [failures](<https://devfeed.tech/tags/failures.md>), [feedback](<https://devfeed.tech/tags/feedback.md>), [fitness](<https://devfeed.tech/tags/fitness.md>), [fixed](<https://devfeed.tech/tags/fixed.md>), [guide](<https://devfeed.tech/tags/guide.md>), [harness](<https://devfeed.tech/tags/harness.md>), [language](<https://devfeed.tech/tags/language.md>), [language-model](<https://devfeed.tech/tags/language-model.md>)

### AI overview

GRAFT guides evolutionary agent design with parent-specific evidence from successful and failed executions, while aggregate fitness still governs selection and survival. Its semantic crossover integrates mutation, and its feedback harness reuses bounded run samples without reference answers or additional task executions. Across six reasoning and programming settings, GRAFT reaches 82.0% mean test accuracy, 9.8 percentage points above the strongest baseline. Controlled ablations report gains from execution evidence with initialization, search rules, repair policies, and evaluation budgets held fixed. The supplied text is an abstract, so it does not provide further implementation details.

### Source excerpt

arXiv:2611.23509v1 Announce Type: new Abstract: Automated agent design searches for executable architectures that improve LLM-based agent performance, with evolutionary methods using language-model crossover to recombine promising mechanisms for task decomposition, model coordination, and verification. This search, however, is typically guided by aggregate fitness alone, which hides the instance-level successes and failures behind each score. We introduce Execution Guided Genetic Recombination for Agent Design (GRAFT), which uses aggregate fitness for selection and survival and parent-specific execution evidence to guide semantic crossover with integrated mutation. Its feedback harness reuses bounded samples of successful and failed runs from fitness evaluation, withholding reference answers and requiring no additional task executions. We evaluate GRAFT across six reasoning and programming settings using open-weight optimizer and executor models. It achieves a mean test accuracy of 82.0%, exceeding the strongest baseline by 9.8% points. Controlled ablations show that adding execution evidence improves test accuracy with initialization, search rules, repair policies, and evaluation budgets held fixed. Our results support the value of parent-specific execution evidence for evolutionary agent design.

## Bad Genius: Counterfactual-Guided Harness Evolution Beyond Task-Specific Shortcuts

DevFeed: [Bad Genius: Counterfactual-Guided Harness Evolution Beyond Task-Specific Shortcuts](<https://devfeed.tech/articles/bad-genius-counterfactual-guided-harness-evolution-beyond-task-specific-shortcuts-142560.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2609.18366>)

Author: Guojun Zhu, Xunheng Huang, Peng Yin, Jiahui Xie, Sanguo Zhang, Doudou Zhou

Published: 2026-11-19T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Agent Harness](<https://devfeed.tech/topics/agent-harness.md>), [benchmark overfitting machine learning](<https://devfeed.tech/topics/benchmark-overfitting-machine-learning.md>), [Multi-step AI processes](<https://devfeed.tech/topics/multi-step-ai-processes.md>)

Tags: [agent](<https://devfeed.tech/tags/agent.md>), [agent-harness](<https://devfeed.tech/tags/agent-harness.md>), [archive](<https://devfeed.tech/tags/archive.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [evaluation](<https://devfeed.tech/tags/evaluation.md>), [evolution](<https://devfeed.tech/tags/evolution.md>), [formatting](<https://devfeed.tech/tags/formatting.md>), [foundation-model](<https://devfeed.tech/tags/foundation-model.md>), [harness](<https://devfeed.tech/tags/harness.md>), [order](<https://devfeed.tech/tags/order.md>), [protocol](<https://devfeed.tech/tags/protocol.md>), [search](<https://devfeed.tech/tags/search.md>), [semantics](<https://devfeed.tech/tags/semantics.md>), [shortcuts](<https://devfeed.tech/tags/shortcuts.md>), [stat-ml](<https://devfeed.tech/tags/stat-ml.md>), [swe-bench](<https://devfeed.tech/tags/swe-bench.md>), [swe-bench-pro](<https://devfeed.tech/tags/swe-bench-pro.md>)

### AI overview

CHASE tests whether agent-harness improvements survive protocol changes that preserve task semantics. A Proposer edits a harness around a fixed foundation model; a Challenger searches for changes that reduce its gain over the initial harness. Executable checks verify task preservation, and confirmed held-out gain losses are archived to constrain later updates. The authors state conditions under which a finite archive bounds losses on an ideal benchmark without shortcuts. On Syn-Ledger with known shortcuts removed, CHASE reached 38.30% accuracy versus 18.27% for HarnessEvolve. On 21 SWE-Bench Pro V2 HARD-51 certification tasks, changing issue formatting and tool-output field order cut HarnessEvolve's gain from 8.57 to 3.81 percentage points; CHASE retained 10.48 of its 11.43-point gain.

### Source excerpt

arXiv:2609.18366v4 Announce Type: replace Abstract: An optimized agent harness can improve on held-out tasks by relying on formatting conventions that those tasks share with the rest of the benchmark. We introduce Counterfactual Harness Search and Evolution (CHASE) to test whether harness improvements survive changes to the evaluation protocol. A Proposer edits the harness around a fixed foundation model. A Challenger then searches for a protocol change that preserves the tasks but reduces the updated harness's gain over the initial harness. Executable checks test whether the transformation preserves task semantics. If the gain loss is confirmed on held-out tasks, CHASE adds the transformation to an archive that constrains subsequent harness updates. We give conditions under which this finite archive bounds the gain loss on an ideal benchmark with shortcuts removed. We evaluate CHASE on Syn-Ledger, OfficeQA, FinanceBench, and SWE-Bench Pro V2. On Syn-Ledger, removing the known shortcuts yields 38.30% accuracy for CHASE and 18.27% for HarnessEvolve. On the 21 certification tasks from SWE-Bench Pro V2 HARD-51, changing the issue formatting and tool-output field order reduces HarnessEvolve's gain over the initial harness from 8.57 to 3.81 percentage points. The final CHASE harness retains 10.48 points of its 11.43-point gain under the same change.

## Arbor: How Search Policy Shapes Agent Behavior in System Optimization

DevFeed: [Arbor: How Search Policy Shapes Agent Behavior in System Optimization](<https://devfeed.tech/articles/arbor-how-search-policy-shapes-agent-behavior-in-system-optimization-142530.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2606.12563>)

Author: Neha Prakriya, Chaojun Hou, Zheng Gong, Huasha Zhao, Yanyuan Qin, Chris Cai, Xi Zhao, Zhenyu Gu, Emad Barsoum

Published: 2026-11-19T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [LLM optimization](<https://devfeed.tech/topics/llm-optimization.md>), [AI Agent](<https://devfeed.tech/topics/ai-agent.md>), [Multi Agent Systems](<https://devfeed.tech/topics/multi-agent-systems.md>), [Agent Harness](<https://devfeed.tech/topics/agent-harness.md>), [Post-training optimization](<https://devfeed.tech/topics/post-training-optimization.md>)

Tags: [agent](<https://devfeed.tech/tags/agent.md>), [behavior](<https://devfeed.tech/tags/behavior.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [optimization](<https://devfeed.tech/tags/optimization.md>), [policy](<https://devfeed.tech/tags/policy.md>), [search](<https://devfeed.tech/tags/search.md>), [system](<https://devfeed.tech/tags/system.md>)

### AI overview

Arbor organizes accelerator optimization as an explicit tree that agents can traverse, revisit, and deepen. Its shared harness provides containerized isolation, automatic rollback, and end-to-end validation; search options include scored bandit traversal, parallel swarms, and depth-first sweeps. The abstract reports gains of 40%-193% across six production inference models on two GPU generations, 42.8% for LLM training, and 41% for recommendation training. In controlled comparisons using the same agent, tree, and tools, bandit search gained 30.1% through targeted framework fixes, sweeps 26.4% through deep kernel rewrites, and swarms 33.4% by combining independent changes. A loop that chose its own targets gained 14% and wrote four times more code per percentage point of improvement than Arbor's structured policies. The paper also introduces an evaluation framework covering efficiency, grounding, robustness, and depth.

### Source excerpt

arXiv:2606.12563v2 Announce Type: replace Abstract: Achieving peak performance on modern accelerators requires coordinated optimization across kernels, operator fusion, KV-cache management, quantization, scheduling, and communication overlap, a process that demands weeks of expert effort per model and hardware platform. We present Arbor, a system that gives autonomous agents an explicit optimization tree to reason over, backtrack through, and deepen with experience, with pluggable search policies, scored bandit traversal, parallel agent swarms, and depth-first sweep, sharing a common harness of containerized isolation, automatic rollback, and end-to-end validation. Arbor generalizes across domains: +40% to +193% on six production inference models across two GPU generations, +42.8% on LLM training, and +41% on recommendation training, where agents autonomously wrote kernel-level and framework-level optimizations spanning thousands of lines of code. In controlled ablations where every search policy receives the same agent, tree, and tools, we find that search structure shapes both the magnitude and the nature of the optimizations. The bandit achieves +30.1% through targeted framework fixes, the sweep +26.4% through deep kernel rewrites, and the swarm +33.4% by compounding independent changes. An agent-driven loop that receives the same harness, tools, and information, but selects its own targets reaches +14%, writing 4x more code per percentage point of improvement than Arbor's structured search policies. We additionally introduce an evaluation framework for autonomous loop-based agents spanning efficiency, grounding, robustness, and depth, designed to characterize agent search behavior across long-running optimization campaigns.

## Hearsay: Who Can Audit the Record a Deployed Agent Harness Writes?

DevFeed: [Hearsay: Who Can Audit the Record a Deployed Agent Harness Writes?](<https://devfeed.tech/articles/hearsay-who-can-audit-the-record-a-deployed-agent-harness-writes-142117.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2609.32495>)

Author: Jiahong Dai, Zhuochen Yang, Pengyang Shao, Kelvin Ng, Zhongyi Liu, Chengquan Ju, Yuting He, Bo Hu

Published: 2026-11-19T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Cryptography and Security](<https://devfeed.tech/sources/arxiv-cryptography-and-security.md>), [arXiv: Software Engineering](<https://devfeed.tech/sources/arxiv-software-engineering.md>)

Topics: [audit](<https://devfeed.tech/topics/audit.md>), [Agent Harness](<https://devfeed.tech/topics/agent-harness.md>), [Audit-then-score](<https://devfeed.tech/topics/audit-then-score.md>)

Tags: [agent](<https://devfeed.tech/tags/agent.md>), [agent-harness](<https://devfeed.tech/tags/agent-harness.md>), [audit](<https://devfeed.tech/tags/audit.md>), [benchmark](<https://devfeed.tech/tags/benchmark.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-ce](<https://devfeed.tech/tags/cs-ce.md>), [cs-cr](<https://devfeed.tech/tags/cs-cr.md>), [cs-se](<https://devfeed.tech/tags/cs-se.md>), [frameworks](<https://devfeed.tech/tags/frameworks.md>), [harness](<https://devfeed.tech/tags/harness.md>), [ledger](<https://devfeed.tech/tags/ledger.md>), [model](<https://devfeed.tech/tags/model.md>), [records](<https://devfeed.tech/tags/records.md>), [sandbox](<https://devfeed.tech/tags/sandbox.md>), [tasks](<https://devfeed.tech/tags/tasks.md>), [writer](<https://devfeed.tech/tags/writer.md>)

### AI overview

The paper argues that an agent run record cannot independently verify itself when the harness is its sole writer. In a benchmark spanning five harnesses, fourteen tasks, and examiners from three vendors plus a human panel, readers often identified faults but could not check the record; model examiners overstated corroboration, and even the strongest detected only half of inserted alterations. The authors report that all sixteen surveyed frameworks had the harness as the only writer. Their proposed sandbox adds a ledger of exchanges between harness and model and compares it with the record in both directions to detect omissions and additions. On unseen records, it caught 64 of 66 alterations without flagging 84 clean records, and caught 109 of 115 held-out records across five formats.

### Source excerpt

arXiv:2609.32495v2 Announce Type: replace-cross Abstract: The record of an agent's run is written by the harness that drives the agent. When the run is later disputed, the auditor was not there and has only this record. We ask whether someone who was not there can check the record, and what decides the answer. It depends on who wrote what a check can rely on: if the harness wrote all of it, the record is only the harness's own word, and we call it hearsay. We build Hearsay, a benchmark that examines the record and not the task: five harnesses run fourteen tasks, model examiners from three vendors and a human panel read the records and answer questions about the runs, and every passage they cite is labelled mechanically by who wrote it. We also audit what is deployed: in all sixteen frameworks, the harness is the only writer. With one writer, readers can name the fault from the record, in up to nine runs in ten, but they cannot check the record. The model examiners claim corroboration for 18 to 55 points more of their evidence than the labels support, a gap we then predicted for runs driven by another vendor's model and found again. And the best of them notices only half of the alterations we make to a record, while the two that name the fault most often cannot tell what a harness left out. With a second writer, the record can be checked. Our sandbox keeps a ledger of what passes between harness and model, and reading the ledger against the record in both directions shows what the record leaves out and what it adds. We test this as an auditor would use it, on one unseen record at a time. On runs driven by a third vendor's model, with every rule committed before the records existed, it reports 64 of 66 alterations and flags none of 84 clean records. On held-out records in five formats it reports 109 of 115. What makes a record evidence is a second writer, not a better reader.

## ETA: Exit on Tool Arrival in Speculative Agent Loops

DevFeed: [ETA: Exit on Tool Arrival in Speculative Agent Loops](<https://devfeed.tech/articles/eta-exit-on-tool-arrival-in-speculative-agent-loops-142124.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.23239>)

Author: Saptarshi Mitra, Xuan Zhu, Rohith Kukkala, Sitao Huang, Karthik Vijayan

Published: 2026-11-19T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Software Engineering](<https://devfeed.tech/sources/arxiv-software-engineering.md>)

Topics: [Agent Harness](<https://devfeed.tech/topics/agent-harness.md>), [llm-reasoning](<https://devfeed.tech/topics/llm-reasoning.md>), [Chain-of-thought](<https://devfeed.tech/topics/chain-of-thought.md>)

Tags: [abstract](<https://devfeed.tech/tags/abstract.md>), [agent](<https://devfeed.tech/tags/agent.md>), [agent-loops](<https://devfeed.tech/tags/agent-loops.md>), [call](<https://devfeed.tech/tags/call.md>), [cost](<https://devfeed.tech/tags/cost.md>), [cs-se](<https://devfeed.tech/tags/cs-se.md>), [decision](<https://devfeed.tech/tags/decision.md>), [decode](<https://devfeed.tech/tags/decode.md>), [dispatch](<https://devfeed.tech/tags/dispatch.md>), [fewer](<https://devfeed.tech/tags/fewer.md>), [fork](<https://devfeed.tech/tags/fork.md>), [free](<https://devfeed.tech/tags/free.md>), [latency](<https://devfeed.tech/tags/latency.md>), [overlap](<https://devfeed.tech/tags/overlap.md>), [prediction](<https://devfeed.tech/tags/prediction.md>), [quality](<https://devfeed.tech/tags/quality.md>), [reasoning](<https://devfeed.tech/tags/reasoning.md>), [sensor](<https://devfeed.tech/tags/sensor.md>), [serial](<https://devfeed.tech/tags/serial.md>), [stable](<https://devfeed.tech/tags/stable.md>), [tool](<https://devfeed.tech/tags/tool.md>)

### AI overview

ETA speeds up tool-using agent turns by treating a speculative tool call as a signal of the model's decision state. It ends the main reasoning decode only when the predicted call is stable, matches the dispatched call, a free probe indicates a call will be emitted, and the tool result has arrived; the abstract says this bounds per-turn regret by the remaining decode cost. Reported speedups are 1.33x on tau2 airline/retail, 1.26x on tau2 telecom, and 1.13x on BFCL v4 multi-turn. On tau2, ETA decodes 24% fewer tokens at nearly unchanged per-token rate, without reported loss of action quality.

### Source excerpt

arXiv:2611.23239v1 Announce Type: new Abstract: Tool-using language agents in explicit thinking modes generate long chains of thought before acting, making each turn inherently serial: later tokens depend on the tool observation. Recent speculative tool dispatch methods overlap tool execution with reasoning, but offers limited benefit when reasoning dominates latency. We present ETA, that treats the speculative fork not as an actuator but as a sensor of the model's decision state, using it to terminate the reasoning of the main decode. It truncates reasoning only when the prediction is stable, agrees with the dispatched call, passes a free probe measuring whether a call will be emitted at all, and the tool result has arrived: a deadline that bounds per-turn regret by the remaining decode cost. ETA achieves speedups of 1.33x on tau2 airline/retail, 1.26x on tau2 telecom, and 1.13x on BFCL v4 multi-turn. On tau2, it decodes 24% fewer tokens at a nearly unchanged per-token rate without dropping action quality.

## ProteinGeoBench: Frontier Structural-Biology Agents Are Fluent at Lookup but Lost in Geometry

DevFeed: [ProteinGeoBench: Frontier Structural-Biology Agents Are Fluent at Lookup but Lost in Geometry](<https://devfeed.tech/articles/proteingeobench-frontier-structural-biology-agents-are-fluent-at-lookup-but-lost-in-geometry-142580.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2610.26829>)

Author: Matt Turk, Brandon L. Neel, Diana M. Lopez, Zachary D. Smith, Grace N. Johnecheck, James B. Golden

Published: 2026-11-19T05:00:00Z

Content type: news

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Frontier AI](<https://devfeed.tech/topics/frontier-ai.md>), [Agent Harness](<https://devfeed.tech/topics/agent-harness.md>), [Claude](<https://devfeed.tech/topics/claude.md>)

Tags: [agent-harness](<https://devfeed.tech/tags/agent-harness.md>), [agents](<https://devfeed.tech/tags/agents.md>), [atom](<https://devfeed.tech/tags/atom.md>), [biology](<https://devfeed.tech/tags/biology.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [frontier](<https://devfeed.tech/tags/frontier.md>), [lookup](<https://devfeed.tech/tags/lookup.md>), [network](<https://devfeed.tech/tags/network.md>), [q-bio-bm](<https://devfeed.tech/tags/q-bio-bm.md>), [sequences](<https://devfeed.tech/tags/sequences.md>), [structure](<https://devfeed.tech/tags/structure.md>)

### AI overview

ProteinGeoBench evaluates agents on 131 protein-structure tasks spanning fact lookup, sequence checks, coordinate-based geometry, and structural reasoning. For three of four frontier models, lookup scores exceeded calculation scores; Claude Opus 5 scored 0.907 on lookup versus 0.716 on calculation, while Gemini 3.1 Pro was the exception. Geometry tasks were overrepresented among failures: half of the 12 tasks all four models failed were calculations, and eight were unsolved by any of 26 models. The top three models were within 0.031 points, and rankings varied across bootstrap resamples. Scores also depended on the harness and network access: on a 15-task subset, blocking eight structural databases reduced scores, removing network access reduced them further, and provider command-line agents shifted scores by up to 0.09 in either direction. The authors recommend reporting task-type scores, runs per task, harness, and network access.

### Source excerpt

arXiv:2610.26829v2 Announce Type: replace Abstract: ProteinGeoBench is a set of 131 tasks that give an agent protein structure files and ask it to look up recorded facts, check sequences, calculate geometry from atom coordinates or reason about structures. We ran four frontier models on every task several times (4,249 runs) and 26 models on a subset. A single overall score hides two different skills. Three of the four frontier models are best at looking up facts and worst at calculating from coordinates: Claude Opus 5 scores 0.907 on lookup and 0.716 on calculation. Gemini 3.1 Pro is the exception. Geometry is overrepresented among the hardest tasks. Half of the twelve tasks that all four models fail are calculations, although calculations are a fifth of the suite, and eight of the twelve were solved by none of the 26 models. The three leading models are within 0.031 of one another, too close to order, and with one run per task a different model ranks first in 24% of bootstrap resamples. Reports should therefore give scores by task type and the number of runs per task. They should also state the agent harness, the software that runs the model, and its network access, because both change scores. On a 15-task subset, blocking eight structural databases lowered scores, and removing network access lowered them more. Replacing our harness with each provider's own command-line agent moved scores by up to 0.09 in either direction. Across the 26 models, those that scored higher on the other tasks also scored higher on calculation.

## Who Is Behind the Harness? Verifying Model Identity in Coding Agents through Behavioral Fingerprinting

DevFeed: [Who Is Behind the Harness? Verifying Model Identity in Coding Agents through Behavioral Fingerprinting](<https://devfeed.tech/articles/who-is-behind-the-harness-verifying-model-identity-in-coding-agents-through-behavioral-fingerprinting-142116.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2609.28559>)

Author: Chuyi Wang, Xiaohui Xie, Tongze Wang, Fangchen Luo, Yong Cui

Published: 2026-11-19T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Cryptography and Security](<https://devfeed.tech/sources/arxiv-cryptography-and-security.md>), [arXiv: Software Engineering](<https://devfeed.tech/sources/arxiv-software-engineering.md>)

Topics: [LLM security](<https://devfeed.tech/topics/llm-security.md>), [Agent Harness](<https://devfeed.tech/topics/agent-harness.md>), [AI-assisted coding](<https://devfeed.tech/topics/ai-assisted-coding.md>), [test](<https://devfeed.tech/topics/test.md>)

Tags: [actions](<https://devfeed.tech/tags/actions.md>), [agents](<https://devfeed.tech/tags/agents.md>), [black-box](<https://devfeed.tech/tags/black-box.md>), [coding-agents](<https://devfeed.tech/tags/coding-agents.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cr](<https://devfeed.tech/tags/cs-cr.md>), [cs-se](<https://devfeed.tech/tags/cs-se.md>), [execution](<https://devfeed.tech/tags/execution.md>), [failure-recovery](<https://devfeed.tech/tags/failure-recovery.md>), [harness](<https://devfeed.tech/tags/harness.md>), [identity](<https://devfeed.tech/tags/identity.md>), [lidar](<https://devfeed.tech/tags/lidar.md>), [llm](<https://devfeed.tech/tags/llm.md>), [llms](<https://devfeed.tech/tags/llms.md>), [model](<https://devfeed.tech/tags/model.md>), [model-weights](<https://devfeed.tech/tags/model-weights.md>), [models](<https://devfeed.tech/tags/models.md>), [opencode](<https://devfeed.tech/tags/opencode.md>), [openhands](<https://devfeed.tech/tags/openhands.md>), [runtime](<https://devfeed.tech/tags/runtime.md>), [security](<https://devfeed.tech/tags/security.md>), [top](<https://devfeed.tech/tags/top.md>), [verification](<https://devfeed.tech/tags/verification.md>)

### AI overview

LIDAR identifies coding-agent models by actively probing their runtime decisions rather than relying on final text or token distributions. Three probe pairs test post-edit verification, recovery from transient failures, and resolution of specification-test conflicts. It represents execution trajectories with instance-level and distribution-level features, then compares them with clean references using a lightweight probabilistic identifier. The method needs no access to weights, logits, or provider internals, and uses separate references for OpenCode, OpenHands, and mini-swe-agent. Across 24 models from eight families, closed-set Top-1 accuracy ranged from 91.39% to 98.89%, exceeding three baselines in each harness. Including unregistered models from the same families reduced balanced accuracy to 79.58%-80.42%.

### Source excerpt

arXiv:2609.28559v2 Announce Type: replace-cross Abstract: LLMs increasingly operate through coding-agent harnesses that inspect repositories, invoke tools, and modify files. Substituting the model behind such an agent can therefore change security-relevant decisions, including whether it verifies changes or recovers safely from failures. Existing LLM fingerprints largely infer identity from direct text or token distributions. In coding agents, these signals are mediated by system instructions, controller logic, tools, and execution feedback, limiting their transfer. We present LIDAR (LLM Identification from Decisions and Actions at Runtime), an active black-box fingerprinting method for coding-agent execution. Three coding probe pairs expose post-edit verification, transient-failure recovery, and specification--test conflict resolution under controlled changes. LIDAR represents the resulting trajectories with complementary instance-level and distribution-level features and compares them with clean references using a lightweight probabilistic identifier. It requires no access to model weights, logits, or provider internals. We evaluate 24 models from eight families on OpenCode, OpenHands, and mini-swe-agent, fitting separate references for each harness. Closed-set Top-1 accuracy ranges from 91.39\% to 98.89\% across the three harnesses, exceeding all three baselines in each setting. When the audit also includes unregistered models from the same families, balanced accuracy ranges from 79.58\% to 80.42\%. These results show that agent execution behavior provides model-identity evidence beyond final outputs.

## Securing and Auditing Long-Horizon LLM Agents: State-Transfer Auditing and Controls in the Agent Harness

DevFeed: [Securing and Auditing Long-Horizon LLM Agents: State-Transfer Auditing and Controls in the Agent Harness](<https://devfeed.tech/articles/securing-and-auditing-long-horizon-llm-agents-state-transfer-auditing-and-controls-in-the-agent-harness-143119.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.23346>)

Author: Deep Shah, Nehal Kathrotia

Published: 2026-11-19T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Cryptography and Security](<https://devfeed.tech/sources/arxiv-cryptography-and-security.md>)

Topics: [Agent Harness](<https://devfeed.tech/topics/agent-harness.md>), [LLM security](<https://devfeed.tech/topics/llm-security.md>), [ai security](<https://devfeed.tech/topics/ai-security.md>)

Tags: [actions](<https://devfeed.tech/tags/actions.md>), [agent](<https://devfeed.tech/tags/agent.md>), [agent-harness](<https://devfeed.tech/tags/agent-harness.md>), [agents](<https://devfeed.tech/tags/agents.md>), [analyze](<https://devfeed.tech/tags/analyze.md>), [architecture](<https://devfeed.tech/tags/architecture.md>), [attribution](<https://devfeed.tech/tags/attribution.md>), [audit](<https://devfeed.tech/tags/audit.md>), [auditing](<https://devfeed.tech/tags/auditing.md>), [authorization](<https://devfeed.tech/tags/authorization.md>), [boundary](<https://devfeed.tech/tags/boundary.md>), [browsers](<https://devfeed.tech/tags/browsers.md>), [cache](<https://devfeed.tech/tags/cache.md>), [controls](<https://devfeed.tech/tags/controls.md>), [cs-cr](<https://devfeed.tech/tags/cs-cr.md>), [harness](<https://devfeed.tech/tags/harness.md>), [horizon](<https://devfeed.tech/tags/horizon.md>), [llm](<https://devfeed.tech/tags/llm.md>), [llm-agents](<https://devfeed.tech/tags/llm-agents.md>), [state](<https://devfeed.tech/tags/state.md>), [state-transfer](<https://devfeed.tech/tags/state-transfer.md>)

### AI overview

The paper argues that important failures in long-horizon LLM agents occur when state moves between working context, session history, persistent memory, and external systems--transfers conventional prompt-and-tool logs may miss. It proposes hash-chained audit records at each transfer boundary, combining deterministic checks with model-based semantic checks and linking internal context, memory, and inference-server cache state to external actions. The records are also intended to support controls such as least-privilege capabilities, replay-resistant authorization outside the prompt, information flow control, obligation monitoring, and transactional recovery. A reference middleware evaluated on 240 multi-step fault-injection trajectories reportedly localized causes and detected violations missed by lifecycle and tool-only logs, with microsecond deterministic overhead. This summary is based on the abstract.

### Source excerpt

arXiv:2611.23346v1 Announce Type: new Abstract: Long-horizon LLM agents across codebases, web browsers, and enterprise infrastructure rely on an execution harness to move state between the working context, session history, persistent memory, and external environment. While conventional execution traces log model prompts and tool calls, critical security and reliability failures occur during state transfers: a transfer can admit unchecked content (indirect prompt injections, unverified memory writes), rewrite state lossily (context compaction dropping user constraints, memory consolidation stripping source authority), or restore state incompletely (workspace rollbacks leaving aborted KV state active on the inference server). Because current traces omit what crosses these boundaries, failure attribution and runtime monitors fail on multi-step trajectories. We present a state-transfer audit architecture for the agent harness. At each transfer boundary, we define an audit point specifying the minimal hash-chained record the harness emits, separating low-overhead deterministic checks from model-based semantic checks and binding internal context, memory, and inference-server cache state to external actions. We show that the same transfer records required for causal failure attribution supply the durable state needed by runtime security controls, including least-privilege capabilities, replay-resistant authorization outside the prompt, information flow control, obligation monitors, and transactional recovery across multi-agent and infrastructure pipelines. Finally, we analyze multimodal GUI adaptations and implement and evaluate a reference harness middleware across 240 multi-step fault-injection trajectories, showing that transfer records localize root causes and catch cross-transfer violations missed by lifecycle and tool-only logs at microsecond deterministic overhead.

## What Does a Harness Buy? Tokens, Mostly

DevFeed: [What Does a Harness Buy? Tokens, Mostly](<https://devfeed.tech/articles/what-does-a-harness-buy-tokens-mostly-142118.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2610.04433>)

Author: Yangze Liu, Zhongyi Han

Published: 2026-11-19T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>), [arXiv: Software Engineering](<https://devfeed.tech/sources/arxiv-software-engineering.md>)

Topics: [Agent Harness](<https://devfeed.tech/topics/agent-harness.md>), [AI Context Management](<https://devfeed.tech/topics/ai-context-management.md>), [AI Chat](<https://devfeed.tech/topics/ai-chat.md>), [Claude Code](<https://devfeed.tech/topics/claude-code.md>)

Tags: [agent](<https://devfeed.tech/tags/agent.md>), [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [code](<https://devfeed.tech/tags/code.md>), [coding](<https://devfeed.tech/tags/coding.md>), [context](<https://devfeed.tech/tags/context.md>), [context-management](<https://devfeed.tech/tags/context-management.md>), [cost](<https://devfeed.tech/tags/cost.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [cs-se](<https://devfeed.tech/tags/cs-se.md>), [harness](<https://devfeed.tech/tags/harness.md>), [language-model](<https://devfeed.tech/tags/language-model.md>), [leaderboards](<https://devfeed.tech/tags/leaderboards.md>), [model](<https://devfeed.tech/tags/model.md>), [models](<https://devfeed.tech/tags/models.md>), [opencode](<https://devfeed.tech/tags/opencode.md>), [pass](<https://devfeed.tech/tags/pass.md>), [points](<https://devfeed.tech/tags/points.md>), [prompt](<https://devfeed.tech/tags/prompt.md>), [swe-bench](<https://devfeed.tech/tags/swe-bench.md>), [system](<https://devfeed.tech/tags/system.md>), [tokens](<https://devfeed.tech/tags/tokens.md>), [tools](<https://devfeed.tech/tags/tools.md>)

### AI overview

Across seven models tested with Claude Code, mini-SWE-agent, and OpenCode, no harness leads consistently. On SWE-bench Verified and two other benchmarks, bounded differences between harnesses are 2-5 points, and swapping harnesses changes about as many task outcomes as rerunning the same setup. Some model-harness pairings show larger, model-dependent effects: OpenCode trails by 8-9 points on one model and leads by 14 on another. Cost differences also vary by model, ranging from under 10% to 3-4 times. The authors conclude that harness comparisons should be reported alongside reruns; the abstract does not establish a universally better harness.

### Source excerpt

arXiv:2610.04433v2 Announce Type: replace-cross Abstract: A coding agent is a language model inside a harness: the system prompt, the tools, and the context management that let a chat model work in a repository. Most of the engineering on these agents goes into the harness, and vendors and leaderboards credit it with score gains. Earlier studies have fixed the model and swapped the harness, but have not measured whether the swap changes the pass rate more than running the same harness again. We run seven models through three production harnesses, Claude Code, mini-SWE-agent, and OpenCode, on SWE-bench Verified and two benchmarks of other kinds, and rerun them unchanged. No harness leads consistently across models. Where the data can bound the difference between two harnesses, the bound is 2 to 5 points, and a harness swap changes about as many task outcomes as a rerun. The exceptions are harness effects in particular pairings of model and harness, whose direction changes with the model: OpenCode trails by 8 to 9 points on one model and leads by 14 on another. Cost differences between harnesses also depend on the model, from under 10% to 3 to 4 times. Which harness is better therefore has no answer apart from the model, and a harness difference should be reported next to a rerun.

## Idempotency Is Not Recovery: A Fault-Injection Study of Language-Model Agent Tool Actions

DevFeed: [Idempotency Is Not Recovery: A Fault-Injection Study of Language-Model Agent Tool Actions](<https://devfeed.tech/articles/idempotency-is-not-recovery-a-fault-injection-study-of-language-model-agent-tool-actions-142183.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.23353>)

Author: Song Xu

Published: 2026-11-19T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Software Engineering](<https://devfeed.tech/sources/arxiv-software-engineering.md>)

Topics: [Agent Harness](<https://devfeed.tech/topics/agent-harness.md>), [LLM security](<https://devfeed.tech/topics/llm-security.md>), [Benchmark](<https://devfeed.tech/topics/benchmark.md>)

Tags: [ablation](<https://devfeed.tech/tags/ablation.md>), [about](<https://devfeed.tech/tags/about.md>), [abstract](<https://devfeed.tech/tags/abstract.md>), [actions](<https://devfeed.tech/tags/actions.md>), [after](<https://devfeed.tech/tags/after.md>), [agent](<https://devfeed.tech/tags/agent.md>), [agent-runtime](<https://devfeed.tech/tags/agent-runtime.md>), [arxiv](<https://devfeed.tech/tags/arxiv.md>), [authorization](<https://devfeed.tech/tags/authorization.md>), [benchmark](<https://devfeed.tech/tags/benchmark.md>), [call](<https://devfeed.tech/tags/call.md>), [cases](<https://devfeed.tech/tags/cases.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-se](<https://devfeed.tech/tags/cs-se.md>), [fault-injection](<https://devfeed.tech/tags/fault-injection.md>), [idempotency](<https://devfeed.tech/tags/idempotency.md>), [language](<https://devfeed.tech/tags/language.md>), [language-model](<https://devfeed.tech/tags/language-model.md>), [model](<https://devfeed.tech/tags/model.md>), [recovery](<https://devfeed.tech/tags/recovery.md>), [study](<https://devfeed.tech/tags/study.md>), [tool](<https://devfeed.tech/tags/tool.md>)

### AI overview

In the tested agent runtime, suppressing duplicate tool effects did not by itself guarantee recovery or a clear outcome. Across Windows and Linux trials, the full configuration combined authorization, an action ledger, target-side idempotency, reconciliation, and evidence sealing. It recorded no repeated effects in 200 crash-and-replay pairs, versus 117 for each simpler baseline. A ledger-related ablation produced 170 orphaned effects in 170 trials and fewer recovery successes; disabling reconciliation left all 100 tested post-effect crash outcomes ambiguous despite no observed duplicates. The full runtime ran at about one-seventh the throughput of fail-open, and evidence-write faults left 75 of 100 trials per platform unresolved. A separate supplement using the same inputs found a similar pattern under response loss or delay. Results are limited to the stated benchmark.

### Source excerpt

arXiv:2611.23353v1 Announce Type: new Abstract: When a language-model agent's tool call is interrupted, the action may already have changed an external system without confirmation. Retrying risks duplication; assuming success may conceal an action that never occurred. We study these failure modes in a controlled agent runtime combining authorization, an action ledger, target-side idempotency, reconciliation, and evidence sealing. We inject ten single-host fault classes and compare four configurations and three ablations using a target-side oracle separate from the runtime ledger. On each of Windows and Linux, 7,513 trials used 96 captured model responses from 24 constructed cases. In 200 known original crash and replay pairs, the full configuration recorded 0/200 repeated effects versus 117/200 for each simpler baseline. A ledger-related ablation had 170/170 orphaned effects versus 0/170 and 200/300 recovery successes versus 300/300 for the full configuration. Disabling reconciliation left 100/100 known post-effect crash outcomes ambiguous despite no observed duplicates. A separate supplement designed after the initial results and reusing the same inputs found a similar pattern under response loss or delay. The full runtime had about one-seventh of fail-open throughput, while injected evidence-write faults left 75/100 trials per platform unresolved. These findings distinguish duplicate suppression from recovery and identify their measured reliability and performance tradeoffs within the tested benchmark.

## ElephantAgent: Contextual State Continuity in Agentic Systems

DevFeed: [ElephantAgent: Contextual State Continuity in Agentic Systems](<https://devfeed.tech/articles/elephantagent-contextual-state-continuity-in-agentic-systems-142538.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2607.01919>)

Author: Jiankai Jin, Xiangzheng Zhang, Zhao Liu, Wenzhuo Xu, Dongdong Yang, Deyue Zhang, Quanchen Zou

Published: 2026-11-19T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Cryptography and Security](<https://devfeed.tech/sources/arxiv-cryptography-and-security.md>)

Topics: [Agent Harness](<https://devfeed.tech/topics/agent-harness.md>), [ai security](<https://devfeed.tech/topics/ai-security.md>), [LLM security](<https://devfeed.tech/topics/llm-security.md>), [AI Context Management](<https://devfeed.tech/topics/ai-context-management.md>), [Agentic AI](<https://devfeed.tech/topics/what-is-agentic-ai.md>), [DeepSeek Harness](<https://devfeed.tech/topics/dsh.md>), [agentic-coding](<https://devfeed.tech/topics/agentic-coding.md>)

Tags: [agentic](<https://devfeed.tech/tags/agentic.md>), [agentic-systems](<https://devfeed.tech/tags/agentic-systems.md>), [arxiv](<https://devfeed.tech/tags/arxiv.md>), [attack](<https://devfeed.tech/tags/attack.md>), [context](<https://devfeed.tech/tags/context.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cr](<https://devfeed.tech/tags/cs-cr.md>), [deepseek-harness](<https://devfeed.tech/tags/deepseek-harness.md>), [digest](<https://devfeed.tech/tags/digest.md>), [execution](<https://devfeed.tech/tags/execution.md>), [hardware](<https://devfeed.tech/tags/hardware.md>), [memory](<https://devfeed.tech/tags/memory.md>), [out-of-band](<https://devfeed.tech/tags/out-of-band.md>), [overhead](<https://devfeed.tech/tags/overhead.md>), [planning](<https://devfeed.tech/tags/planning.md>), [prompt](<https://devfeed.tech/tags/prompt.md>), [prompt-injection](<https://devfeed.tech/tags/prompt-injection.md>), [protocol](<https://devfeed.tech/tags/protocol.md>), [rollback](<https://devfeed.tech/tags/rollback.md>), [security](<https://devfeed.tech/tags/security.md>), [state](<https://devfeed.tech/tags/state.md>), [system](<https://devfeed.tech/tags/system.md>)

### AI overview

ElephantAgent aims to prevent out-of-band tool and memory poisoning by checking whether a bounded, security-critical portion of an agent's context matches its latest authorized state. Before each planning and execution cycle, it recomputes a digest and verifies it against a linearizable ledger of state transitions maintained over replicated trusted hardware; historical traceability makes transitions attributable and recoverable. In tests, ElephantDSH, an integration with DeepSeek Harness, withstood the reported out-of-band attacks and recovered from in-band prompt injection in all trials. Of five evaluated defenses that check only incoming context, all leaked targeted content under rollback and three did so under rug-pull attacks. Normal coding tasks incurred 1.02-1.07x relative overhead over DSH, with model latency variation removed.

### Source excerpt

arXiv:2607.01919v2 Announce Type: replace Abstract: Agentic systems enhance their capabilities by invoking external tools and maintaining persistent memory. However, recent tool and memory poisoning attacks can covertly bias agent behavior. We observe that such attacks mounted out of band share a single enabler: each succeeds only by presenting the agent with a contextual state (e.g., tool state and memory) that is not the latest faithfully evolved one. This observation suggests that a continuity check over the contextual state covers this family of attacks. We present ElephantAgent, a protocol that enforces Contextual State Continuity over the bounded, security-critical subset of the agent's context. Before each planning-and-execution cycle, ElephantAgent recomputes the digest of the local contextual state and verifies it against the latest authorized digest in a linearizable ledger of state transitions, maintained over replicated trusted hardware. The protocol also provides Historical Traceability, a secondary guarantee that keeps state transitions attributable and recoverable. We integrate ElephantAgent into DeepSeek Harness (DSH) and implement ElephantDSH, an agentic system protected by contextual state verification. ElephantDSH withstood all out-of-band attack attempts, from rug pulls and rollbacks to attacks on its own mechanism, and recovered from an in-band prompt injection in all trials. By comparison, existing defenses typically check only what enters the context: of the five we evaluated, all leaked the targeted content under the rollback, and three also under the rug pull. ElephantDSH completed all normal coding tasks with a relative overhead of 1.02-1.07$\times$ over the original DSH with model latency variation removed.

## RefinementGuard: Counterexample-Directed Semantic Qualification for Hidden Agent-Tool Execution

DevFeed: [RefinementGuard: Counterexample-Directed Semantic Qualification for Hidden Agent-Tool Execution](<https://devfeed.tech/articles/refinementguard-counterexample-directed-semantic-qualification-for-hidden-agent-tool-execution-142188.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.23611>)

Author: Adhyayan Veer Singh, Fanjiang Ye, Yuke Wang

Published: 2026-11-19T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Software Engineering](<https://devfeed.tech/sources/arxiv-software-engineering.md>)

Topics: [Agent Harness](<https://devfeed.tech/topics/agent-harness.md>), [AI Agent](<https://devfeed.tech/topics/ai-agent.md>), [Test-driven development](<https://devfeed.tech/topics/tdd.md>)

Tags: [admission](<https://devfeed.tech/tags/admission.md>), [agent](<https://devfeed.tech/tags/agent.md>), [asynchronous](<https://devfeed.tech/tags/asynchronous.md>), [contracts](<https://devfeed.tech/tags/contracts.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-se](<https://devfeed.tech/tags/cs-se.md>), [effectful](<https://devfeed.tech/tags/effectful.md>), [execution](<https://devfeed.tech/tags/execution.md>), [fail](<https://devfeed.tech/tags/fail.md>), [graph](<https://devfeed.tech/tags/graph.md>), [hidden](<https://devfeed.tech/tags/hidden.md>), [independent](<https://devfeed.tech/tags/independent.md>), [latency](<https://devfeed.tech/tags/latency.md>), [localization](<https://devfeed.tech/tags/localization.md>), [parallel](<https://devfeed.tech/tags/parallel.md>), [parallelism](<https://devfeed.tech/tags/parallelism.md>), [patch](<https://devfeed.tech/tags/patch.md>), [runtime](<https://devfeed.tech/tags/runtime.md>), [semantic](<https://devfeed.tech/tags/semantic.md>), [tool](<https://devfeed.tech/tags/tool.md>), [tool-execution](<https://devfeed.tech/tags/tool-execution.md>), [translation](<https://devfeed.tech/tags/translation.md>)

### AI overview

RefinementGuard qualifies hidden, effectful agent-tool execution by comparing a candidate parallel graph with an independent reference on isolated state, using explicit obligations for the parent operation and fail-closed, artifact-bound admission. In controlled repository evaluations, obligation-based validation rejected 49 of 100 activated non-equivalent faults and accepted all 60 valid variants; exact equality rejected all faults but also all valid variants. The 51 residual misses point to specification gaps. On a disjoint held-out split, results cover localization only: B3 rejected 38 of 56 localization defects, while 21 patch mutants failed structurally. Under controlled asynchronous delays, guarded localization had a 4.50x lower median latency than serial execution and exposed 74.3% less parent-visible context. Patch validation had no demonstrated speedup, and zero-delay runs showed overhead.

### Source excerpt

arXiv:2611.23611v1 Announce Type: cross Abstract: Tool runtimes can lower an agent-selected operation into a hidden parallel graph, reducing latency and context while risking semantic mistranslation. We introduce RefinementGuard: translation validation for hidden, effectful agent-tool lowerings, coupled to artifact-bound, fail-closed runtime admission. Qualification compares a candidate graph with an independent reference on isolated state using explicit parent-operation obligations. We evaluate two repository-level contracts, bug localization and patch validation, on controlled repositories. Among 100 activated, non-equivalent development/qualification faults adjudicated by one human, parent-semantic validation rejects 49/100 (operator-cluster bootstrap 95% CI [33.7%, 64.8%]). The objective is selective admission, not maximum mutation rejection: exact equality rejects 100/100 faults but also all 60 valid variants, whereas obligation-based validation accepts 60/60; its 51 residual misses identify concrete specification gaps. On a frozen disjoint input split, the held-out B2-B3 result is localization-only because 21 patch mutants fail structurally; B3 rejects 38/56 localization defects. In a controlled asynchronous-delay regime, the localization serial/guarded median-latency ratio is 4.50x, and hidden execution exposes 74.3% less parent-visible context than visible execution. Patch validation shows no demonstrated speedup; zero-delay runs expose overhead. End-to-end qualification thus detects lower-layer misses while preserving localization's parallelism.

## Why the Harness Matters: Early Lessons From the Evo Code Scanning Agent Preview

DevFeed: [Why the Harness Matters: Early Lessons From the Evo Code Scanning Agent Preview](<https://devfeed.tech/articles/why-the-harness-matters-early-lessons-from-the-evo-code-scanning-agent-preview-146509.md>)

Original publisher: [Read original article](<https://snyk.io/blog/why-harness-matters-evo-code-scanning-agent/>)

Author: Brian Clark; Adrian Meier; Pauline Bolignano

Published: 2026-11-19T04:00:00Z

Content type: news

Language: en

Sources: [Snyk](<https://devfeed.tech/sources/blog-rss-feed-snyk.md>)

Topics: [code security](<https://devfeed.tech/topics/code-security.md>), [Agent Harness](<https://devfeed.tech/topics/agent-harness.md>), [Static code analysis](<https://devfeed.tech/topics/static-code-analysis.md>), [AI Agent](<https://devfeed.tech/topics/ai-agent.md>), [LLM security](<https://devfeed.tech/topics/llm-security.md>), [Security](<https://devfeed.tech/topics/security.md>)

Tags: [2026](<https://devfeed.tech/tags/2026.md>), [adoption](<https://devfeed.tech/tags/adoption.md>), [agent](<https://devfeed.tech/tags/agent.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-harness](<https://devfeed.tech/tags/ai-harness.md>), [ai-model](<https://devfeed.tech/tags/ai-model.md>), [ai-reasoning](<https://devfeed.tech/tags/ai-reasoning.md>), [ai-security-agents](<https://devfeed.tech/tags/ai-security-agents.md>), [blog](<https://devfeed.tech/tags/blog.md>), [budget](<https://devfeed.tech/tags/budget.md>), [claude](<https://devfeed.tech/tags/claude.md>), [code](<https://devfeed.tech/tags/code.md>), [code-scanning](<https://devfeed.tech/tags/code-scanning.md>), [code-security](<https://devfeed.tech/tags/code-security.md>), [codex](<https://devfeed.tech/tags/codex.md>), [compute](<https://devfeed.tech/tags/compute.md>), [developer](<https://devfeed.tech/tags/developer.md>), [engineering](<https://devfeed.tech/tags/engineering.md>), [harness](<https://devfeed.tech/tags/harness.md>), [interest](<https://devfeed.tech/tags/interest.md>), [precision](<https://devfeed.tech/tags/precision.md>), [preview](<https://devfeed.tech/tags/preview.md>), [security](<https://devfeed.tech/tags/security.md>), [security-labs](<https://devfeed.tech/tags/security-labs.md>), [snyk-code](<https://devfeed.tech/tags/snyk-code.md>), [tests](<https://devfeed.tech/tags/tests.md>), [vulnerabilities](<https://devfeed.tech/tags/vulnerabilities.md>), [vulnerability-insights](<https://devfeed.tech/tags/vulnerability-insights.md>)

### AI overview

At equal compute, Evo Code Scanning Agent's security harness found a larger share of known vulnerabilities than either same-model lab tool across 14 applications: 57% versus Claude Security's 32% on Claude Sonnet 5.5, and 66% versus Codex Security deep's 43% on GPT-6.1 Sol. Precision was lower: 65% versus 94% for Claude and 90% versus 97.5% for Codex. Each lab tool also found some vulnerabilities Evo missed. Since each pair used the same model, the article attributes differences to the surrounding harness--how it plans work, divides code, investigates candidates, and selects findings. Results are early and based on Snyk's internal benchmark of 1,426 known vulnerabilities; Evo is still in experimental preview, and no tool found them all. The authors recommend treating AI scanners as one layer alongside SAST, SCA, and code review.

### Source excerpt

In early tests, Evo Code Scanning Agent's security harness found more known vulnerabilities than Claude Security and Codex Security at the same compute budget, with lower precision. See what the results reveal about the role of the harness in AI code scanning.

## Your coding agent bill doubled. Here's how to fix it.

DevFeed: [Your coding agent bill doubled. Here's how to fix it.](<https://devfeed.tech/articles/your-coding-agent-bill-doubled-here-s-how-to-fix-it-142925.md>)

Original publisher: [Read original article](<https://www.langchain.com/blog/fix-your-coding-agent-bill>)

Author: LangChain Accounts

Published: 2026-11-18T19:33:12Z

Content type: article

Language: en

Sources: [LangChain Blog](<https://devfeed.tech/sources/langchain-blog.md>)

Topics: [codex](<https://devfeed.tech/topics/codex.md>), [Claude Code](<https://devfeed.tech/topics/claude-code.md>), [cursor](<https://devfeed.tech/topics/cursor.md>), [GitHub Copilot](<https://devfeed.tech/topics/github-copilot.md>), [Agent Harness](<https://devfeed.tech/topics/agent-harness.md>), [Multi Agent Systems](<https://devfeed.tech/topics/multi-agent-systems.md>)

Tags: [claude-code](<https://devfeed.tech/tags/claude-code.md>), [coding-agent](<https://devfeed.tech/tags/coding-agent.md>), [copilot](<https://devfeed.tech/tags/copilot.md>), [cost](<https://devfeed.tech/tags/cost.md>), [cost-management](<https://devfeed.tech/tags/cost-management.md>), [cursor](<https://devfeed.tech/tags/cursor.md>), [engine](<https://devfeed.tech/tags/engine.md>), [gateway](<https://devfeed.tech/tags/gateway.md>), [trace](<https://devfeed.tech/tags/trace.md>)

### AI overview

Coding agent costs become hard to assess when teams use multiple tools that record activity in incompatible formats. The article proposes a cycle of shared visibility, cost comparison, optimization, and spending controls. LangSmith normalizes traces from Claude Code, Codex, Cursor, GitHub Copilot Chat, Pi, and OpenCode, including model usage, tool calls, and subagent activity, so teams can query and compare sessions in one place. Engine analyzes sessions for waste such as repeated context retrieval and suggests changes; LangSmith's LLM Gateway provides user, team, and organization cost caps, with open source model routing described as forthcoming. The article recommends starting with observability and adding optimization and governance as needed. It says teams standardized on one tool with an adequate native dashboard may not need another layer until they adopt a second agent.

### Source excerpt

Learn why coding agent bills spiral out of control -- and how to trace, compare, and govern spend across Claude Code, Cursor, Copilot, and more in one place.

## The Board Pack for an AI Agent: 12 Things a Bank Board Should See Before Approving

DevFeed: [The Board Pack for an AI Agent: 12 Things a Bank Board Should See Before Approving](<https://devfeed.tech/articles/the-board-pack-for-an-ai-agent-12-things-a-bank-board-should-see-before-approving-141054.md>)

Original publisher: [Read original article](<https://www.lyzr.ai/blog/twelve-things-a-bank-board-should-see/>)

Author: Lyzr Team

Published: 2026-11-18T09:20:59Z

Content type: article

Language: en

Sources: [Lyzr.ai](<https://devfeed.tech/sources/lyzr-ai.md>)

Topics: [AI Agent](<https://devfeed.tech/topics/ai-agent.md>), [human in the loop agent](<https://devfeed.tech/topics/human-in-the-loop-agent.md>), [Agent Harness](<https://devfeed.tech/topics/agent-harness.md>), [agent observability](<https://devfeed.tech/topics/agent-observability.md>), [AI software supply chain security](<https://devfeed.tech/topics/ai-software-supply-chain-security.md>)

Tags: [accountability](<https://devfeed.tech/tags/accountability.md>), [actions](<https://devfeed.tech/tags/actions.md>), [adversarial](<https://devfeed.tech/tags/adversarial.md>), [agent](<https://devfeed.tech/tags/agent.md>), [agent-control](<https://devfeed.tech/tags/agent-control.md>), [agent-observability](<https://devfeed.tech/tags/agent-observability.md>), [agent-safety](<https://devfeed.tech/tags/agent-safety.md>), [agentic](<https://devfeed.tech/tags/agentic.md>), [agentic-ai](<https://devfeed.tech/tags/agentic-ai.md>), [agents](<https://devfeed.tech/tags/agents.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-agent](<https://devfeed.tech/tags/ai-agent.md>), [ai-agent-observability](<https://devfeed.tech/tags/ai-agent-observability.md>), [ai-agent-safety](<https://devfeed.tech/tags/ai-agent-safety.md>), [ai-agents](<https://devfeed.tech/tags/ai-agents.md>), [ai-incident](<https://devfeed.tech/tags/ai-incident.md>), [ai-policy](<https://devfeed.tech/tags/ai-policy.md>), [ai-security](<https://devfeed.tech/tags/ai-security.md>), [ai-security-risks](<https://devfeed.tech/tags/ai-security-risks.md>), [bank](<https://devfeed.tech/tags/bank.md>), [board](<https://devfeed.tech/tags/board.md>)

### AI overview

The article proposes a 12 item board pack, grouped around an agent's boundaries, accountable people, evidence, and alignment with supervisory expectations and business value. It calls for action limits and fallback behavior, named owners and an independent stop authority, failure-focused testing and agent-specific security evidence, third-party provenance, measurable costs and benefits, and ongoing reporting. Scrutiny should scale with autonomy and potential harm. The authors say no US regulator has published a board checklist for AI agents; this is their synthesis of supervisory signals and practitioner guidance, much of it vendor advice, not a regulatory requirement. They describe Madison as maintaining governance records and Opencontroller as supplying runtime controls and evidence; neither replaces board judgment, and a gateway governs only traffic that passes through it.

### Source excerpt

A bank board should approve the next AI agent only after seeing a short pack that answers twelve questions in four groups: what the agent...

## Revision-Consistent Code Context for Agents That Edit Their Environment

DevFeed: [Revision-Consistent Code Context for Agents That Edit Their Environment](<https://devfeed.tech/articles/revision-consistent-code-context-for-agents-that-edit-their-environment-139576.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.21616>)

Author: Meng Shi

Published: 2026-11-18T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Software Engineering](<https://devfeed.tech/sources/arxiv-software-engineering.md>)

Topics: [AI Context Management](<https://devfeed.tech/topics/ai-context-management.md>), [code search](<https://devfeed.tech/topics/code-search.md>), [Agent Harness](<https://devfeed.tech/topics/agent-harness.md>), [async](<https://devfeed.tech/topics/async.md>)

Tags: [abstract](<https://devfeed.tech/tags/abstract.md>), [after](<https://devfeed.tech/tags/after.md>), [agent](<https://devfeed.tech/tags/agent.md>), [agents](<https://devfeed.tech/tags/agents.md>), [all](<https://devfeed.tech/tags/all.md>), [arxiv](<https://devfeed.tech/tags/arxiv.md>), [boundary](<https://devfeed.tech/tags/boundary.md>), [call](<https://devfeed.tech/tags/call.md>), [can](<https://devfeed.tech/tags/can.md>), [code](<https://devfeed.tech/tags/code.md>), [code-intelligence](<https://devfeed.tech/tags/code-intelligence.md>), [comparison](<https://devfeed.tech/tags/comparison.md>), [complete](<https://devfeed.tech/tags/complete.md>), [composition](<https://devfeed.tech/tags/composition.md>), [condition](<https://devfeed.tech/tags/condition.md>), [content](<https://devfeed.tech/tags/content.md>), [context](<https://devfeed.tech/tags/context.md>), [contribution](<https://devfeed.tech/tags/contribution.md>), [cs-se](<https://devfeed.tech/tags/cs-se.md>), [environment](<https://devfeed.tech/tags/environment.md>), [revision](<https://devfeed.tech/tags/revision.md>)

### AI overview

The paper proposes a complete-revision publication contract to prevent code-intelligence queries from combining repository facts from different update states. A service must wait, report unavailability, or return a complete old revision before publication; afterward, it returns the complete new revision, with a shared version identity across all views in each response. In Codetrip, which joins source, symbol, and graph facts, this approach produced no unsafe responses in 150 Gin v1.12.0 update trials, compared with two for live-source composition. Journaled, idempotent publication recovered to context equivalent to a clean rebuild in all 300 evaluable trials across frozen Gin and Cobra, with median restart-to-context latency of 77-131 ms. A separate rename probe found mixed context in 0/30 Codetrip trials and 30/30 trials each for CodeGraph and codebase-memory-mcp before clean-full-equivalent convergence. These results are limited to the reported controlled trials.

### Source excerpt

arXiv:2611.21616v1 Announce Type: new Abstract: Editing agents query repositories while simultaneously changing them. Source content, symbols, graph relations, and retrieval state may be maintained by different stores and refreshed at different times. During an update, a successful query can therefore combine facts that never belonged to the same repository state. We call this condition mixed-revision code context. This paper proposes an Agent-visible complete-revision publication contract for composite code-intelligence queries. Before a query-visible publication point, a service must wait, report unavailability, or serve a complete old revision; after that point, it serves the complete new revision. The contract requires a shared version identity and publication boundary across every view returned in one response. We instantiate the method in Codetrip, whose brief operation joins source, symbol, and graph facts. In a controlled comparison on frozen Gin v1.12.0, live-source composition produced two unsafe responses in 150 update trials, whereas complete-revision publication produced none under the same five edit scenarios. We further implement journaled, idempotent publication and terminate the real MCP process at five durable boundaries. All 300 evaluable recovery trials on frozen Gin and Cobra returned complete-new context equivalent to a clean full rebuild, with median restart-to-context latency of 77--131 ms. In a separate rename probe, mixed context occurred in 0/30 Codetrip trials and 30/30 trials for both CodeGraph and codebase-memory-mcp before clean-full-equivalent convergence. The contribution is an Agent-facing safety contract, implementation pattern, and measurement protocol for joining live repository bytes with asynchronously derived facts. Every successful composite response consequently has a testable repository-state meaning, including after interruption.

## Evolution Under Fire: Evolving Agent Safety Harnesses with Adaptive Injection Stressors

DevFeed: [Evolution Under Fire: Evolving Agent Safety Harnesses with Adaptive Injection Stressors](<https://devfeed.tech/articles/evolution-under-fire-evolving-agent-safety-harnesses-with-adaptive-injection-stressors-140052.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.22583>)

Author: Yujun Wang, Yilun Liu, Jinhe Bi, Krishna Kanth Nakka, Xuebing Zhou, Hinrich Sch{\\"u}tze, Yunpu Ma

Published: 2026-11-18T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Cryptography and Security](<https://devfeed.tech/sources/arxiv-cryptography-and-security.md>)

Topics: [Agent Harness](<https://devfeed.tech/topics/agent-harness.md>), [prompt injection](<https://devfeed.tech/topics/prompt-injection.md>), [AI Agent](<https://devfeed.tech/topics/ai-agent.md>)

Tags: [abstract](<https://devfeed.tech/tags/abstract.md>), [act](<https://devfeed.tech/tags/act.md>), [adaptive](<https://devfeed.tech/tags/adaptive.md>), [agent](<https://devfeed.tech/tags/agent.md>), [agent-safety](<https://devfeed.tech/tags/agent-safety.md>), [agents](<https://devfeed.tech/tags/agents.md>), [also](<https://devfeed.tech/tags/also.md>), [arxiv](<https://devfeed.tech/tags/arxiv.md>), [attack](<https://devfeed.tech/tags/attack.md>), [attacks](<https://devfeed.tech/tags/attacks.md>), [benchmark](<https://devfeed.tech/tags/benchmark.md>), [can](<https://devfeed.tech/tags/can.md>), [central](<https://devfeed.tech/tags/central.md>), [challenge](<https://devfeed.tech/tags/challenge.md>), [channel](<https://devfeed.tech/tags/channel.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cl](<https://devfeed.tech/tags/cs-cl.md>), [cs-cr](<https://devfeed.tech/tags/cs-cr.md>), [evolution](<https://devfeed.tech/tags/evolution.md>), [fire](<https://devfeed.tech/tags/fire.md>), [safety](<https://devfeed.tech/tags/safety.md>), [under](<https://devfeed.tech/tags/under.md>), [with](<https://devfeed.tech/tags/with.md>)

### AI overview

Evolution Under Fire alternates between evolving executable safety harnesses and generating adaptive, utility-preserving injection attacks. A utility-constrained, Pareto-scheduled tree search uses execution evidence while holding the task model, benchmark, tools, and evaluator fixed; discovered attacks then guide another harness-evolution stage. In reported results, the AgentDyn/GPT-4o-mini attack success rate fell from 50.4% to 3.0%, with improved clean and attacked utility over the initial harness and a stronger security-utility trade-off than reproduced defenses. On Agent-SafetyBench with Qwen2.5-7B-Instruct, it fell from 82.6% to 62.0% while helpfulness and runtime validity were retained. Without further evolution, transfer to Qwen3.5-35B-A3B reduced attack success from 32.1% to 0.9% and improved attacked-task utility.

### Source excerpt

arXiv:2611.22583v1 Announce Type: cross Abstract: Tool-using language-model agents must act on information from untrusted emails, webpages, and documents. This creates a central security challenge: malicious instructions and legitimate task directives can arrive through the same interaction channel. Existing defenses are largely hand-designed and static, making further improvements dependent on repeated manual redesign. We introduce Evolution Under Fire, a staged framework that couples safety-harness evolution with adaptive stressor evolution. Utility-constrained, Pareto-scheduled tree search evolves complete executable safety harnesses from execution evidence while keeping the task model, benchmark, tools, and evaluator fixed. Adaptive stressor evolution then discovers utility-preserving attacks that guide a subsequent stage of harness evolution. On AgentDyn with GPT-4o-mini, an evolved harness reduces attack success from 50.4% to 3.0%, improves clean and attacked utility over the initial harness, and offers a stronger security-utility trade-off than reproduced defenses. On Agent-SafetyBench with Qwen2.5-7B-Instruct, harness evolution reduces attack success from 82.6% to 62.0% while retaining helpfulness and runtime validity. Evolved harnesses also transfer without further evolution to Qwen3.5-35B-A3B, reducing attack success from 32.1% to 0.9% while improving attacked-task utility.

[Next page](<https://devfeed.tech/topics/agent-harness.md?cursor=WyIyMDI2LTExLTE4VDA1OjAwOjAwKzAwOjAwIiwgImU3ZTU4NDYxLTE3NTctNDY4Ni05NGMzLTJiOWMyN2YxYzc5NCJd>)