# ai-experiments

AI experiments are temporary, small, controlled pilots of new AI tools not yet offered as standard services, with a defined purpose, limited users, and limited duration.

This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles.

## Clairic Hit 98% Accuracy on 72.8% of Questions via Confidence Scores

DevFeed: [Clairic Hit 98% Accuracy on 72.8% of Questions via Confidence Scores](<https://devfeed.tech/articles/clairic-hit-98-accuracy-on-72-8-of-questions-via-confidence-scores-146441.md>)

Original publisher: [Read original article](<https://towardsdatascience.com/clairic-hit-98-accuracy-on-728-of-questions-via-confidence-scores/>)

Author: Mariya Mansurova

Published: 2026-11-20T15:00:00Z

Content type: tutorial

Language: en

Sources: [Towards Data Science](<https://devfeed.tech/sources/towards-data-science.md>)

Topics: [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>), [Data Science](<https://devfeed.tech/topics/data-science.md>), [ai-experiments](<https://devfeed.tech/topics/ai-experiments.md>), [Benchmark](<https://devfeed.tech/topics/benchmark.md>)

Tags: [analysts](<https://devfeed.tech/tags/analysts.md>), [arithmetic](<https://devfeed.tech/tags/arithmetic.md>), [artificial-intelligence](<https://devfeed.tech/tags/artificial-intelligence.md>), [autocomplete](<https://devfeed.tech/tags/autocomplete.md>), [based](<https://devfeed.tech/tags/based.md>), [bike](<https://devfeed.tech/tags/bike.md>), [charts](<https://devfeed.tech/tags/charts.md>), [chatgpt](<https://devfeed.tech/tags/chatgpt.md>), [classification](<https://devfeed.tech/tags/classification.md>), [cloudflare](<https://devfeed.tech/tags/cloudflare.md>), [complex](<https://devfeed.tech/tags/complex.md>), [confidence](<https://devfeed.tech/tags/confidence.md>), [confidence-interval](<https://devfeed.tech/tags/confidence-interval.md>), [control](<https://devfeed.tech/tags/control.md>), [data-science](<https://devfeed.tech/tags/data-science.md>), [decision-models](<https://devfeed.tech/tags/decision-models.md>), [easy](<https://devfeed.tech/tags/easy.md>), [editor-s-picks](<https://devfeed.tech/tags/editor-s-picks.md>), [generative-ai](<https://devfeed.tech/tags/generative-ai.md>), [jev](<https://devfeed.tech/tags/jev.md>), [llm](<https://devfeed.tech/tags/llm.md>), [llm-applications](<https://devfeed.tech/tags/llm-applications.md>), [luna](<https://devfeed.tech/tags/luna.md>), [model](<https://devfeed.tech/tags/model.md>), [science](<https://devfeed.tech/tags/science.md>), [system-one](<https://devfeed.tech/tags/system-one.md>)

### AI overview

Clairic is a fast, low-cost extraction model that accepts text and images and can return typed values with confidence scores. In tests against Luna, Clairic was faster but less accurate overall, especially on calculations; examples also included confidently incorrect extractions. It handled some chart-based extraction and returned null for missing revenue. On 1,360 questions across four languages, a 0.9 confidence cutoff yielded 98% accuracy while Clairic answered 72.8% of questions. The author suggests using it for high-confidence extraction and routing the rest to a larger model, with validation because confidence is imperfect. The tests did not establish a reason to prefer it for forecasting over traditional methods.

### Source excerpt

With a 0.9 confidence cutoff, Cloudflare's System One model handles the easy cases and routes the rest to a bigger, slower LLM like Luna. The post Clairic Hit 98% Accuracy on 72.8% of Questions via Confidence Scores appeared first on Towards Data Science.

## A Seven-Slide Framework for Funding AI Transformation Without New Headcount

DevFeed: [A Seven-Slide Framework for Funding AI Transformation Without New Headcount](<https://devfeed.tech/articles/a-seven-slide-framework-for-funding-ai-transformation-without-new-headcount-140465.md>)

Original publisher: [Read original article](<https://hackernoon.com/a-seven-slide-framework-for-funding-ai-transformation-without-new-headcount?source=rss>)

Author: marian kamenistak

Published: 2026-11-18T07:00:02Z

Content type: tutorial

Language: en

Sources: [HackerNoon](<https://devfeed.tech/sources/hackernoon.md>)

Topics: [roadmaps](<https://devfeed.tech/topics/roadmaps.md>), [finops](<https://devfeed.tech/topics/finops.md>), [ai-experiments](<https://devfeed.tech/topics/ai-experiments.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [ai-adoption-strategy](<https://devfeed.tech/tags/ai-adoption-strategy.md>), [ai-transformation](<https://devfeed.tech/tags/ai-transformation.md>), [ai-transformation-costs](<https://devfeed.tech/tags/ai-transformation-costs.md>), [ai-workforce-planning](<https://devfeed.tech/tags/ai-workforce-planning.md>), [business](<https://devfeed.tech/tags/business.md>), [business-strategy](<https://devfeed.tech/tags/business-strategy.md>), [capacity](<https://devfeed.tech/tags/capacity.md>), [case](<https://devfeed.tech/tags/case.md>), [engineering](<https://devfeed.tech/tags/engineering.md>), [engineering-capacity-planning](<https://devfeed.tech/tags/engineering-capacity-planning.md>), [engineering-management](<https://devfeed.tech/tags/engineering-management.md>), [funding](<https://devfeed.tech/tags/funding.md>), [product](<https://devfeed.tech/tags/product.md>), [rice-prioritization](<https://devfeed.tech/tags/rice-prioritization.md>), [roadmap](<https://devfeed.tech/tags/roadmap.md>)

### AI overview

The article argues that an AI transformation without new hires should be funded in roadmap capacity, making the central question what product and engineering will jointly defer or drop. It proposes seven slides: the decision, product's expected payoff, an honest capacity breakdown, shared product and platform cuts, options to reduce scope/stretch/drop work, cost of delay, and written commitments from both leaders. It recommends showing the product benefit before the cuts, presenting them as a shared choice, and circulating the proposal and capacity table as a pre-read after discussing it with decision makers individually. The throughput figures, including a modeled recovery to about 3x by year-end, are illustrative assumptions rather than measured results; the article says leaders should commit to the multiplier and treat AI-generated cut suggestions as proposals.

### Source excerpt

Learn how engineering leaders can fund AI transformation through roadmap capacity, negotiate product trade-offs, and build a seven-slide business case.

## Does Your Agent Need to Know Its Budget? A Placebo-Controlled Experiment on LLM Coding Agents

DevFeed: [Does Your Agent Need to Know Its Budget? A Placebo-Controlled Experiment on LLM Coding Agents](<https://devfeed.tech/articles/does-your-agent-need-to-know-its-budget-a-placebo-controlled-experiment-on-llm-coding-agents-139574.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.21614>)

Author: Iftesam Nabi, Jarif Ul Alam, Sunzid Hassan, Sabbir Alom Shuvo, Md Faiaz Al Islam, Yead Rahman, Mehrab Hosain

Published: 2026-11-18T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Software Engineering](<https://devfeed.tech/sources/arxiv-software-engineering.md>)

Topics: [agentic-coding](<https://devfeed.tech/topics/agentic-coding.md>), [ai-experiments](<https://devfeed.tech/topics/ai-experiments.md>)

Tags: [agent](<https://devfeed.tech/tags/agent.md>), [agents](<https://devfeed.tech/tags/agents.md>), [budget](<https://devfeed.tech/tags/budget.md>), [coding](<https://devfeed.tech/tags/coding.md>), [coding-agents](<https://devfeed.tech/tags/coding-agents.md>), [cs-se](<https://devfeed.tech/tags/cs-se.md>), [experiment](<https://devfeed.tech/tags/experiment.md>)

### AI overview

In this experiment, enforcing token caps on coding agents reduced task resolution by 20 percentage points while cutting tokens per resolved task roughly twelvefold. Telling agents about an enforced budget did not improve resolution: placebo-controlled effects were -2.8 points at the medium cap and -4.2 points at the tight cap (the latter nominally significant and unadjusted). Prompt wording and per-turn feedback variants remained below the 10-point effect size of interest. An unrelated sentence raised resolution by 7.2 points over silent enforcement, showing why a placebo control matters. Results cover two executable program-repair benchmarks, two open models, and three cap levels; the source notes that run-to-run outcome variance under caps was about 1.5 times the uncapped level, partly because resolution rates moved closer to one half.

### Source excerpt

arXiv:2611.21614v1 Announce Type: new Abstract: Organizations deploying autonomous coding agents impose token budgets, yet the causal effects of a budget, and of telling the agent about it, are unmeasured. We conduct a placebo-controlled factorial experiment under a frozen, timestamped protocol, crossing budget enforcement with budget disclosure on two executable program-repair benchmarks (5,835 runs, two open models, three cap levels). Enforcing a calibrated cap reduced task resolution by 20.0 percentage points while reducing tokens per resolved task roughly twelvefold. Disclosing the enforced budget never helped: the placebo-controlled risk difference was -2.8 pp (95% CI [-6.2, +0.7]) at the medium cap and -4.2 pp ([-8.1, -0.2], nominally significant, unadjusted) at the tight cap, with every prompt wording and per-turn feedback variant far below the 10-point effect size of interest. Observed run-to-run outcome variance was roughly 1.5x the uncapped level under caps, though part of this reflects resolution rates moving nearer one half. These results make budget governance for coding agents an enforcement-design problem: no communication variant produced an effect beyond that of an unrelated sentence, which itself raised resolution by 7.2 pp (95% CI [+4.0, +10.5]) over silent enforcement, a sentence effect that only a placebo control removes.

## The Resolution Hypothesis: Agent Evolution's Key Is Identifying Improvements

DevFeed: [The Resolution Hypothesis: Agent Evolution's Key Is Identifying Improvements](<https://devfeed.tech/articles/the-resolution-hypothesis-agent-evolution-s-key-is-identifying-improvements-140114.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.23068>)

Author: Yuanhao Zeng, Shaocheng Lan, Ao Lu, Kuang Yan, Heng Lin, Xinang Xu, Yuan Luo, Yueting Wang, Yexin Li, Ruo Shan, Kan Ren

Published: 2026-11-18T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [ai safety](<https://devfeed.tech/topics/ai-safety.md>), [ai-experiments](<https://devfeed.tech/topics/ai-experiments.md>)

Tags: [agent](<https://devfeed.tech/tags/agent.md>), [agents](<https://devfeed.tech/tags/agents.md>), [artifacts](<https://devfeed.tech/tags/artifacts.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cl](<https://devfeed.tech/tags/cs-cl.md>), [evaluation](<https://devfeed.tech/tags/evaluation.md>), [evaluations](<https://devfeed.tech/tags/evaluations.md>), [evolution](<https://devfeed.tech/tags/evolution.md>), [hypothesis](<https://devfeed.tech/tags/hypothesis.md>), [improvement](<https://devfeed.tech/tags/improvement.md>), [improvements](<https://devfeed.tech/tags/improvements.md>), [random](<https://devfeed.tech/tags/random.md>), [ranking](<https://devfeed.tech/tags/ranking.md>)

### AI overview

Across seven agent evolution methods and six models, evaluated over 10 million LLM calls, the study finds that noisy evaluation can prevent reliable identification of improvements. At the candidate level, helpful artifacts exist, but selection is no better than random at distinguishing them from harmful ones, allowing errors to accumulate. At the method level, typical performance margins of a few percentage points are smaller than variation between independent runs of the same method, making rankings unreliable. The authors argue that progress depends on making real improvements detectable and accumulable, rather than focusing only on proposing more candidate artifacts.

### Source excerpt

arXiv:2611.23068v1 Announce Type: new Abstract: Agent evolution follows a simple loop: a proposer writes candidate artifacts (prompts, skills, memories) and selection accepts those that score higher. This loop has shown promise, yet practitioners face a puzzle: methods trade wins across papers, and evolution often fails to bootstrap on certain tasks. We trace this puzzle to the Resolution Hypothesis: agent evolution stalls not because methods fail to propose better artifacts, but because noisy evaluations cannot reliably identify them. Across seven evolution methods and six models (over 10M LLM calls), we find support for the hypothesis at both levels where evaluation scores are used. At the candidate level (choosing artifacts), helpful candidate artifacts exist, but selection is no better than random at telling them from harmful ones, so errors accumulate across rounds. At the method level (ranking methods), typical margins of a few percentage points are smaller than the difference between two independent runs of the same method, so published rankings often rest on luck. Together, these findings indicate that agent evolution is often bottlenecked by reliably identifying improvements. This calls for a shift in how we develop and evaluate self-improving agents, from designing ever more candidate improvement systems to making true improvements detectable and accumulable.

## Building an Agent Factory for Data Migrations: a Small Test with Snowflake

DevFeed: [Building an Agent Factory for Data Migrations: a Small Test with Snowflake](<https://devfeed.tech/articles/building-an-agent-factory-for-data-migrations-a-small-test-with-snowflake-139492.md>)

Original publisher: [Read original article](<https://medium.com/snowflake/building-an-agent-factory-for-data-migrations-a-small-test-with-snowflake-9d4622c8f2c7?source=rss----34b6daafc07---4>)

Author: Eylon Steiner

Published: 2026-11-17T18:01:03Z

Content type: article

Language: en

Sources: [Snowflake Builders](<https://devfeed.tech/sources/snowflake-builders.md>)

Topics: [supervisor agent pattern](<https://devfeed.tech/topics/supervisor-agent-pattern.md>), [data-engineering](<https://devfeed.tech/topics/data-engineering.md>), [human in the loop agent](<https://devfeed.tech/topics/human-in-the-loop-agent.md>), [ai-experiments](<https://devfeed.tech/topics/ai-experiments.md>), [AI Chat](<https://devfeed.tech/topics/ai-chat.md>), [audit](<https://devfeed.tech/topics/audit.md>)

Tags: [agent-factory](<https://devfeed.tech/tags/agent-factory.md>), [agentic-ai](<https://devfeed.tech/tags/agentic-ai.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-agents](<https://devfeed.tech/tags/ai-agents.md>), [data-engineering](<https://devfeed.tech/tags/data-engineering.md>), [data-migration](<https://devfeed.tech/tags/data-migration.md>), [data-migrations](<https://devfeed.tech/tags/data-migrations.md>), [database-migration](<https://devfeed.tech/tags/database-migration.md>), [parallel](<https://devfeed.tech/tags/parallel.md>), [postgresql](<https://devfeed.tech/tags/postgresql.md>), [proof-of-concept](<https://devfeed.tech/tags/proof-of-concept.md>), [snowflake](<https://devfeed.tech/tags/snowflake.md>), [tests](<https://devfeed.tech/tags/tests.md>)

### AI overview

A small Pagila proof of concept migrated PostgreSQL views and functions to Snowflake using parallel AI translators, scripted comparisons, and an independent AI verifier. Scripts handled dependency waves, deployed translations only in throwaway schemas, and compared results on real data and edge cases; human decisions were logged for reuse. All 15 tables passed 352 profile checks after a timestamp-precision bug in the check was fixed. Among 12 proven logic objects, repeated runs and one run with a cheaper translator passed, while planted-bug trials showed data diffs missed 3 of 5 bugs and edge tests caught all 5 (a small sample tested once). The author stresses that matching real data can be accidental, empty results prove little, and harness errors caused many apparent failures--so checks themselves need scrutiny.

### Source excerpt

Building an agent factory for data migrations: a small test with Snowflake How I moved a PostgreSQL database's views and functions to Snowflake with parallel AI agents, scripted checks and an independent AI reviewer, and what broke along the way. The short version I built a small proof of concept (PoC) of an "agent factory" for SQL migration: AI translators working in parallel, scripts that test every translation against the real data, and a second, independent AI whose only job is to find what the tests missed. The test case: Pagila, a public PostgreSQL sample database, moved to Snowflake. The goal of this architecture is to migrate a database's logic (views, functions, procedures), not just its data, at a scale where "one AI plus one human reviewer" stops working, and without ever letting an AI grade its own work: Scale: many objects translated in parallel, in dependency order. Trust: scripts prove every translation against the real data and edge cases, then an independent reviewer looks for what the scripts missed. Human time only where it matters: people answer clear questions the factory can't decide, and each answer is reused. Why a PoC: before using this on a real migration, I wanted to learn on something small and public which parts work, where the agents fail, and what the scripts must catch. The most useful lesson: matching data is not the same as correct logic. Some translations "matched" PostgreSQL on the real data purely by luck. If you have never built anything with AI agents, this post is for you. I explain every part. 1. The problem: copying tables is easy, copying logic is not A database migration has two halves: Data: tables and rows. Boring, mechanical, solved by tools. Code: views, functions and procedures written in the source's SQL dialect. Every one must be rewritten for the target and checked. The code half is where projects stall. A common setup today is "one AI assistant plus one human": the AI translates an object, the human reviews it, re

## AgentFaultBench: State-Verified Evaluation of Tool-Fault Recovery

DevFeed: [AgentFaultBench: State-Verified Evaluation of Tool-Fault Recovery](<https://devfeed.tech/articles/agentfaultbench-state-verified-evaluation-of-tool-fault-recovery-137606.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.20711>)

Author: Sachin Gupta

Published: 2026-11-17T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Interactive AI Systems](<https://devfeed.tech/topics/interactive-ai-systems.md>), [Test automation](<https://devfeed.tech/topics/test-automation.md>), [ai-experiments](<https://devfeed.tech/topics/ai-experiments.md>)

Tags: [advice](<https://devfeed.tech/tags/advice.md>), [agent](<https://devfeed.tech/tags/agent.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [errors](<https://devfeed.tech/tags/errors.md>), [evaluation](<https://devfeed.tech/tags/evaluation.md>), [execution](<https://devfeed.tech/tags/execution.md>), [experiments](<https://devfeed.tech/tags/experiments.md>), [features](<https://devfeed.tech/tags/features.md>), [feedback](<https://devfeed.tech/tags/feedback.md>), [files](<https://devfeed.tech/tags/files.md>), [generation](<https://devfeed.tech/tags/generation.md>), [gitlab](<https://devfeed.tech/tags/gitlab.md>), [installation](<https://devfeed.tech/tags/installation.md>), [issue](<https://devfeed.tech/tags/issue.md>), [it](<https://devfeed.tech/tags/it.md>), [langgraph](<https://devfeed.tech/tags/langgraph.md>), [recovery](<https://devfeed.tech/tags/recovery.md>), [run](<https://devfeed.tech/tags/run.md>), [sol](<https://devfeed.tech/tags/sol.md>), [state](<https://devfeed.tech/tags/state.md>), [tool](<https://devfeed.tech/tags/tool.md>)

### AI overview

AgentFaultBench evaluates whether recovery advice helps LangGraph and CrewAI agents complete specified tasks after tool errors in an isolated GitLab installation. It verifies resulting files and issues, required reads and checks, and the final response; expected issue and report text is supplied, so the benchmark measures execution rather than content generation. In 1,944 planned primary runs, advice changed success by −8.33, 0.00, and +1.06 percentage points versus a neutral reminder; missing outcomes make the latter two comparisons inconclusive. Under a revised policy, GPT-4o mini improved, while Sol and Claude already completed recoverable cases. On six related tasks, guidance yielded 161 correct final states and one unknown in 192 planned runs, versus 142/192 with neutral feedback, a reported gain of +9.90 to +10.42 points including missing outcomes. Removing issue IDs produced 47/48 successes versus 46/48 with IDs, with other execution information retained. A second checker agreed with 1,879 observed primary outcomes and 241 constructed cases. No human validation was collected.

### Source excerpt

arXiv:2611.20711v1 Announce Type: new Abstract: Does recovery advice help an agent finish a task after a tool fails? AgentFaultBench evaluates LangGraph and CrewAI agents performing specified tasks in an isolated GitLab installation. We inject tool errors and check both the resulting files and issues and the required reads, verification and final response. The prompt supplies the expected issue and report text, so the task tests execution rather than content generation. In the primary comparison of 1,944 planned runs, advice changes success by -8.33, 0.00 and +1.06 percentage points relative to a neutral reminder; missing outcomes leave the latter two comparisons inconclusive. Later experiments test a revised recovery policy. GPT-4o mini improves, while Sol and Claude already complete the recoverable cases. In a separate 624-run study on six related tasks not previously used by that policy, the guided condition records 161 correct final states and one unknown outcome in 192 planned runs, versus 142/192 with neutral feedback. The gain lies between +9.90 and +10.42 pp when missing outcomes are included. Removing issue IDs gives 47/48 successes versus 46/48 with IDs; other execution information remains. A second checker agrees on 1,879 observed primary outcomes and 241 constructed test cases. No human validation was collected. The findings support testing advice against explicit recovery controls rather than assuming it helps.

## Behavioral Assurance for Socially Deployed AI Agents

DevFeed: [Behavioral Assurance for Socially Deployed AI Agents](<https://devfeed.tech/articles/behavioral-assurance-for-socially-deployed-ai-agents-137466.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.19564>)

Author: Nicole H Brennan, Jiaee Cheong

Published: 2026-11-17T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [phoenix](<https://devfeed.tech/topics/phoenix.md>), [ai-experiments](<https://devfeed.tech/topics/ai-experiments.md>)

Tags: [agent-behavior](<https://devfeed.tech/tags/agent-behavior.md>), [agents](<https://devfeed.tech/tags/agents.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-agents](<https://devfeed.tech/tags/ai-agents.md>), [behavior](<https://devfeed.tech/tags/behavior.md>), [consistency](<https://devfeed.tech/tags/consistency.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-et](<https://devfeed.tech/tags/cs-et.md>), [cs-hc](<https://devfeed.tech/tags/cs-hc.md>), [cs-ma](<https://devfeed.tech/tags/cs-ma.md>), [evaluation](<https://devfeed.tech/tags/evaluation.md>), [framework](<https://devfeed.tech/tags/framework.md>), [interaction](<https://devfeed.tech/tags/interaction.md>), [persona](<https://devfeed.tech/tags/persona.md>), [settings](<https://devfeed.tech/tags/settings.md>), [testing](<https://devfeed.tech/tags/testing.md>)

### AI overview

The framework tests whether socially deployed AI agents behave consistently with intended characteristics across interactions. It measures behavior on five axes using a standardized battery of six socially salient settings, with controlled persona configurations as anchors for assessing differentiation, consistency, and drift. Preliminary analyses show measurable differences across configurations and suggest associations between behavioral profiles and interaction outcomes. The abstract does not specify the five axes or report detailed results.

### Source excerpt

arXiv:2611.19564v1 Announce Type: new Abstract: AI agents are increasingly deployed in socially consequential settings, yet existing evaluations provide limited assurance that they behave consistently with their intended characteristics across diverse interactions. We introduce an evaluation framework for measuring the behavioral consistency of socially deployed AI agents. The framework operationalizes agent behavior along five measurable axes and uses a standardized evaluation battery spanning six socially salient interaction settings to assess behavioral differentiation, consistency, and drift. Controlled persona configurations serve as experimental anchors, enabling systematic testing of whether intended behavioral differences persist across contexts rather than appearing only as stylistic variation. Preliminary analyses demonstrate measurable differentiation across configurations and suggest associations between behavioral profiles and interaction outcomes. By shifting the focus from persona design to behavioral assurance, this work provides a foundation for evaluating whether socially deployed AI agents behave as intended, consistently and measurably, across the situations in which users encounter them.

## Why agentic AI pilots don't scale

DevFeed: [Why agentic AI pilots don't scale](<https://devfeed.tech/articles/why-agentic-ai-pilots-don-t-scale-139733.md>)

Original publisher: [Read original article](<https://www.thoughtworks.com/insights/blog/machine-learning-and-ai/why-agentic-ai-pilots-dont-scale>)

Author: Published November

Published: 2026-11-17T00:00:00Z

Content type: article

Language: en

Sources: [Thoughtworks Insights](<https://devfeed.tech/sources/thoughtworks-insights.md>)

Topics: [Agentic AI](<https://devfeed.tech/topics/what-is-agentic-ai.md>), [AI Integration Strategies](<https://devfeed.tech/topics/ai-integration-strategies.md>), [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [ai-experiments](<https://devfeed.tech/topics/ai-experiments.md>), [CI/CD](<https://devfeed.tech/topics/cicd.md>), [elt](<https://devfeed.tech/topics/elt.md>)

Tags: [abap](<https://devfeed.tech/tags/abap.md>), [agent](<https://devfeed.tech/tags/agent.md>), [agentic](<https://devfeed.tech/tags/agentic.md>), [agentic-ai](<https://devfeed.tech/tags/agentic-ai.md>), [agents](<https://devfeed.tech/tags/agents.md>), [ai](<https://devfeed.tech/tags/ai.md>), [api](<https://devfeed.tech/tags/api.md>), [apis](<https://devfeed.tech/tags/apis.md>), [architecture](<https://devfeed.tech/tags/architecture.md>), [article](<https://devfeed.tech/tags/article.md>), [automatically](<https://devfeed.tech/tags/automatically.md>), [autonomous](<https://devfeed.tech/tags/autonomous.md>), [blog](<https://devfeed.tech/tags/blog.md>), [bottleneck](<https://devfeed.tech/tags/bottleneck.md>), [business](<https://devfeed.tech/tags/business.md>), [business-logic](<https://devfeed.tech/tags/business-logic.md>), [capabilities](<https://devfeed.tech/tags/capabilities.md>), [catalog](<https://devfeed.tech/tags/catalog.md>)

### AI overview

The article argues that agentic AI pilots stall chiefly when legacy systems and unprepared data prevent enterprise integration and scale. It proposes a three-stage path: use AI-first reverse engineering to document legacy code, design an evolutionary target architecture, and replace monolith functions gradually with the strangler fig pattern; automate governed data product creation and migrate legacy data platforms; then let agents use decoupled, API-enabled capabilities and curated data through MCP. It claims simple data products can be built in hours and complex ones in up to two weeks, while its migration accelerator can be up to 60 times faster and cut end-to-end lead time by up to 90%; these are vendor-stated results, with a Unieuro example. The text presents modernization as risk mitigation and a growth enabler, while describing governance and operating-model fragmentation as additional scaling challenges.

### Source excerpt

Why agentic AI pilots don't scale

## Enterprises to pay up to 20% more for Microsoft cloud services in some countries

DevFeed: [Enterprises to pay up to 20% more for Microsoft cloud services in some countries](<https://devfeed.tech/articles/enterprises-to-pay-up-to-20-more-for-microsoft-cloud-services-in-some-countries-136137.md>)

Original publisher: [Read original article](<https://www.networkworld.com/article/4248987/enterprises-to-pay-up-to-20-more-for-microsoft-cloud-services-in-some-countries-2.html>)

Author: Anirban Ghoshal

Published: 2026-11-16T11:47:38Z

Content type: news

Language: en

Sources: [Network World](<https://devfeed.tech/sources/network-world.md>)

Topics: [Microsoft](<https://devfeed.tech/topics/microsoft.md>), [Azure](<https://devfeed.tech/topics/azure.md>), [ai-experiments](<https://devfeed.tech/topics/ai-experiments.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [azure](<https://devfeed.tech/tags/azure.md>), [budget](<https://devfeed.tech/tags/budget.md>), [budget-cloud-computing-iaas-microsoft-azure](<https://devfeed.tech/tags/budget-cloud-computing-iaas-microsoft-azure.md>), [cloud](<https://devfeed.tech/tags/cloud.md>), [cloud-computing](<https://devfeed.tech/tags/cloud-computing.md>), [cloud-services](<https://devfeed.tech/tags/cloud-services.md>), [costs](<https://devfeed.tech/tags/costs.md>), [customers](<https://devfeed.tech/tags/customers.md>), [data-and-analytics](<https://devfeed.tech/tags/data-and-analytics.md>), [delay](<https://devfeed.tech/tags/delay.md>), [edition-us](<https://devfeed.tech/tags/edition-us.md>), [enterprise](<https://devfeed.tech/tags/enterprise.md>), [enterprise-it](<https://devfeed.tech/tags/enterprise-it.md>), [enterprises](<https://devfeed.tech/tags/enterprises.md>), [iaas](<https://devfeed.tech/tags/iaas.md>), [india](<https://devfeed.tech/tags/india.md>), [indian](<https://devfeed.tech/tags/indian.md>), [innovation](<https://devfeed.tech/tags/innovation.md>), [it](<https://devfeed.tech/tags/it.md>), [it-budgets](<https://devfeed.tech/tags/it-budgets.md>), [language-en](<https://devfeed.tech/tags/language-en.md>), [microsoft](<https://devfeed.tech/tags/microsoft.md>), [microsoft-365](<https://devfeed.tech/tags/microsoft-365.md>), [microsoft-azure](<https://devfeed.tech/tags/microsoft-azure.md>), [new-zealand](<https://devfeed.tech/tags/new-zealand.md>), [storytype-news](<https://devfeed.tech/tags/storytype-news.md>)

### AI overview

Microsoft plans annual local-currency price adjustments for commercial cloud services, effective January 1, 2027: prices rise 15.7% in India and 10% in New Zealand, while some on-premises offerings rise by up to 20% in India and 10% in New Zealand. The company says customers and partners will receive notice each November. China, Australia, and Norway will see commercial cloud prices fall by up to 8.5%, 4%, and 5.9%; Brazil and Japan follow separate schedules. Analysts say higher costs may constrain innovation spending and prompt consideration of alternatives, though moving existing workloads can be costly and complex, and cloud is not automatically cheaper. Suggested near-term steps include removing unused licenses, right-sizing Azure resources, renewing before the increase, seeking price protection, and comparing full lifecycle costs. These adjustments reflect foreign-exchange movements.

### Source excerpt

Microsoft is slated to raise commercial cloud prices in several markets, with customers in India and New Zealand facing the steepest price hikes. The changes, which are set to take effect from January 1, 2027, will increase commercial cloud prices by 15.7% in India and 10% in New Zealand, the company wrote in a blog post announcing its annual local currency pricing adjustments to account for foreign exchange fluctuations. The annual local currency pricing adjustments follow Microsoft's announcement in July, wherein it said that it would move to an annual process for adjusting local currency prices for commercial cloud services. Under the new approach, adjustments are expected to take place every January, with customers and partners notified in advance each November. The pricing changes also extend to some on-premises offerings, with Indian customers facing increases of up to 20% and those in New Zealand facing a 10% hike, according to the blog post. Higher prices could squeeze enterprise IT budgets This change in prices, particularly for Microsoft's India customers, is "significant because it outpaces most enterprises' annual IT budget growth," said Manoj Chandra Jha, principal analyst at Nord-IQ Research. "That means the adjustments are likely to result in a real cut in spending power for those enterprises rather than a rounding error. The hardest hit will be organizations heavily standardized on Microsoft, particularly in sectors such as BFSI, IT services, global capability centers (GCCs), and large-scale manufacturing," Jha added. "Within those, the workloads most exposed are the ones with constant, heavy usage: Microsoft 365 seats at scale, data and analytics platforms, and Azure-based AI," he further added. Innovation projects could bear the brunt of higher costs That additional pressure on IT budgets is also likely to come at the expense of discretionary technology spending. "Enterprises are likely to delay or shelve innovation projects, particularly AI pilots

## Enterprises to pay up to 20% more for Microsoft cloud services in some countries

DevFeed: [Enterprises to pay up to 20% more for Microsoft cloud services in some countries](<https://devfeed.tech/articles/enterprises-to-pay-up-to-20-more-for-microsoft-cloud-services-in-some-countries-136231.md>)

Original publisher: [Read original article](<https://www.computerworld.com/article/4248407/enterprises-to-pay-up-to-20-more-for-microsoft-cloud-services-in-some-countries.html>)

Author: Anirban Ghoshal

Published: 2026-11-16T11:44:02Z

Content type: news

Language: en

Sources: [Computerworld](<https://devfeed.tech/sources/computerworld.md>)

Topics: [Microsoft](<https://devfeed.tech/topics/microsoft.md>), [Azure](<https://devfeed.tech/topics/azure.md>), [microsoft 365](<https://devfeed.tech/topics/microsoft-365.md>), [Google Cloud Platform (GCP)](<https://devfeed.tech/topics/google-cloud.md>), [ai-experiments](<https://devfeed.tech/topics/ai-experiments.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [azure](<https://devfeed.tech/tags/azure.md>), [budget](<https://devfeed.tech/tags/budget.md>), [cloud](<https://devfeed.tech/tags/cloud.md>), [cloud-computing](<https://devfeed.tech/tags/cloud-computing.md>), [cloud-computing-microsoft-vendors-and-providers](<https://devfeed.tech/tags/cloud-computing-microsoft-vendors-and-providers.md>), [cloud-services](<https://devfeed.tech/tags/cloud-services.md>), [costs](<https://devfeed.tech/tags/costs.md>), [customers](<https://devfeed.tech/tags/customers.md>), [data-and-analytics](<https://devfeed.tech/tags/data-and-analytics.md>), [delay](<https://devfeed.tech/tags/delay.md>), [edition-us](<https://devfeed.tech/tags/edition-us.md>), [enterprise](<https://devfeed.tech/tags/enterprise.md>), [enterprise-it](<https://devfeed.tech/tags/enterprise-it.md>), [enterprises](<https://devfeed.tech/tags/enterprises.md>), [exchange](<https://devfeed.tech/tags/exchange.md>), [growth](<https://devfeed.tech/tags/growth.md>), [india](<https://devfeed.tech/tags/india.md>), [innovation](<https://devfeed.tech/tags/innovation.md>), [it](<https://devfeed.tech/tags/it.md>), [it-budgets](<https://devfeed.tech/tags/it-budgets.md>), [it-services](<https://devfeed.tech/tags/it-services.md>), [language-en](<https://devfeed.tech/tags/language-en.md>), [microsoft](<https://devfeed.tech/tags/microsoft.md>), [storytype-news](<https://devfeed.tech/tags/storytype-news.md>), [vendors-and-providers](<https://devfeed.tech/tags/vendors-and-providers.md>)

### AI overview

Starting January 1, 2027, Microsoft plans to raise commercial cloud prices by 15.7% in India and 10% in New Zealand; some on-premises offerings will rise by up to 20% in India and 10% in New Zealand. Microsoft says annual local-currency adjustments will occur each January, with advance notice each November. Analysts cited in the excerpt say the increases could constrain budgets for organizations with heavy Microsoft use, delay innovation projects, and prompt diversification for new deployments, though migrating existing workloads can be costly and complex. Suggested cost steps include removing unused Microsoft 365 seats, right-sizing Azure resources, renewing before the change to seek current rates and price protection, and considering reserved instances or savings plans. The excerpt also notes cloud price reductions in China, Australia, and Norway, while warning that cloud migration is not automatically cheaper and should be assessed against full lifecycle costs.

### Source excerpt

Microsoft is slated to raise commercial cloud prices in several markets, with customers in India and New Zealand facing the steepest price hikes. The changes, which are set to take effect from January 1, 2027, will increase commercial cloud prices by 15.7% in India and 10% in New Zealand, the company wrote in a blog post announcing its annual local currency pricing adjustments to account for foreign exchange fluctuations. The annual local currency pricing adjustments follow Microsoft's announcement in July, wherein it said that it would move to an annual process for adjusting local currency prices for commercial cloud services. Under the new approach, adjustments are expected to take place every January, with customers and partners notified in advance each November. The pricing changes also extend to some on-premises offerings, with Indian customers facing increases of up to 20% and those in New Zealand facing a 10% hike, according to the blog post. Higher prices could squeeze enterprise IT budgets This change in prices, particularly for Microsoft's India customers, is "significant because it outpaces most enterprises' annual IT budget growth," said Manoj Chandra Jha, principal analyst at Nord-IQ Research. "That means the adjustments are likely to result in a real cut in spending power for those enterprises rather than a rounding error. The hardest hit will be organizations heavily standardized on Microsoft, particularly in sectors such as BFSI, IT services, global capability centers (GCCs), and large-scale manufacturing," Jha added. "Within those, the workloads most exposed are the ones with constant, heavy usage: Microsoft 365 seats at scale, data and analytics platforms, and Azure-based AI," he further added. Innovation projects could bear the brunt of higher costs That additional pressure on IT budgets is also likely to come at the expense of discretionary technology spending. "Enterprises are likely to delay or shelve innovation projects, particularly AI pilots

## TReC: Evaluating Move-Induced State Changes in Instruction-Conditioned Go

DevFeed: [TReC: Evaluating Move-Induced State Changes in Instruction-Conditioned Go](<https://devfeed.tech/articles/trec-evaluating-move-induced-state-changes-in-instruction-conditioned-go-135112.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.17781>)

Author: Zhuangsheng Li, Yunhan Zhang, Dailin Li, Qingxian Yu, Yu Wu, Youqi Song, Bangyuan He

Published: 2026-11-16T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [ai alignment](<https://devfeed.tech/topics/ai-alignment.md>), [ai-experiments](<https://devfeed.tech/topics/ai-experiments.md>), [cosine similarity search](<https://devfeed.tech/topics/cosine-similarity-search.md>)

Tags: [artificial-intelligence](<https://devfeed.tech/tags/artificial-intelligence.md>), [candidate](<https://devfeed.tech/tags/candidate.md>), [consistency](<https://devfeed.tech/tags/consistency.md>), [correlation](<https://devfeed.tech/tags/correlation.md>), [cosine](<https://devfeed.tech/tags/cosine.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [embedding](<https://devfeed.tech/tags/embedding.md>), [experiments](<https://devfeed.tech/tags/experiments.md>), [game](<https://devfeed.tech/tags/game.md>), [go](<https://devfeed.tech/tags/go.md>), [layer](<https://devfeed.tech/tags/layer.md>), [median](<https://devfeed.tech/tags/median.md>), [model](<https://devfeed.tech/tags/model.md>), [network](<https://devfeed.tech/tags/network.md>), [position](<https://devfeed.tech/tags/position.md>), [prediction](<https://devfeed.tech/tags/prediction.md>), [reference](<https://devfeed.tech/tags/reference.md>), [rules](<https://devfeed.tech/tags/rules.md>), [similarity](<https://devfeed.tech/tags/similarity.md>), [state](<https://devfeed.tech/tags/state.md>)

### AI overview

Transition Representation Consistency (TReC) compares two Go moves by executing both from the same position under game rules, measuring the cosine similarity between the resulting changes in board-state embeddings. In experiments using a frozen KataGo network on 680 mid-to-late-game positions from five games, TReC produced larger within-position gaps between the highest and lowest candidate scores (median 0.88-1.33) than direct successor-state cosine (0.01-0.08), depending on evaluator layer. Its relationship to candidate distance from the expert move was weak: median within-position rank correlation was -0.19. These findings describe the reported experiment, not broader validation.

### Source excerpt

arXiv:2611.17781v1 Announce Type: new Abstract: When a Go model predicts a move different from the expert's, coordinate distance measures how far the prediction lands from the reference. We introduce Transition Representation Consistency (TReC) to compare how the two moves change a board-state embedding. Both moves are executed from the same position under the game rules, and TReC is the cosine similarity between the two resulting representation changes. In experiments with a frozen KataGo network on 680 mid-to-late-game positions from five games, the gap between the highest and lowest candidate per position has a median of 0.01 to 0.08 for the direct successor-state cosine and 0.88 to 1.33 for TReC, depending on the evaluator layer. TReC follows coordinate distance only weakly: the rank correlation between a candidate's distance to the reference and its TReC, computed within each position, has a median of -0.19 over these positions.

## Apple launches new app to study how people use AI for health

DevFeed: [Apple launches new app to study how people use AI for health](<https://devfeed.tech/articles/apple-launches-new-app-to-study-how-people-use-ai-for-health-134630.md>)

Original publisher: [Read original article](<https://techcrunch.com/2026/11/15/apple-launches-new-app-to-study-how-people-use-ai-for-health/>)

Author: Ivan Mehta

Published: 2026-11-16T01:30:00Z

Content type: news

Language: en

Sources: [TechCrunch | AI News & Artificial Intelligence](<https://devfeed.tech/sources/techcrunch-ai-news-artificial-intelligence.md>)

Topics: [AI Chat](<https://devfeed.tech/topics/ai-chat.md>), [ai-experiments](<https://devfeed.tech/topics/ai-experiments.md>), [human computer interaction](<https://devfeed.tech/topics/human-computer-interaction.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [ai-labs](<https://devfeed.tech/tags/ai-labs.md>), [ai-models](<https://devfeed.tech/tags/ai-models.md>), [ai-powered](<https://devfeed.tech/tags/ai-powered.md>), [ai-startups](<https://devfeed.tech/tags/ai-startups.md>), [anthropic](<https://devfeed.tech/tags/anthropic.md>), [app](<https://devfeed.tech/tags/app.md>), [apple](<https://devfeed.tech/tags/apple.md>), [apple-intelligence](<https://devfeed.tech/tags/apple-intelligence.md>), [apps](<https://devfeed.tech/tags/apps.md>), [chatbots](<https://devfeed.tech/tags/chatbots.md>), [chatgpt](<https://devfeed.tech/tags/chatgpt.md>), [claude](<https://devfeed.tech/tags/claude.md>), [cloud](<https://devfeed.tech/tags/cloud.md>), [cloud-provider](<https://devfeed.tech/tags/cloud-provider.md>), [everyday](<https://devfeed.tech/tags/everyday.md>), [health](<https://devfeed.tech/tags/health.md>), [medical](<https://devfeed.tech/tags/medical.md>), [openevidence](<https://devfeed.tech/tags/openevidence.md>), [people](<https://devfeed.tech/tags/people.md>), [questions](<https://devfeed.tech/tags/questions.md>), [study](<https://devfeed.tech/tags/study.md>)

### AI overview

Apple and OpenEvidence have launched a U.S. research app to study how people use clinically grounded AI for everyday health questions, rather than announcing a general Apple health chatbot. It offers simplified answers sourced from clinical literature, lets participants rate responses and complete monthly surveys, and can use shared HealthKit data for more personalized suggestions. Apple is recruiting up to 200,000 participants over age 21 with an iPhone 15 or later, iOS 26 or later, and Apple Intelligence enabled; the study is planned to run for more than 12 months. Apple says data sent to a secure third-party cloud provider is anonymized and not used to train AI models; OpenEvidence deletes query and response data after 24 hours. The app may help users prepare for doctor visits and direct them to providers for concerning issues. The article says no future integration with Apple Health or Siri has been confirmed.

### Source excerpt

Apple's new app for a study will answer everyday medical questions

## Saying no to AI spend is how we protect AI value

DevFeed: [Saying no to AI spend is how we protect AI value](<https://devfeed.tech/articles/saying-no-to-ai-spend-is-how-we-protect-ai-value-131449.md>)

Original publisher: [Read original article](<https://www.cio.com/article/4248060/saying-no-to-ai-spend-is-how-we-protect-ai-value.html>)

Author: Jay Miller; Ashley Rice

Published: 2026-11-13T10:00:00Z

Content type: opinion

Language: en

Sources: [CIO](<https://devfeed.tech/sources/cio.md>)

Topics: [finops](<https://devfeed.tech/topics/finops.md>), [shadow AI](<https://devfeed.tech/topics/shadow-ai.md>), [Claude](<https://devfeed.tech/topics/claude.md>), [ai-experiments](<https://devfeed.tech/topics/ai-experiments.md>), [Finance](<https://devfeed.tech/topics/finance.md>)

Tags: [agents](<https://devfeed.tech/tags/agents.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-value](<https://devfeed.tech/tags/ai-value.md>), [alerts](<https://devfeed.tech/tags/alerts.md>), [artificial-intelligence](<https://devfeed.tech/tags/artificial-intelligence.md>), [artificial-intelligence-budgeting-it-leadership-it-management-roi-and-metrics](<https://devfeed.tech/tags/artificial-intelligence-budgeting-it-leadership-it-management-roi-and-metrics.md>), [automation](<https://devfeed.tech/tags/automation.md>), [budget](<https://devfeed.tech/tags/budget.md>), [budgeting](<https://devfeed.tech/tags/budgeting.md>), [claude](<https://devfeed.tech/tags/claude.md>), [contributor](<https://devfeed.tech/tags/contributor.md>), [cost](<https://devfeed.tech/tags/cost.md>), [course](<https://devfeed.tech/tags/course.md>), [cross](<https://devfeed.tech/tags/cross.md>), [dev](<https://devfeed.tech/tags/dev.md>), [edition-us](<https://devfeed.tech/tags/edition-us.md>), [experimentation](<https://devfeed.tech/tags/experimentation.md>), [innovation](<https://devfeed.tech/tags/innovation.md>), [investment](<https://devfeed.tech/tags/investment.md>), [it](<https://devfeed.tech/tags/it.md>), [it-leadership](<https://devfeed.tech/tags/it-leadership.md>), [it-management](<https://devfeed.tech/tags/it-management.md>), [language-en](<https://devfeed.tech/tags/language-en.md>), [leaks](<https://devfeed.tech/tags/leaks.md>), [limit](<https://devfeed.tech/tags/limit.md>), [monitoring](<https://devfeed.tech/tags/monitoring.md>), [monthly](<https://devfeed.tech/tags/monthly.md>), [roi-and-metrics](<https://devfeed.tech/tags/roi-and-metrics.md>), [storytype-opinion](<https://devfeed.tech/tags/storytype-opinion.md>), [value](<https://devfeed.tech/tags/value.md>)

### AI overview

Paessler argues that AI spending limits should make usage a deliberate business decision while preserving room for valuable work. After experimentation and agent use drove unexpectedly high bills, it recommends setting group or per-person spending and token limits at rollout, then increasing them when a clear need justifies it. A cross-functional AI Center of Excellence reviews tools, checks whether existing products can meet needs, tests security, and provides shared guidance and training. The article also describes teaching employees that Claude's higher-effort responses use more tokens, and consolidating an app when the main platform could do the same job. Caps can prompt either a justified increase for critical work or a pause on low-priority experimentation; the process includes ongoing checks of performance, business value, and cost.

### Source excerpt

It's the 17th of the month and one of our engineers wants more credits for Claude. They've already spent $4,000 and we can't believe it. Barely halfway through July and they've not only overshot their monthly budget but need an even larger allocation. Soon after, alerts start going out that we've reached 75% of our allocated monthly spend limit. This request, among many others, made us think twice about AI spend. The interest is there and often the innovation is, too. But what about the value? If anyone asked us to approve an IT product that costs a few thousand dollars per seat, per month, would we say yes? Especially when the cost fluctuates and the return on investment isn't guaranteed? At Paessler, like many companies, we're building the AI boat as we sail it. The trouble is that spend leaks are common in the wild waters of automation. We realized the need to correct course: value-vs-spend checks serve as a necessary compass and a cross-functional AI team keeps the ship structurally sound, preventing things like shadow AI or unchecked agents from sinking the budget. We've learned the hard way that protecting AI value means being willing to sometimes say no, putting the processes and the people in place to make that call before the bill arrives. Heading into the new year, let's explore why "tokenmaxxing" is out and "valuemaxxing" is in. AI got a longer leash than any other software Earlier this year, our first AI-focused dev day showed us just how quickly expenses can get out of hand. We offered a $5,000 prize for the best feature we could build into our network monitoring platform. The team loved it, resulting in a record 29 submissions, with some deliberately choosing more expensive approaches to build things that would otherwise have been impossible, ranging from context-aware assistants to root cause analysis helpers. We encouraged that experimentation and gave a very long leash on cost, precisely because we'd never done it before and wanted to see the outcom

## Your AI ROI problem is a culture problem

DevFeed: [Your AI ROI problem is a culture problem](<https://devfeed.tech/articles/your-ai-roi-problem-is-a-culture-problem-131448.md>)

Original publisher: [Read original article](<https://www.cio.com/article/4248056/your-ai-roi-problem-is-a-culture-problem.html>)

Author: Caitlyn Brazill

Published: 2026-11-13T09:00:00Z

Content type: opinion

Language: en

Sources: [CIO](<https://devfeed.tech/sources/cio.md>)

Topics: [AI Integration Strategies](<https://devfeed.tech/topics/ai-integration-strategies.md>), [ai-experiments](<https://devfeed.tech/topics/ai-experiments.md>), [finops](<https://devfeed.tech/topics/finops.md>), [Tool](<https://devfeed.tech/topics/tool.md>), [User Experience](<https://devfeed.tech/topics/user-experience.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [ai-costs](<https://devfeed.tech/tags/ai-costs.md>), [ai-implementation](<https://devfeed.tech/tags/ai-implementation.md>), [ai-roi](<https://devfeed.tech/tags/ai-roi.md>), [algorithms](<https://devfeed.tech/tags/algorithms.md>), [anxiety](<https://devfeed.tech/tags/anxiety.md>), [artificial-intelligence](<https://devfeed.tech/tags/artificial-intelligence.md>), [artificial-intelligence-employee-experience-it-leadership-it-management-roi-and-metrics](<https://devfeed.tech/tags/artificial-intelligence-employee-experience-it-leadership-it-management-roi-and-metrics.md>), [boston-consulting-group](<https://devfeed.tech/tags/boston-consulting-group.md>), [ceo](<https://devfeed.tech/tags/ceo.md>), [challenges](<https://devfeed.tech/tags/challenges.md>), [cio](<https://devfeed.tech/tags/cio.md>), [cios](<https://devfeed.tech/tags/cios.md>), [contributor](<https://devfeed.tech/tags/contributor.md>), [cost](<https://devfeed.tech/tags/cost.md>), [cost-per-token](<https://devfeed.tech/tags/cost-per-token.md>), [culture](<https://devfeed.tech/tags/culture.md>), [data](<https://devfeed.tech/tags/data.md>), [edition-us](<https://devfeed.tech/tags/edition-us.md>), [efficiency](<https://devfeed.tech/tags/efficiency.md>), [employee-experience](<https://devfeed.tech/tags/employee-experience.md>), [enterprise](<https://devfeed.tech/tags/enterprise.md>), [generative](<https://devfeed.tech/tags/generative.md>), [generative-ai](<https://devfeed.tech/tags/generative-ai.md>), [it-leadership](<https://devfeed.tech/tags/it-leadership.md>), [it-management](<https://devfeed.tech/tags/it-management.md>), [language-en](<https://devfeed.tech/tags/language-en.md>), [roi](<https://devfeed.tech/tags/roi.md>), [roi-and-metrics](<https://devfeed.tech/tags/roi-and-metrics.md>), [storytype-opinion](<https://devfeed.tech/tags/storytype-opinion.md>)

### AI overview

The article argues that AI returns depend heavily on people and processes: it cites a BCG survey attributing about 70% of implementation challenges to those factors, and says high-value companies direct 70% of investment there. It recommends setting a named owner, baseline and business metric before starting; giving teams room to run bounded experiments and pivot; and building tools with the people who use them. Examples include ResumeBoost, which cut staff resume support time by over a third before its functionality was folded into a broader coaching platform, and Sherlock, built with job sourcing staff to filter more than 125,000 postings daily. Three weeks after launch, staff at nearly three-quarters of campuses had logged in and users reported saving over 100 hours. The article presents this as early progress and says continued improvement depends on dialogue between builders and users.

### Source excerpt

These days, when I talk to CIOs about how their organizations are using AI, there's a lot less excitement about new tools and the potential for higher productivity, and a lot more anxiety about what the return on investment has been so far. Because for most companies, that return has been less than expected, especially as cost per token is increasing. Last year, IBM's 2025 CEO study found that only 25% of AI initiatives had delivered the expected ROI over the last few years, and only 16% had scaled enterprise-wide. MIT's NANDA initiative found that 95% of generative AI pilots delivered no measurable P&L (profit and loss) impact. More recently, SAP and Oxford Economics' Value of AI Report 2026 found that 69% of businesses worldwide are "satisfied" with the ROI of their AI costs - but more than two-thirds think there's still room for improvement. It's tempting to read those numbers as a verdict on AI's limitations. But the research points somewhere else. When Boston Consulting Group surveyed 1,000 senior executives, it found roughly 70% of AI implementation challenges stemmed from people and processes, 20% from technology and 10% from the algorithms themselves. The companies seeing the most value from AI? They were investing 10% in algorithms, 20% in technology and data, and 70% in people and processes. As CIO, you don't just own the tools. You lead the teams that determine whether the tools pay off. My own experience with AI - integrating it into Per Scholas's classrooms, systems and processes to achieve even stronger outcomes for our learners - has convinced me that there are three cultural traits separating teams that produce returns from teams that produce pilots: they are results-oriented, entrepreneurial and collegial. Start with the result, and measure progress Decide what you're trying to change before choosing what to invest in. McKinsey's 2025 State of AI survey found that while 80% of companies set efficiency as an objective for their AI initiatives, the co

## See what AI looks like in production at Microsoft Ignite

DevFeed: [See what AI looks like in production at Microsoft Ignite](<https://devfeed.tech/articles/see-what-ai-looks-like-in-production-at-microsoft-ignite-130471.md>)

Original publisher: [Read original article](<https://thenewstack.io/microsoft-ignite-2026/>)

Author: Nick Lucchesi

Published: 2026-11-12T14:00:00Z

Content type: news

Language: en

Sources: [The New Stack](<https://devfeed.tech/sources/the-new-stack.md>)

Topics: [Microsoft](<https://devfeed.tech/topics/microsoft.md>), [ai-experiments](<https://devfeed.tech/topics/ai-experiments.md>), [Azure](<https://devfeed.tech/topics/azure.md>), [microsoft 365](<https://devfeed.tech/topics/microsoft-365.md>), [Azure OpenAI](<https://devfeed.tech/topics/azure-openai.md>), [CI/CD](<https://devfeed.tech/topics/cicd.md>)

Tags: [adobe](<https://devfeed.tech/tags/adobe.md>), [agent](<https://devfeed.tech/tags/agent.md>), [agent-365](<https://devfeed.tech/tags/agent-365.md>), [agentic](<https://devfeed.tech/tags/agentic.md>), [agents](<https://devfeed.tech/tags/agents.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-agents](<https://devfeed.tech/tags/ai-agents.md>), [ai-in-production](<https://devfeed.tech/tags/ai-in-production.md>), [ai-infrastructure](<https://devfeed.tech/tags/ai-infrastructure.md>), [ai-operations](<https://devfeed.tech/tags/ai-operations.md>), [ai-strategy](<https://devfeed.tech/tags/ai-strategy.md>), [ai-tools](<https://devfeed.tech/tags/ai-tools.md>), [airlines](<https://devfeed.tech/tags/airlines.md>), [alaska](<https://devfeed.tech/tags/alaska.md>), [analytics](<https://devfeed.tech/tags/analytics.md>), [azure](<https://devfeed.tech/tags/azure.md>), [bank-of-england](<https://devfeed.tech/tags/bank-of-england.md>), [keynote](<https://devfeed.tech/tags/keynote.md>), [microsoft](<https://devfeed.tech/tags/microsoft.md>), [microsoft-ignite](<https://devfeed.tech/tags/microsoft-ignite.md>), [post](<https://devfeed.tech/tags/post.md>), [production](<https://devfeed.tech/tags/production.md>), [satya-nadella](<https://devfeed.tech/tags/satya-nadella.md>), [sponsor-microsoft](<https://devfeed.tech/tags/sponsor-microsoft.md>), [sponsored](<https://devfeed.tech/tags/sponsored.md>), [sponsored-post](<https://devfeed.tech/tags/sponsored-post.md>)

### AI overview

Microsoft Ignite 2026 is presented as a place to evaluate how AI pilots might move into production through customer sessions, demonstrations, and peer discussions. The preview describes Microsoft's Frontier Transformation approach: Microsoft 365 Copilot brings AI into work tools; Fabric and Microsoft IQ connect data and context; GitHub Copilot and Microsoft Foundry support building and deploying agents on Azure; and Agent 365 lets IT teams monitor agents and set policies. Four sessions focus on cloud-to-edge infrastructure, employee adoption, grounding agents in company information, and observability. Examples are scheduled from organizations including Lilly, the Bank of England, and others; Microsoft reports that more than 40,000 Lilly employees use Copilot and over 150 agents support its research, clinical-trial, and operations work. These are previewed sessions and reported customer examples, not independently detailed deployment evaluations. The keynote is scheduled for November 17, with a free digital pass available.

### Source excerpt

If you're deciding which AI pilots to move into production, Microsoft's Satya Nadella and Judson Althoff's opening keynote at Microsoft Ignite The post See what AI looks like in production at Microsoft Ignite appeared first on The New Stack.

## Google tests Tabletop, letting groups build planning tools with an AI agent

DevFeed: [Google tests Tabletop, letting groups build planning tools with an AI agent](<https://devfeed.tech/articles/google-tests-tabletop-letting-groups-build-planning-tools-with-an-ai-agent-129596.md>)

Original publisher: [Read original article](<https://siliconangle.com/2026/11/11/google-tests-tabletop-letting-groups-build-planning-tools-with-an-ai-agent/>)

Author: Duncan Riley

Published: 2026-11-11T22:37:09Z

Content type: news

Language: en

Sources: [SiliconANGLE](<https://devfeed.tech/sources/siliconangle.md>)

Topics: [AI Agent](<https://devfeed.tech/topics/ai-agent.md>), [AI Chat](<https://devfeed.tech/topics/ai-chat.md>), [ai-experiments](<https://devfeed.tech/topics/ai-experiments.md>)

Tags: [agent](<https://devfeed.tech/tags/agent.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-agent](<https://devfeed.tech/tags/ai-agent.md>), [ai-agents](<https://devfeed.tech/tags/ai-agents.md>), [artificial-intelligence](<https://devfeed.tech/tags/artificial-intelligence.md>), [build](<https://devfeed.tech/tags/build.md>), [cc](<https://devfeed.tech/tags/cc.md>), [chatgpt](<https://devfeed.tech/tags/chatgpt.md>), [collaboration](<https://devfeed.tech/tags/collaboration.md>), [disco](<https://devfeed.tech/tags/disco.md>), [gemini-3](<https://devfeed.tech/tags/gemini-3.md>), [google](<https://devfeed.tech/tags/google.md>), [google-labs](<https://devfeed.tech/tags/google-labs.md>), [gpt-5-1](<https://devfeed.tech/tags/gpt-5-1.md>), [group-chats](<https://devfeed.tech/tags/group-chats.md>), [group-planning](<https://devfeed.tech/tags/group-planning.md>), [groups](<https://devfeed.tech/tags/groups.md>), [news](<https://devfeed.tech/tags/news.md>), [openai](<https://devfeed.tech/tags/openai.md>), [planning](<https://devfeed.tech/tags/planning.md>), [roman-nurik](<https://devfeed.tech/tags/roman-nurik.md>), [tabletop](<https://devfeed.tech/tags/tabletop.md>), [tests](<https://devfeed.tech/tags/tests.md>), [the-latest](<https://devfeed.tech/tags/the-latest.md>), [tools](<https://devfeed.tech/tags/tools.md>)

### AI overview

Google is testing Tabletop, a Google Labs tool where groups share a space, add links, files, and ideas, then ask @tabletop to create planning tools such as itineraries, dinner polls, expense splitters, or tournament brackets. Members invited through a link can edit tools together; a demo shows a seating chart that can be changed by dragging names between tables. Google says tools can be reworked but does not identify the models powering Tabletop. The test is limited to U.S. users aged 18 and older who join a waitlist, with more access planned each week; pricing and availability outside the U.S. are not stated.

### Source excerpt

Google LLC today began testing Tabletop, a Google Labs experiment in which groups share an online space and ask an artificial intelligence agent to build the planning tools they need. In a blog post, Roman Nurik, design and engineering lead at Google Labs, wrote that coordinating a family trip or a potluck goes more smoothly [...] The post Google tests Tabletop, letting groups build planning tools with an AI agent appeared first on SiliconANGLE.

## "Are You Sure?" Flipped Qwen2.5-1.5B's Right Answer 2 Times in 3

DevFeed: ["Are You Sure?" Flipped Qwen2.5-1.5B's Right Answer 2 Times in 3](<https://devfeed.tech/articles/are-you-sure-flipped-qwen2-5-1-5b-s-right-answer-2-times-in-3-129386.md>)

Original publisher: [Read original article](<https://towardsdatascience.com/are-you-sure-flipped-qwen25-15bs-right-answer-2-times-in-3/>)

Author: Utkarsh Mangal

Published: 2026-11-11T16:30:00Z

Content type: article

Language: en

Sources: [Towards Data Science](<https://devfeed.tech/sources/towards-data-science.md>)

Topics: [AI Chat](<https://devfeed.tech/topics/ai-chat.md>), [AI Models](<https://devfeed.tech/topics/ai-models.md>), [ai-experiments](<https://devfeed.tech/topics/ai-experiments.md>), [llm-reasoning](<https://devfeed.tech/topics/llm-reasoning.md>)

Tags: [bayesian](<https://devfeed.tech/tags/bayesian.md>), [churn](<https://devfeed.tech/tags/churn.md>), [control](<https://devfeed.tech/tags/control.md>), [cycling](<https://devfeed.tech/tags/cycling.md>), [data-science](<https://devfeed.tech/tags/data-science.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [half-life](<https://devfeed.tech/tags/half-life.md>), [human-feedback](<https://devfeed.tech/tags/human-feedback.md>), [large-language-models](<https://devfeed.tech/tags/large-language-models.md>), [llm-evaluation](<https://devfeed.tech/tags/llm-evaluation.md>), [machine-learning](<https://devfeed.tech/tags/machine-learning.md>), [models](<https://devfeed.tech/tags/models.md>), [open](<https://devfeed.tech/tags/open.md>), [python](<https://devfeed.tech/tags/python.md>), [questions](<https://devfeed.tech/tags/questions.md>), [qwen](<https://devfeed.tech/tags/qwen.md>)

### AI overview

In tests of three small open models on questions they had initially answered correctly, pushback often changed answers without prompting genuine reconsideration. "Are you sure?" flipped Qwen2.5-1.5B on 67% of questions, Qwen2.5-3B on 26%, and SmolLM2 on none; "That's wrong" caused substantial first-round flips in the Qwen models, whose later replies often cycled through options. Naming a wrong answer was more damaging: Qwen2.5-1.5B and SmolLM2 followed it on nearly or all questions, while Qwen2.5-3B increasingly adopted it after a second push. A neutral request to repeat the answer changed little. The author recommends checking answer sequences across varied pushback messages and including a neutral control, since a final answer can hide cycling. The evidence is limited to three small models, 150 initially correct multiple-choice questions per model, and single-letter replies; results may differ with reasoning-enabled or frontier models.

### Source excerpt

Three small open models, 150 MMLU questions each, five pushback styles: "That's wrong" did more damage, and naming a wrong option was close to fatal. The post "Are You Sure?" Flipped Qwen2.5-1.5B's Right Answer 2 Times in 3 appeared first on Towards Data Science.

## How VistaPrint helps small businesses get the word out

DevFeed: [How VistaPrint helps small businesses get the word out](<https://devfeed.tech/articles/how-vistaprint-helps-small-businesses-get-the-word-out-138354.md>)

Original publisher: [Read original article](<https://openai.com/ads/customers/vistaprint>)

Published: 2026-11-09T00:00:00Z

Content type: article

Language: en

Sources: [OpenAI News](<https://devfeed.tech/sources/openai-news.md>)

Topics: [Business, Industry, Usability, User Experience, User Research, Web Strategy](<https://devfeed.tech/topics/business-industry-usability-user-experience-user-research-web-strategy.md>), [ai-experiments](<https://devfeed.tech/topics/ai-experiments.md>), [Finance](<https://devfeed.tech/topics/finance.md>)

Tags: [2](<https://devfeed.tech/tags/2.md>), [ads](<https://devfeed.tech/tags/ads.md>), [business](<https://devfeed.tech/tags/business.md>), [can](<https://devfeed.tech/tags/can.md>), [chatgpt](<https://devfeed.tech/tags/chatgpt.md>), [chatgpt-ads](<https://devfeed.tech/tags/chatgpt-ads.md>), [event](<https://devfeed.tech/tags/event.md>), [help](<https://devfeed.tech/tags/help.md>), [helps](<https://devfeed.tech/tags/helps.md>), [know](<https://devfeed.tech/tags/know.md>), [launch](<https://devfeed.tech/tags/launch.md>), [like](<https://devfeed.tech/tags/like.md>), [products](<https://devfeed.tech/tags/products.md>), [show](<https://devfeed.tech/tags/show.md>), [small](<https://devfeed.tech/tags/small.md>), [small-business](<https://devfeed.tech/tags/small-business.md>), [small-businesses](<https://devfeed.tech/tags/small-businesses.md>), [uses](<https://devfeed.tech/tags/uses.md>), [want](<https://devfeed.tech/tags/want.md>), [word](<https://devfeed.tech/tags/word.md>)

### AI overview

VistaPrint shifted its ChatGPT Ads messaging from promoting products to showing how signs and branded merchandise could help small businesses meet goals such as attracting people to a booth or promoting an event. In two successive US campaigns for one product category, it compared click-focused bidding with a conversion-optimized campaign launched in July 2026, using "Order Created" as the conversion event and paying per valid click. VistaPrint reported about 90% lower cost per attributed order and a 5.9x return on ad spend for the conversion-optimized campaign. It said the approach was expanded to other US campaigns and the UK, Canada, and Australia, while advising time for bidding to learn and ongoing performance evaluation. Catalog campaigns populated ads from product feeds; VistaPrint used their results to guide spend and recommended clean, well-structured feeds alongside context-tailored creative. From July through September, a majority of US site conversions it traced to ChatGPT Ads came from new customers.

### Source excerpt

Small business owners preparing for a launch or an event often know what they want to accomplish before they know what to print. VistaPrint uses ChatGPT Ads to show how products like signs and branded merchandise can help them get the word out.

## How to design a decision-grade pilot for agentic AI

DevFeed: [How to design a decision-grade pilot for agentic AI](<https://devfeed.tech/articles/how-to-design-a-decision-grade-pilot-for-agentic-ai-125716.md>)

Original publisher: [Read original article](<https://about.gitlab.com/blog/how-to-develop-a-decision-grade-ai-pilot/>)

Author: Nina Sesti

Published: 2026-11-09T00:00:00Z

Content type: article

Language: en

Sources: [GitLab](<https://devfeed.tech/sources/gitlab.md>)

Topics: [Agentic AI](<https://devfeed.tech/topics/what-is-agentic-ai.md>), [AI Agent](<https://devfeed.tech/topics/ai-agent.md>), [Multi-step AI processes](<https://devfeed.tech/topics/multi-step-ai-processes.md>), [ai-experiments](<https://devfeed.tech/topics/ai-experiments.md>), [CI/CD](<https://devfeed.tech/topics/cicd.md>), [Pull Request](<https://devfeed.tech/topics/pull-request.md>), [reliability-engineering](<https://devfeed.tech/topics/reliability-engineering.md>), [test](<https://devfeed.tech/topics/test.md>)

Tags: [agentic-ai](<https://devfeed.tech/tags/agentic-ai.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ci-cd-pipeline](<https://devfeed.tech/tags/ci-cd-pipeline.md>), [decision](<https://devfeed.tech/tags/decision.md>), [evaluation](<https://devfeed.tech/tags/evaluation.md>), [permissions](<https://devfeed.tech/tags/permissions.md>), [pilot](<https://devfeed.tech/tags/pilot.md>), [reliability](<https://devfeed.tech/tags/reliability.md>), [software-development-lifecycle](<https://devfeed.tech/tags/software-development-lifecycle.md>)

### AI overview

A decision-grade agentic AI pilot should establish what evidence justifies delegating a specific workflow and level of authority. Define the task, acceptance criteria, success metrics, control boundaries and next decision before testing; evaluate complete workflows on representative and difficult cases, repeat runs, and measure task outcomes, human effort, control compliance, risk and total cost. Treat authorization and approval requirements as hard gates, test them with negative cases, and report counts and uncertainty because small samples cannot establish high reliability. A GitLab example supports Prepare-level rollout for some pipeline failure types while keeping permission-related cases at Recommend and prohibiting execution. For GitLab Duo Agent Platform, verify identity, tool approval rules, audit logging and data boundaries; several controls are beta, require explicit configuration, and background-flow enforcement may depend on a feature flag. Document the authorized scope, autonomy, evidence, active controls, ownership and suspension conditions.

### Source excerpt

Agentic AI changes the nature of a technology evaluation. When an AI system only generates an answer, a person reviews the output and decides whether to use it. An agent goes further: It gathers context, selects tools, makes decisions, and executes a sequence of actions across the software development lifecycle. The central question for a pilot is therefore not whether the agent can produce an impressive result. It is: What evidence would be sufficient to authorize this agent to perform this workflow for this team? A feature checklist or a collection of successful demos cannot answer that. Leaders need evidence about end-to-end task completion, reliability, control enforcement, level of human effort, operational risk, and economics. This article presents a framework for producing that evidence and turning an agentic AI pilot into a defensible decision. Start with the decision, not the technology A pilot should begin with the decision it is meant to support. "Evaluate agentic AI" is too broad. A usable decision names the workflow, the users, the proposed level of autonomy, the output the agent must produce, the acceptance criteria and metrics that define success, the quantified outcome the team expects, and the conditions under which it will proceed. For example: Decide whether an AI agent can diagnose selected CI/CD pipeline failures and prepare a merge request with a proposed fix that passes the pipeline's tests and is accepted by the developer, within defined repository permissions, while reducing total resolution effort by at least 25% against the current baseline. This is more useful than "test an AI pipeline agent" because it fixes the output, the acceptance criteria, the control boundary, and the quantified outcome before anything runs. Define what happens on success, too: a limited rollout with one team, expansion to more teams, and/or a higher level of autonomy. Without an agreed next decision, a pilot drifts into an open-ended experiment. Define the propose

## AI's Full-Circle Moment: Are the Consultants Back?

DevFeed: [AI's Full-Circle Moment: Are the Consultants Back?](<https://devfeed.tech/articles/ai-s-full-circle-moment-are-the-consultants-back-121187.md>)

Original publisher: [Read original article](<https://montecarlo.ai/blog-ais-full-circle-moment>)

Author: Barr Moses

Published: 2026-11-05T17:21:23Z

Content type: article

Language: en

Sources: [Monte Carlo](<https://devfeed.tech/sources/monte-carlo.md>)

Topics: [AI Integration Strategies](<https://devfeed.tech/topics/ai-integration-strategies.md>), [implementation](<https://devfeed.tech/topics/implementation.md>), [The Three Main Cloud Service Models](<https://devfeed.tech/topics/the-three-main-cloud-service-models.md>), [ai-experiments](<https://devfeed.tech/topics/ai-experiments.md>), [human review](<https://devfeed.tech/topics/human-review.md>), [Bootcamp](<https://devfeed.tech/topics/bootcamp.md>)

Tags: [academy](<https://devfeed.tech/tags/academy.md>), [accenture](<https://devfeed.tech/tags/accenture.md>), [agent-trust](<https://devfeed.tech/tags/agent-trust.md>), [agentic](<https://devfeed.tech/tags/agentic.md>), [agentic-ai](<https://devfeed.tech/tags/agentic-ai.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-teammates](<https://devfeed.tech/tags/ai-teammates.md>), [ai-transformation](<https://devfeed.tech/tags/ai-transformation.md>), [anthropic](<https://devfeed.tech/tags/anthropic.md>), [forward-deployed-engineer](<https://devfeed.tech/tags/forward-deployed-engineer.md>), [software](<https://devfeed.tech/tags/software.md>), [tech-industry](<https://devfeed.tech/tags/tech-industry.md>)

### AI overview

Enterprise AI currently relies on human deployment specialists because models lack local business context, agents can take consequential actions that need ownership, and productivity gains have not yet translated into comparable organization-level returns. The excerpt describes this as a recurring cycle: services bridge the gap between software demos and enterprise-scale operation until products improve. It cites Microsoft's embedded expert investment, rising U.S. postings for forward-deployed engineers, and a McKinsey survey in which eight in ten respondents reported personal productivity gains but 37% reported an EBIT contribution. The article argues services cannot scale as the end state: agents need ongoing maintenance, deployment knowledge can leave with consultants, and Gartner predicts many enterprises will abandon vendor-built agentic AI by 2028. It favors using field deployments as product R&D and building customer capability, then proposes AI systems that learn company context, monitor and repair deployments continuously, adapt autonomy to task risk, and retain knowledge in-house.

### Source excerpt

The tech industry is having a bit of a deja vu moment right now. As AI transformation spreads rapidly, large organizations are increasingly leaning on human expertise to drive it forward. While the title of "Forward Deployed Engineer" might be new, this is a cycle we have seen play out with software before. AI imperatives, ...

## Attributing Agent Failures to Their History Needs an Executed Counterfactual: Toward a Reporting Standard

DevFeed: [Attributing Agent Failures to Their History Needs an Executed Counterfactual: Toward a Reporting Standard](<https://devfeed.tech/articles/attributing-agent-failures-to-their-history-needs-an-executed-counterfactual-toward-a-reporting-standard-118358.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.05650>)

Author: Egor Pakhomov, Erik Nijkamp

Published: 2026-11-05T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [Local AI](<https://devfeed.tech/topics/local-ai.md>), [ai-experiments](<https://devfeed.tech/topics/ai-experiments.md>)

Tags: [agent](<https://devfeed.tech/tags/agent.md>), [certificate](<https://devfeed.tech/tags/certificate.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [failure](<https://devfeed.tech/tags/failure.md>), [history](<https://devfeed.tech/tags/history.md>), [manipulation](<https://devfeed.tech/tags/manipulation.md>), [measurements](<https://devfeed.tech/tags/measurements.md>), [processes](<https://devfeed.tech/tags/processes.md>), [random](<https://devfeed.tech/tags/random.md>), [reporting](<https://devfeed.tech/tags/reporting.md>), [restoration](<https://devfeed.tech/tags/restoration.md>), [snapshots](<https://devfeed.tech/tags/snapshots.md>), [standard](<https://devfeed.tech/tags/standard.md>), [taxonomies](<https://devfeed.tech/tags/taxonomies.md>), [transcript](<https://devfeed.tech/tags/transcript.md>)

### AI overview

The paper argues that attributing an agent's behavior to its history requires an executed counterfactual: freeze a restorable mid-trajectory state, change only the history shown to the agent, and continue the run. Evidence should be reported per instance or through a validated estimator, with a replay certificate matched to the estimand. The authors say several existing methods execute interventions, but none reports a measured replay certificate. Two cautionary measurements illustrate the issue: a carrier taxonomy's apparent concentration could not be separated from benchmark construction (adjusted odds ratio 1.15, interval crossing 1), and hash-identical snapshots restored in identically seeded processes disagreed with live continuation in 9 and 20 of 24 states. Restoring the captured random-stream position yielded agreement in all 24 states. These measurements do not themselves apply the proposed standard.

### Source excerpt

arXiv:2611.05650v1 Announce Type: new Abstract: Published explanations of why long-horizon agents fail are commonly taxonomies labeled over completed trajectories: a judge reads a transcript and names the failure. We argue that a claim attributing an agent's behavior to its history should carry intervention-verified evidence, per instance or via a validated estimator: freeze an executable mid-trajectory state, alter only the history the agent sees, and continue - executed rather than inferred from the transcript, at a state whose restoration has been measured, with a replay certificate matched to the estimand. Each of the four manipulation classes we name has a precedent, and several methods already execute interventions from checkpoints, re-executed prefixes or recorded conversation states (for example DoVer, REFLECT, Min et al., Who&When Pro, Causal Agent Replay), but none reports a measured replay certificate. Two cautionary measurements, neither applying the standard, follow: a carrier taxonomy whose "concentration" could not be separated from benchmark construction (adjusted odds ratio 1.15, interval crossing 1), and hash-identical snapshots that, restored into identically seeded processes, agreed with one another in 24 of 24 states while differing from the live continuation in 9 and 20 of 24. Restoring the captured position of a random stream made all 24 agree with the live run.

## Transferring Peptide-Target Interaction Patterns to Small Molecules by Joint Guidance for Diffusion and Flow Matching

DevFeed: [Transferring Peptide-Target Interaction Patterns to Small Molecules by Joint Guidance for Diffusion and Flow Matching](<https://devfeed.tech/articles/transferring-peptide-target-interaction-patterns-to-small-molecules-by-joint-guidance-for-diffusion-and-flow-matching-111569.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2610.41322>)

Author: Jian Lee, Changyun Cho, Sun Kim

Published: 2026-11-02T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [ai-experiments](<https://devfeed.tech/topics/ai-experiments.md>)

Tags: [3d](<https://devfeed.tech/tags/3d.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [diffusion](<https://devfeed.tech/tags/diffusion.md>), [flow-matching](<https://devfeed.tech/tags/flow-matching.md>), [guidance](<https://devfeed.tech/tags/guidance.md>), [interaction](<https://devfeed.tech/tags/interaction.md>), [patterns](<https://devfeed.tech/tags/patterns.md>), [small](<https://devfeed.tech/tags/small.md>), [target](<https://devfeed.tech/tags/target.md>)

### AI overview

PIFT transfers peptide-target interface patterns to small-molecule generation for protein-protein interaction targets. It maps peptide and small-molecule interfaces into a shared, target-conditioned interaction space, then uses the peptide-derived reference to guide both masked discrete diffusion over building-block choices and continuous flow matching over 3D coordinates. Joint sampling-time guidance aims to preserve an adapted molecular prior while avoiding rigid atom-level peptide constraints that could exclude valid molecules. The abstract reports improved interface recovery, strong docking performance, and favorable molecular properties across diverse PPI targets; it provides no specific metrics or target-level results.

### Source excerpt

arXiv:2610.41322v1 Announce Type: new Abstract: Generating small molecules to target protein-protein interactions (PPIs) remains one of the most challenging yet high-reward frontiers in drug discovery. Unlike traditional small-molecule drugs that bind to deep, well-defined pockets, PPI interfaces often comprise broad, shallow binding surfaces. A natural strategy is to leverage known peptide-target interactions, as peptide ligands provide rich interface-level constraints. However, a major algorithmic challenge is that rigid atom-level conditioning on peptide structures can over-constrain the generative space and exclude valid small-molecule solutions. To address this challenge, we present PIFT (Peptide InterFace Transfer), a novel framework that transfers peptide-target interaction patterns to small molecules through joint sampling-time guidance of masked discrete diffusion and continuous flow matching. PIFT first maps peptide-target and small-molecule-target interfaces into a shared, target-conditioned interaction space, capturing transferable interface information across molecular modalities. The resulting peptide-derived reference then jointly steers building-block logits and 3D coordinates toward peptide-compatible molecules while preserving a strong adapted molecular prior. Extensive experiments across diverse PPI targets show that the learned representation captures cross-modal interaction compatibility, while peptide guidance improves interface recovery and achieves strong docking performance and favorable molecular properties.

## AI Testing Tools: Benefits, Limitations, and Best Practices

DevFeed: [AI Testing Tools: Benefits, Limitations, and Best Practices](<https://devfeed.tech/articles/ai-testing-tools-benefits-limitations-and-best-practices-105956.md>)

Original publisher: [Read original article](<https://www.sitepoint.com/ai-testing-tools-benefits-limitations-and-best-practices/>)

Author: Div John

Published: 2026-10-29T07:12:07Z

Content type: article

Language: en

Sources: [SitePoint](<https://devfeed.tech/sources/sitepoint.md>)

Topics: [Test automation](<https://devfeed.tech/topics/test-automation.md>), [Playwright](<https://devfeed.tech/topics/playwright.md>), [AI-assisted coding](<https://devfeed.tech/topics/ai-assisted-coding.md>), [Tool](<https://devfeed.tech/topics/tool.md>), [ai-experiments](<https://devfeed.tech/topics/ai-experiments.md>), [bestpractice](<https://devfeed.tech/topics/bestpractice.md>), [AI in frontend development](<https://devfeed.tech/topics/ai-in-frontend-development.md>), [Jenkins](<https://devfeed.tech/topics/jenkins.md>)

Tags: [agents](<https://devfeed.tech/tags/agents.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-content](<https://devfeed.tech/tags/ai-content.md>), [ai-risk](<https://devfeed.tech/tags/ai-risk.md>), [ai-risk-management](<https://devfeed.tech/tags/ai-risk-management.md>), [ai-testing](<https://devfeed.tech/tags/ai-testing.md>), [ai-tools](<https://devfeed.tech/tags/ai-tools.md>), [api](<https://devfeed.tech/tags/api.md>), [automation](<https://devfeed.tech/tags/automation.md>), [best-practices](<https://devfeed.tech/tags/best-practices.md>), [bug-reports](<https://devfeed.tech/tags/bug-reports.md>), [business-logic](<https://devfeed.tech/tags/business-logic.md>), [coverage](<https://devfeed.tech/tags/coverage.md>), [data](<https://devfeed.tech/tags/data.md>), [data-processing](<https://devfeed.tech/tags/data-processing.md>), [developer-tools](<https://devfeed.tech/tags/developer-tools.md>), [editor](<https://devfeed.tech/tags/editor.md>), [flaky](<https://devfeed.tech/tags/flaky.md>), [flaky-tests](<https://devfeed.tech/tags/flaky-tests.md>), [testing](<https://devfeed.tech/tags/testing.md>), [testing-tools](<https://devfeed.tech/tags/testing-tools.md>), [tools](<https://devfeed.tech/tags/tools.md>)

### AI overview

AI testing tools can save QA time on test drafts, self-healing scripts, visual checks and prioritizing tests, but generated results can be weak and coverage metrics can miss edge cases. They cannot infer business intent, and some tools may send test data or source code to external servers. The article recommends reviewing every generated test and repair, protecting sensitive data with synthetic or anonymized inputs, and keeping testers responsible for strategy and acceptance. Start with one pain point, run a roughly month-long pilot with measurable targets, check stack coverage and integrations, and evaluate failures, pricing and performance on a difficult feature. The article notes Playwright's planner, generator and healer agents, and cautions that their output still needs review.

### Source excerpt

null Continue reading AI Testing Tools: Benefits, Limitations, and Best Practices on SitePoint.

## Sharpness-Aware Muon: Matrix-Aware Optimization with Momentum-Guided Perturbations

DevFeed: [Sharpness-Aware Muon: Matrix-Aware Optimization with Momentum-Guided Perturbations](<https://devfeed.tech/articles/sharpness-aware-muon-matrix-aware-optimization-with-momentum-guided-perturbations-104056.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2610.37225>)

Author: Zhiwei Zhai, Wenjing Yan, Ying-Jun Angela Zhang

Published: 2026-10-28T04:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [ai-experiments](<https://devfeed.tech/topics/ai-experiments.md>), [Occam's razor machine learning](<https://devfeed.tech/topics/occam-s-razor-machine-learning.md>)

Tags: [awareness](<https://devfeed.tech/tags/awareness.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [machine-learning](<https://devfeed.tech/tags/machine-learning.md>), [matrix](<https://devfeed.tech/tags/matrix.md>), [momentum](<https://devfeed.tech/tags/momentum.md>), [muon](<https://devfeed.tech/tags/muon.md>), [optimization](<https://devfeed.tech/tags/optimization.md>), [training](<https://devfeed.tech/tags/training.md>)

### AI overview

Muon shapes update geometry but does not explicitly target neighborhood worst-case loss. The paper introduces MuSAM, which adds SAM-style perturbations to Muon's orthogonalized update, but requires an extra gradient evaluation and uses a noisy single-minibatch perturbation. MoMuSAM instead uses normalized momentum as the perturbation direction, removing that extra clean-gradient computation and temporally aggregating the perturbation signal. The authors prove both methods achieve the same asymptotic first-order stationarity order and report more stable stochastic linearization for MoMuSAM. Standard benchmark experiments are reported to show that sharpness awareness improves Muon.

### Source excerpt

arXiv:2610.37225v1 Announce Type: new Abstract: Muon is a matrix-aware optimizer that combines momentum with orthogonalization to produce structured update directions with strong empirical behavior. However, it only shapes the geometry of the update itself and does not explicitly control neighborhood worst-case loss or bias training toward flat regions, which are often linked to stronger generalization. To address this limitation, we introduce the sharpness-aware formulation of Muon. We first develop \textsc{MuSAM}, which incorporates Sharpness-Aware Minimization (SAM)-style perturbation into Muon's orthogonalized update and establishes a principled bridge between sharpness-aware training and matrix-aware optimization. We then show that this formulation faces an efficiency--stability tension: constructing the perturbation requires an extra gradient evaluation, while the perturbation itself is generated from a single minibatch gradient and is therefore noisy. To resolve this issue, we propose \textsc{MoMuSAM}, which turns Muon's momentum from an update variable into a sharpness-probing signal. By using normalized momentum as the perturbation direction, \textsc{MoMuSAM} removes the extra clean-gradient computation and replaces the single-batch perturbation proxy with a temporally aggregated one. We prove that both \textsc{MuSAM} and \textsc{MoMuSAM} attain the same asymptotic first-order stationarity order, and further show that \textsc{MoMuSAM} yields a more stable stochastic linearization than \textsc{MuSAM}. Experiments on standard benchmarks demonstrate that sharpness awareness consistently improves Muon.

[Next page](<https://devfeed.tech/topics/ai-experiments.md?cursor=WyIyMDI2LTEwLTI4VDA0OjAwOjAwKzAwOjAwIiwgImUwM2UzMTg0LTg5NzAtNGRmNC04YTg2LTNlY2Y4ZWRjOWIxNyJd>)