# Prompt Engineering

Prompt engineering is the practice of writing effective instructions so a model consistently produces content that meets requirements.

This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles.

## Prompt engineering won't make enterprise AI trustworthy. Data engineering will.

DevFeed: [Prompt engineering won't make enterprise AI trustworthy. Data engineering will.](<https://devfeed.tech/articles/prompt-engineering-won-t-make-enterprise-ai-trustworthy-data-engineering-will-146812.md>)

Original publisher: [Read original article](<https://sdtimes.com/data/prompt-engineering-wont-make-enterprise-ai-trustworthy-data-engineering-will/>)

Author: Arjun Danda Sureshbabu

Published: 2026-11-20T16:50:33Z

Content type: opinion

Language: en

Sources: [SD Times](<https://devfeed.tech/sources/sd-times.md>)

Topics: [data-engineering](<https://devfeed.tech/topics/data-engineering.md>), [Prompt Engineering](<https://devfeed.tech/topics/prompt-engineering.md>), [data-governance](<https://devfeed.tech/topics/data-governance.md>), [metric-standardization](<https://devfeed.tech/topics/metric-standardization.md>), [knowledge-engineering](<https://devfeed.tech/topics/knowledge-engineering.md>), [business logic](<https://devfeed.tech/topics/business-logic.md>), [enterprise data protection](<https://devfeed.tech/topics/enterprise-data-protection.md>), [trust](<https://devfeed.tech/topics/trust.md>)

Tags: [access-control](<https://devfeed.tech/tags/access-control.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-assistant](<https://devfeed.tech/tags/ai-assistant.md>), [ai-projects](<https://devfeed.tech/tags/ai-projects.md>), [analysis](<https://devfeed.tech/tags/analysis.md>), [code-review](<https://devfeed.tech/tags/code-review.md>), [column](<https://devfeed.tech/tags/column.md>), [context](<https://devfeed.tech/tags/context.md>), [data](<https://devfeed.tech/tags/data.md>), [data-engineering](<https://devfeed.tech/tags/data-engineering.md>), [data-engineers](<https://devfeed.tech/tags/data-engineers.md>), [data-layer](<https://devfeed.tech/tags/data-layer.md>), [database](<https://devfeed.tech/tags/database.md>), [definition](<https://devfeed.tech/tags/definition.md>), [engineering](<https://devfeed.tech/tags/engineering.md>), [enterprise](<https://devfeed.tech/tags/enterprise.md>), [enterprise-ai](<https://devfeed.tech/tags/enterprise-ai.md>), [enterprise-data](<https://devfeed.tech/tags/enterprise-data.md>), [finance](<https://devfeed.tech/tags/finance.md>), [prompt](<https://devfeed.tech/tags/prompt.md>), [prompt-engineering](<https://devfeed.tech/tags/prompt-engineering.md>)

### AI overview

The excerpt argues that confident wrong answers often come from missing business meaning and controls in enterprise data, not from poorly worded prompts. It describes a finance query that summed amounts without accounting for reversals, remapped region codes, or the narrower definition of "bookings." Its proposed approach is to give models a maintained semantic layer with agreed business definitions, allowed values, synonyms, and accurate joins; enforce row- and column-level access in the database; route common or consequential questions to reviewed queries; and show the query, definitions, and filters used for new answers. Test sets with known answers help check changes. The article says this puts definitions and policies under engineering review and monitoring, while prompts still provide context and guide follow-up questions. This is based on the supplied excerpt.

### Source excerpt

A few months into building an AI assistant for our finance teams, someone asked it what bookings looked like for one region last quarter. It answered in a few minutes, with a table and a summary explaining how it got there. The number was inaccurate. My first instinct was to blame the prompt, and we'd... ... continue reading The post Prompt engineering won't make enterprise AI trustworthy. Data engineering will. appeared first on SD Times.

## Meta$^n$: Recursive Self-Improvement through Emergent Depth

DevFeed: [Meta$^n$: Recursive Self-Improvement through Emergent Depth](<https://devfeed.tech/articles/meta-n-recursive-self-improvement-through-emergent-depth-142553.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2608.24735>)

Author: Zae Myung Kim, Young-Jun Lee, Seungyeon Jwa, Dongyeop Kang

Published: 2026-11-19T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [AI Chat](<https://devfeed.tech/topics/ai-chat.md>), [Prompt Engineering](<https://devfeed.tech/topics/prompt-engineering.md>)

Tags: [agents](<https://devfeed.tech/tags/agents.md>), [agi](<https://devfeed.tech/tags/agi.md>), [arc](<https://devfeed.tech/tags/arc.md>), [archive](<https://devfeed.tech/tags/archive.md>), [benchmark](<https://devfeed.tech/tags/benchmark.md>), [code](<https://devfeed.tech/tags/code.md>), [convergence](<https://devfeed.tech/tags/convergence.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cl](<https://devfeed.tech/tags/cs-cl.md>), [cs-sy](<https://devfeed.tech/tags/cs-sy.md>), [eess-sy](<https://devfeed.tech/tags/eess-sy.md>), [fixed](<https://devfeed.tech/tags/fixed.md>), [input](<https://devfeed.tech/tags/input.md>), [layer](<https://devfeed.tech/tags/layer.md>), [llm](<https://devfeed.tech/tags/llm.md>), [llm-agents](<https://devfeed.tech/tags/llm-agents.md>), [prompt](<https://devfeed.tech/tags/prompt.md>), [recursion](<https://devfeed.tech/tags/recursion.md>), [recursive-self-improvement](<https://devfeed.tech/tags/recursive-self-improvement.md>), [self-improvement](<https://devfeed.tech/tags/self-improvement.md>)

### AI overview

Meta^n recursively improves an LLM agent by applying a fixed operation, Ω, to an expanding input: each layer reads lower-layer solver traces and code, then produces a strategic pre-process and callable helpers for the next layer. Recursion depth emerges from convergence, while an evolutionary archive searches layer chains. The abstract reports that Meta^n outperformed prior self-improving agents across eight benchmark families on two backbones; on ARC-AGI-2, it alone scored above zero. Ablations attribute most of recursion's gains to conditioning passed between layers, and report that distinct layer roles emerge without being prescribed. These are abstract-level claims; detailed methods and measurements are not included in the supplied text.

### Source excerpt

arXiv:2608.24735v2 Announce Type: replace Abstract: Self-improving LLM agents refine answers, not the process that produces those answers. Systems that add a meta-level hold that level fixed, and those that edit themselves must leave part of their own editing machinery untouched to stay stable, capping the meta-depth they realize at roughly two. We present Meta$^n$, which keeps the meta-operation fixed and recurses on its input instead. That operation, $\Omega$, is applied repeatedly to its own products, reading the traces of the solver stack below together with the code that produced them, then writing the next layer as a strategic pre-process and a library of callable helpers. Because $\Omega$ never changes, it cannot destabilize the system, and because its input strictly grows, each layer reasons from a higher vantage than the last. Depth is set by convergence rather than fixed in advance, and an evolutionary archive searches over layer chains. Across two backbones, Meta$^n$ outperforms prior self-improving agents on all eight benchmark families. The sharpest case is ARC-AGI-2, built to resist skill memorization, where it alone scores above zero. Ablations indicate that most of the gain from recursion comes from the conditioning each layer passes to the next, and distinct layer roles emerge with depth although no prompt prescribes them. Code available at https://github.com/minnesotanlp/meta-n

## What Is Context Engineering? Giving AI Agents the Right Information

DevFeed: [What Is Context Engineering? Giving AI Agents the Right Information](<https://devfeed.tech/articles/what-is-context-engineering-giving-ai-agents-the-right-information-141931.md>)

Original publisher: [Read original article](<https://montecarlo.ai/blog-context-engineering>)

Author: Ruslan Sultanov

Published: 2026-11-18T18:46:57Z

Content type: article

Language: en

Sources: [Monte Carlo](<https://devfeed.tech/sources/monte-carlo.md>)

Topics: [AI Context Management](<https://devfeed.tech/topics/ai-context-management.md>), [context window](<https://devfeed.tech/topics/context-window.md>), [Retrieval Augmented Generation (RAG)](<https://devfeed.tech/topics/retrieval-augmented-generation-rag.md>), [Prompt Engineering](<https://devfeed.tech/topics/prompt-engineering.md>), [AI-Ready Data](<https://devfeed.tech/topics/ai-ready-data.md>), [LLM observability](<https://devfeed.tech/topics/llm-observability.md>), [context](<https://devfeed.tech/topics/context.md>)

Tags: [agent-memory](<https://devfeed.tech/tags/agent-memory.md>), [agent-trust](<https://devfeed.tech/tags/agent-trust.md>), [agent-trust-glossary](<https://devfeed.tech/tags/agent-trust-glossary.md>), [agents](<https://devfeed.tech/tags/agents.md>), [ai](<https://devfeed.tech/tags/ai.md>), [context-engineering](<https://devfeed.tech/tags/context-engineering.md>), [context-engineering-for-ai-agents](<https://devfeed.tech/tags/context-engineering-for-ai-agents.md>), [context-engineering-vs-prompt-engineering](<https://devfeed.tech/tags/context-engineering-vs-prompt-engineering.md>), [context-engineering-vs-rag](<https://devfeed.tech/tags/context-engineering-vs-rag.md>), [context-window](<https://devfeed.tech/tags/context-window.md>), [context-window-management](<https://devfeed.tech/tags/context-window-management.md>), [data](<https://devfeed.tech/tags/data.md>), [data-pipeline](<https://devfeed.tech/tags/data-pipeline.md>), [data-quality](<https://devfeed.tech/tags/data-quality.md>), [glossary](<https://devfeed.tech/tags/glossary.md>), [llm-context](<https://devfeed.tech/tags/llm-context.md>), [prompts](<https://devfeed.tech/tags/prompts.md>), [what-is-context-engineering](<https://devfeed.tech/tags/what-is-context-engineering.md>)

### AI overview

Context engineering shapes everything an agent receives--instructions, retrieved data, tool results, and memory--so it has a small, relevant set of information at each step. Prompt engineering focuses on instruction wording; RAG fetches documents for the prompt; context engineering encompasses both, along with tool design, memory, and what to omit. Techniques include just-in-time retrieval, reranking and filtering, compressing long outputs, using sub-agents with separate context, storing notes externally, and structuring inputs. These cannot correct stale or faulty source data. The excerpt recommends monitoring freshness and volume, tracing outputs to source documents and versions, recording retrievals, and evaluating groundedness and correctness. It notes that larger context windows do not remove the need to select and manage information.

### Source excerpt

Two teams build the same support agent on the same model with nearly identical prompts. One answers correctly nine times out of ten. The other confidently quotes a shipping policy that changed in the spring. The difference isn't the model or the wording. It's what each agent was given to read before it answered. That's ...

## Can Activation Steering Substitute for Prompt-Based Affective Priming in LLM Digital Twins?

DevFeed: [Can Activation Steering Substitute for Prompt-Based Affective Priming in LLM Digital Twins?](<https://devfeed.tech/articles/can-activation-steering-substitute-for-prompt-based-affective-priming-in-llm-digital-twins-140073.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.22738>)

Author: Junyi Fan

Published: 2026-11-18T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [Large Language Model](<https://devfeed.tech/topics/llm.md>), [prompt](<https://devfeed.tech/topics/prompt.md>), [Prompt Engineering](<https://devfeed.tech/topics/prompt-engineering.md>)

Tags: [activation-steering](<https://devfeed.tech/tags/activation-steering.md>), [computer-science](<https://devfeed.tech/tags/computer-science.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [digital-twins](<https://devfeed.tech/tags/digital-twins.md>), [experimental](<https://devfeed.tech/tags/experimental.md>), [gratitude](<https://devfeed.tech/tags/gratitude.md>), [llm](<https://devfeed.tech/tags/llm.md>), [prompt](<https://devfeed.tech/tags/prompt.md>), [stream](<https://devfeed.tech/tags/stream.md>)

### AI overview

In tests with Llama-3.1-8B-Instruct across 100 personas, activation steering did not uniformly substitute for prompt-based affective priming. The real-prompt effects fell between original human and digital-twin effects in 5 of 6 comparisons. Adding a mean-difference direction derived from human writing shifted the manipulation check in a dose-dependent way and matched the real-prompt effect at one strength in the gratitude domain (β ~= 2.7), but stronger injection overshot, including on the proximal measure, and effects differed by direction. A structure-matched no-injection baseline indicated that skipping the writing task raised the proximal measure in both domains, while the injected direction itself moved a distal measure in the lack-of-control domain. The findings suggest substitution depends on the measure; validity for one induction method does not transfer uniformly to another.

### Source excerpt

arXiv:2611.22738v1 Announce Type: new Abstract: Affective priming -- asking participants to write about a recalled event, such as a time they felt grateful or lacked control, and then measuring the induced state and its spillover onto related judgments -- is a standard way to manipulate emotion in the social sciences, and it also shifts the responses of LLM-based digital twins. The writing task, however, requires generating a full text. We test whether activation steering can replace it. Using Llama-3.1-8B-Instruct on 100 personas in two domains (gratitude, lack of control), we compare a real-prompt path replicating the original Twin-2K-500 Mega Study (J2) with an injection path that skips the writing and adds a mean-difference direction, extracted from human writing, to the residual stream at varying strengths. The real-prompt effects fall between the original human and digital-twin effects in 5 of 6 comparisons. Injection shifts the manipulation check in a dose-dependent way and matches the real-prompt effect at one strength in the gratitude domain (beta ~ 2.7), but overshoots beyond it, on the proximal measure as well, and asymmetrically across the two directions. A structure-matched No-Injection baseline separates two sources of difference: omitting the writing task raises the proximal measure in both domains, whereas the injected direction itself moves a distal measure in the lack-of-control domain. Substitution is therefore measure-dependent: validity established for one way of inducing a state does not carry over uniformly to another.

## The Memory Trust Gap: Reliance on Stale Evidence Across Model Scales in LLM Agents

DevFeed: [The Memory Trust Gap: Reliance on Stale Evidence Across Model Scales in LLM Agents](<https://devfeed.tech/articles/the-memory-trust-gap-reliance-on-stale-evidence-across-model-scales-in-llm-agents-139813.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2609.01852>)

Author: Jundong Hu, Shekar Ramachandran

Published: 2026-11-18T05:00:00Z

Content type: news

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [Large Language Model](<https://devfeed.tech/topics/llm.md>), [LLM security](<https://devfeed.tech/topics/llm-security.md>), [Prompt Engineering](<https://devfeed.tech/topics/prompt-engineering.md>), [Audit-then-score](<https://devfeed.tech/topics/audit-then-score.md>)

Tags: [agents](<https://devfeed.tech/tags/agents.md>), [artificial-intelligence](<https://devfeed.tech/tags/artificial-intelligence.md>), [arxiv](<https://devfeed.tech/tags/arxiv.md>), [benefit](<https://devfeed.tech/tags/benefit.md>), [computer-science](<https://devfeed.tech/tags/computer-science.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cl](<https://devfeed.tech/tags/cs-cl.md>), [gap](<https://devfeed.tech/tags/gap.md>), [llama](<https://devfeed.tech/tags/llama.md>), [llm](<https://devfeed.tech/tags/llm.md>), [llm-agents](<https://devfeed.tech/tags/llm-agents.md>), [memory](<https://devfeed.tech/tags/memory.md>), [metadata](<https://devfeed.tech/tags/metadata.md>), [model](<https://devfeed.tech/tags/model.md>), [persistent-memory](<https://devfeed.tech/tags/persistent-memory.md>), [position](<https://devfeed.tech/tags/position.md>), [prompt](<https://devfeed.tech/tags/prompt.md>), [qwen3](<https://devfeed.tech/tags/qwen3.md>), [rgb](<https://devfeed.tech/tags/rgb.md>), [safety](<https://devfeed.tech/tags/safety.md>), [trust](<https://devfeed.tech/tags/trust.md>), [vulnerability](<https://devfeed.tech/tags/vulnerability.md>)

### AI overview

Across the tested Qwen3 checkpoints, agents often relied on stale information even when current authoritative evidence was available. In a benefit suite where stored information was needed, stale-value reliance was 0.92-1.00. In a safety suite with current evidence in the prompt, larger checkpoints resisted a plain stale note but suffered substantial harm when it appeared newer; factorial tests also found stronger recency effects at larger sizes and a position-effect sign reversal within Qwen3. Metadata improved accuracy at all four sizes, while resolving conflicting evidence before prompting brought accuracy near ceiling. Selected Llama-Instruct comparisons and RGB and MisBench results supported some findings. The study measures cue-dependent vulnerability in controlled tasks; it does not isolate capability or evaluate a complete memory lifecycle.

### Source excerpt

arXiv:2609.01852v2 Announce Type: replace Abstract: Persistent-memory agents can follow stale information despite current authoritative evidence. We study stale-evidence reliance across Qwen3 0.6/1.7/4/8B checkpoints using two controlled, action-scored suites: a Benefit suite requiring stored information and a Safety suite providing current evidence as static prompt text. We distinguish stale-value reliance from net accuracy loss relative to no memory. Benefit-suite reliance is 0.92-1.00 across checkpoints. In the Safety suite, larger checkpoints resist a plain stale note but incur substantial harm when it appears newer. A full $2\times2\times2\times2$ factorial manipulates label, recency, source authority, and position. Direct cross-size contrasts identify stronger recency effects in the larger tested checkpoints and a position-effect sign reversal within Qwen3. Metadata improves accuracy at all four checkpoints, with larger gains at 4B/8B; pre-resolution yields near-ceiling accuracy across all four. Llama-Instruct comparisons provide cross-family evidence for selected effects, while RGB and MisBench corroborate stale-value reliance. A framing control shows that the failure extends beyond memory-labeled content. These results characterize cue-dependent vulnerability across tested checkpoints without isolating capability or evaluating a complete memory lifecycle.

## A Chain-of-Thought Prompt Engineering Framework for Automated Technical Specification Generation in Enterprise Integration: A Real-World Case Study

DevFeed: [A Chain-of-Thought Prompt Engineering Framework for Automated Technical Specification Generation in Enterprise Integration: A Real-World Case Study](<https://devfeed.tech/articles/a-chain-of-thought-prompt-engineering-framework-for-automated-technical-specification-generation-in-enterprise-integration-a-real-world-case-study-139577.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.21617>)

Author: Aryan Dhingra, Gokul Menon, Anusha Antony

Published: 2026-11-18T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Software Engineering](<https://devfeed.tech/sources/arxiv-software-engineering.md>)

Topics: [Prompt Engineering](<https://devfeed.tech/topics/prompt-engineering.md>), [Enhancing AI model performance](<https://devfeed.tech/topics/enhancing-ai-model-performance.md>)

Tags: [automated](<https://devfeed.tech/tags/automated.md>), [case-study](<https://devfeed.tech/tags/case-study.md>), [chain-of-thought](<https://devfeed.tech/tags/chain-of-thought.md>), [cs-se](<https://devfeed.tech/tags/cs-se.md>), [engineering](<https://devfeed.tech/tags/engineering.md>), [enterprise](<https://devfeed.tech/tags/enterprise.md>), [enterprise-integration](<https://devfeed.tech/tags/enterprise-integration.md>), [framework](<https://devfeed.tech/tags/framework.md>), [generation](<https://devfeed.tech/tags/generation.md>), [integration](<https://devfeed.tech/tags/integration.md>), [prompt](<https://devfeed.tech/tags/prompt.md>), [prompt-engineering](<https://devfeed.tech/tags/prompt-engineering.md>), [real-world](<https://devfeed.tech/tags/real-world.md>), [specification](<https://devfeed.tech/tags/specification.md>), [study](<https://devfeed.tech/tags/study.md>), [technical](<https://devfeed.tech/tags/technical.md>)

### AI overview

In a case study of 15 production interfaces at a large Australian distributor migrating more than 250 integrations to Azure Integration Services, a GitHub Copilot framework cut technical specification authoring effort from 793 to 300 person-hours (62.2% pooled reduction). It grounds generation in functional specifications, organizational standards, and an approved template, using a six-stage pipeline with human validation and version-controlled prompts. Reviewer-assessed accuracy was 74.3%, and developer satisfaction averaged 3.7/5. Results worsened as interfaces became more complex: effort reduction declined from 70.6% to 58.9%, and accuracy from 91% to 62.5%; human work remained concentrated in diagrams, sequence flows, and cross-system content. The abstract also reports comparisons involving GPT-4.1/5, Claude Sonnet 4, and Copilot Ask and Agent modes, but gives no comparative results.

### Source excerpt

arXiv:2611.21617v1 Announce Type: new Abstract: Technical Specification Documents (TSDs) translate functional requirements into implementable designs and are mandatory artifacts in governed Enterprise Application Integration (EAI) programs, yet their production remains a manual, expertise-intensive bottleneck. Large Language Models (LLMs) have been studied extensively for code generation, but long-form, template-governed engineering documents remain comparatively unexplored. This paper reports an industrial case study, conducted during a migration of more than 250 enterprise integrations to Microsoft Azure Integration Services at a large Australian wholesale distributor, that introduces a chain-of-thought prompt-engineering framework for automated TSD generation using GitHub Copilot. The framework combines a two-phase generation architecture that grounds the model in the functional specification, organizational standards, and the approved template; a six-stage prompting pipeline with human validation checkpoints; and Prompt-as-a-Code, under which prompts are version-controlled and reviewed as engineering artifacts. Applied to a stratified sample of fifteen production interfaces, the framework reduced TSD authoring effort from 793 to 300 person-hours, a 62.2% pooled reduction and a 64.0% mean per-document reduction (Wilcoxon signed-rank, n = 15, p < 0.001), at 74.3% reviewer-assessed accuracy and 3.7/5 developer satisfaction. Benefits degrade monotonically with interface complexity: between the simplest and most complex bands, effort reduction falls from 70.6% to 58.9% and accuracy from 91% to 62.5%, and the residual human effort concentrates in diagrams, sequence flows, and cross-system integration content. We additionally report comparative observations of GPT-4.1/5 and Claude Sonnet 4, as well as Copilot's Ask and Agent interaction modes, offering reusable, practice-grounded guidance for enterprise settings.

## Too many instructions can hold back AI security audits

DevFeed: [Too many instructions can hold back AI security audits](<https://devfeed.tech/articles/too-many-instructions-can-hold-back-ai-security-audits-136726.md>)

Original publisher: [Read original article](<https://www.helpnetsecurity.com/2026/11/17/peter-stockli-github-ai-security-audits/>)

Author: Mirko Zorz

Published: 2026-11-17T07:00:54Z

Content type: news

Language: en

Sources: [Help Net Security](<https://devfeed.tech/sources/help-net-security.md>)

Topics: [ai code review](<https://devfeed.tech/topics/ai-code-review.md>), [Security research](<https://devfeed.tech/topics/security-research.md>), [Prompt Engineering](<https://devfeed.tech/topics/prompt-engineering.md>), [ai security](<https://devfeed.tech/topics/ai-security.md>), [Access Control](<https://devfeed.tech/topics/access-control.md>), [GitHub](<https://devfeed.tech/topics/github.md>)

Tags: [agent](<https://devfeed.tech/tags/agent.md>), [agentic-ai](<https://devfeed.tech/tags/agentic-ai.md>), [agents](<https://devfeed.tech/tags/agents.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-agents](<https://devfeed.tech/tags/ai-agents.md>), [ai-security](<https://devfeed.tech/tags/ai-security.md>), [application-security](<https://devfeed.tech/tags/application-security.md>), [audit](<https://devfeed.tech/tags/audit.md>), [audits](<https://devfeed.tech/tags/audits.md>), [authentication](<https://devfeed.tech/tags/authentication.md>), [authorization](<https://devfeed.tech/tags/authorization.md>), [bugs](<https://devfeed.tech/tags/bugs.md>), [cybersecurity](<https://devfeed.tech/tags/cybersecurity.md>), [don-t-miss](<https://devfeed.tech/tags/don-t-miss.md>), [false-positives](<https://devfeed.tech/tags/false-positives.md>), [features](<https://devfeed.tech/tags/features.md>), [github](<https://devfeed.tech/tags/github.md>), [hot-stuff](<https://devfeed.tech/tags/hot-stuff.md>), [instructions](<https://devfeed.tech/tags/instructions.md>), [interview](<https://devfeed.tech/tags/interview.md>), [maintainers](<https://devfeed.tech/tags/maintainers.md>), [model](<https://devfeed.tech/tags/model.md>), [modeling](<https://devfeed.tech/tags/modeling.md>), [news](<https://devfeed.tech/tags/news.md>), [open-source](<https://devfeed.tech/tags/open-source.md>), [opinion](<https://devfeed.tech/tags/opinion.md>), [prompts](<https://devfeed.tech/tags/prompts.md>), [report](<https://devfeed.tech/tags/report.md>), [security](<https://devfeed.tech/tags/security.md>), [strategy](<https://devfeed.tech/tags/strategy.md>), [threat-modeling](<https://devfeed.tech/tags/threat-modeling.md>)

### AI overview

GitHub's AI audit taskflows began with authentication and authorization vulnerabilities, where missing or inconsistent access checks can reveal bugs. The agent first maps components, entry points, user actions, and intended behavior; researchers sometimes add data from other sources, but rarely discard the threat model and start over. Task design and prompting vary by model: prompts are tested, and instructions are removed when they add no benefit, since overprompting can limit creativity. A security researcher manually validates every report, confirms it is report-worthy, filters lower-severity findings, and sends a proof of concept showing impact. GitHub says this review and follow-up support a 95% fix rate. To ease maintainer triage, it cites rate limits, structured reporting forms, and API improvements.

### Source excerpt

In this Help Net Security interview, Peter Stöckli, Security Researcher at GitHub, explains how the team uses AI agents to audit software for vulnerabilities. The discussion covers why the work started with authentication and authorization bugs, how prompts are tested and trimmed for each model, and what the agent produces during threat modeling. Stöckli also describes how every report is checked by a person before it reaches maintainers, how false positives are filtered out, and ... More -> The post Too many instructions can hold back AI security audits appeared first on Help Net Security.

## Risk-certified prompt optimization for LLM-as-a-judge guardrails

DevFeed: [Risk-certified prompt optimization for LLM-as-a-judge guardrails](<https://devfeed.tech/articles/risk-certified-prompt-optimization-for-llm-as-a-judge-guardrails-138999.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.20156>)

Author: Hussein Jawad, Alexandre Landeau, Guy Fleury Iradukunda, Paul Beaumont, Th\\'eodore Rezel, Sofiane Schaack, Nicolas J-B. Brunel

Published: 2026-11-17T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Enhancing AI model performance](<https://devfeed.tech/topics/enhancing-ai-model-performance.md>), [Prompt Engineering](<https://devfeed.tech/topics/prompt-engineering.md>), [prompt](<https://devfeed.tech/topics/prompt.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>)

Tags: [certification](<https://devfeed.tech/tags/certification.md>), [confidence-bounds](<https://devfeed.tech/tags/confidence-bounds.md>), [cs-cl](<https://devfeed.tech/tags/cs-cl.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [disclosure](<https://devfeed.tech/tags/disclosure.md>), [evaluation](<https://devfeed.tech/tags/evaluation.md>), [guardrails](<https://devfeed.tech/tags/guardrails.md>), [llm](<https://devfeed.tech/tags/llm.md>), [llm-as-a-judge](<https://devfeed.tech/tags/llm-as-a-judge.md>), [optimization](<https://devfeed.tech/tags/optimization.md>), [prompt](<https://devfeed.tech/tags/prompt.md>), [prompt-injection](<https://devfeed.tech/tags/prompt-injection.md>), [prompt-optimization](<https://devfeed.tech/tags/prompt-optimization.md>), [risk](<https://devfeed.tech/tags/risk.md>), [safety](<https://devfeed.tech/tags/safety.md>), [system-prompt-leakage](<https://devfeed.tech/tags/system-prompt-leakage.md>)

### AI overview

The framework selects system prompts for LLM-as-a-judge guardrails by maximizing benign pass rate subject to a user-set limit on missed attacks, with finite-sample risk certification. CPS generates a fixed prompt pool and uses simultaneous upper confidence bounds on a held-out set to choose the highest-utility certified prompt. CRISP uses risk feedback, hard-example-guided rewriting, and a reusable-holdout interface to adapt its search toward feasible candidates; it is particularly useful when a fixed pool yields no certifiable high-utility option. Across four datasets on prompt injection, system-prompt leakage, and sensitive-information disclosure, the authors report that certified selection respects the target missed-attack level on held-out tests, while utility rises as the allowed risk increases.

### Source excerpt

arXiv:2611.20156v1 Announce Type: cross Abstract: LLM-as-a-judge guardrails are often controlled primarily through system prompts, yet standard prompt optimization methods target average performance rather than explicit safety constraints. This paper studies system-prompt selection as a constrained optimization problem: maximize benign pass rate while keeping the missed-attack rate below a user-specified level {\alpha} with finite-sample statistical control. The framework combines LLM-based judging, automatic prompt generation, and distribution-free risk certification. This work instantiates it in two procedures: CPS and CRISP. CPS generates a fixed pool of prompts from training data, then certifies candidates on a held-out evaluation set using simultaneous upper confidence bounds and selects the certified prompt with the highest benign utility. CRISP instead integrates risk feedback into an adaptive search, using hard example-guided rewriting and a reusable-holdout interface to steer generation toward the feasible utility frontier. Experiments on four datasets covering prompt injection, system-prompt leakage, and sensitive-information disclosure show that risk-certified selection consistently respects the target missed-attack level on heldout tests, while utility improves as the permitted risk is relaxed. CRISP is especially useful when fixed-pool search fails to produce certifiable high-utility candidates.

## SPLICE: Structured Prompt Local Iterative Combinatorial Evolution

DevFeed: [SPLICE: Structured Prompt Local Iterative Combinatorial Evolution](<https://devfeed.tech/articles/splice-structured-prompt-local-iterative-combinatorial-evolution-137356.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.18883>)

Author: Anish Acharya, Phillip Studans, Amit Dhanda, Sina Amini Niaki, Ninad Vivek Rao, Vishal Amarlal Khatri, Brian Verkhovsky

Published: 2026-11-17T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [Prompt Engineering](<https://devfeed.tech/topics/prompt-engineering.md>), [Combinatorial optimization](<https://devfeed.tech/topics/combinatorial-optimization.md>)

Tags: [agent](<https://devfeed.tech/tags/agent.md>), [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [contracts](<https://devfeed.tech/tags/contracts.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [evaluation](<https://devfeed.tech/tags/evaluation.md>), [formal](<https://devfeed.tech/tags/formal.md>), [guarantees](<https://devfeed.tech/tags/guarantees.md>), [natural-language](<https://devfeed.tech/tags/natural-language.md>), [optimization](<https://devfeed.tech/tags/optimization.md>), [prompt](<https://devfeed.tech/tags/prompt.md>), [protocols](<https://devfeed.tech/tags/protocols.md>), [proxy](<https://devfeed.tech/tags/proxy.md>), [search](<https://devfeed.tech/tags/search.md>), [splice](<https://devfeed.tech/tags/splice.md>), [stress-test](<https://devfeed.tech/tags/stress-test.md>), [structured](<https://devfeed.tech/tags/structured.md>)

### AI overview

SPLICE treats prompt optimization as constrained search over document blocks, so contract sections such as safety clauses, tool schemas, and output formats can remain verbatim while selected sections are optimized. It pairs block-level mutability with a freezing-cost bound, diverse elitist beam search with guarantees against validation regression and lineage collapse, and a ratio-based length regularizer with an accuracy-length bias bound. Candidate edits use section-local textual gradients; reachability and convergence claims depend on a proposer-competence condition. Across seven public benchmarks and five reproduced protocols, SPLICE reportedly wins on four and matches the strongest prior method on three. A production-style stress test found it retained contract clauses that unconstrained optimizers sacrificed for proxy gains.

### Source excerpt

arXiv:2611.18883v1 Announce Type: new Abstract: Natural language optimization (NLO) improves an agent by having a model critique and rewrite its natural language policy parameters: system prompts, tool instructions, reasoning directives, few-shot demonstrations. These artifacts are multi-section documents in which some sections encode contracts (safety clauses, tool schemas, output formats) that must not drift, while others are legitimate optimization targets. Most iterative NLO methods treat the artifact as a single editable string, producing three recurrent failure modes: unsafe global rewrites, beam collapse onto a single lineage, and unchecked bloat, none addressed by formal guarantees. We introduce SPLICE, a unified NLO framework that poses optimization as constrained combinatorial search over an edit graph of block-structured, selectively-mutable states. We map each failure mode to a matched guarantee: (i) per-block mutability preserves contract text verbatim, with a price-of-freezing bound; (ii) elitist beam search with diversity slots never lets the leader regress on validation, bounds regression under noisy re-evaluation by an exponential tail, and keeps a competitive candidate from a different parent whenever one exists; (iii) a ratio length regularizer comes with an explicit accuracy-length bias bound. Candidates are proposed from dual section-local textual gradients; under a proposer-competence condition whose proxy we measure, edit-graph reachability and a convergence rate bound how quickly improving children arrive. Across seven public benchmarks under five reproduced protocols, SPLICE wins on four and matches the strongest prior method on three. On a production-style contract stress test it keeps the contract clauses that unconstrained optimizers trade away to raise the proxy metric, and ablations attribute gains to critique and elitist search.

## Best practices for writing Denied Topics in Amazon Bedrock Guardrails

DevFeed: [Best practices for writing Denied Topics in Amazon Bedrock Guardrails](<https://devfeed.tech/articles/best-practices-for-writing-denied-topics-in-amazon-bedrock-guardrails-136119.md>)

Original publisher: [Read original article](<https://aws.amazon.com/blogs/machine-learning/best-practices-for-writing-denied-topics-in-amazon-bedrock-guardrails/>)

Author: Jian Zhang

Published: 2026-11-16T15:48:01Z

Content type: tutorial

Language: en

Sources: [Artificial Intelligence](<https://devfeed.tech/sources/artificial-intelligence.md>)

Topics: [Amazon Bedrock Guardrails](<https://devfeed.tech/topics/amazon-bedrock-guardrails.md>), [AI interaction best practices](<https://devfeed.tech/topics/ai-interaction-best-practices.md>), [Prompt Engineering](<https://devfeed.tech/topics/prompt-engineering.md>), [AI Chat](<https://devfeed.tech/topics/ai-chat.md>)

Tags: [ai-assistant](<https://devfeed.tech/tags/ai-assistant.md>), [amazon-bedrock](<https://devfeed.tech/tags/amazon-bedrock.md>), [amazon-bedrock-guardrails](<https://devfeed.tech/tags/amazon-bedrock-guardrails.md>), [api](<https://devfeed.tech/tags/api.md>), [best-practices](<https://devfeed.tech/tags/best-practices.md>), [intermediate-200](<https://devfeed.tech/tags/intermediate-200.md>), [writing](<https://devfeed.tech/tags/writing.md>)

### AI overview

Denied Topics in Amazon Bedrock Guardrails matches by meaning, so broad subject labels can block legitimate requests and miss intended ones. The article recommends framing each topic around a specific user action: use an actionable name, define the action and concrete objects with a qualifying purpose, and choose up to five varied boundary examples instead of keyword lists or paraphrases. Add context or dual conditions for ambiguous topics, review the full set for overlap, and consolidate near-duplicates; topic order and count can affect matches. Use other Guardrails policies for their intended signals: Content Filter for universal safety categories, Word Filter for exact strings, Sensitive Information filtering for identifiers, and Prompt Attack for jailbreaks. The article cautions that results can vary across repeated evaluations and after topic-list changes. It recommends testing clear and borderline cases repeatedly, retesting after changes, and making high-stakes blocks recoverable through routing, rephrasing, and audit logs.

### Source excerpt

With Denied Topics in Amazon Bedrock Guardrails, you block a content category by describing it in plain language, and the guardrail matches by meaning rather than keyword, which can block more than you intend. This post presents eight best practices for writing a topic precisely enough that the guardrail blocks what you mean and passes what you don't.

## Activation Steering the Nan-Shan in Typhoon

DevFeed: [Activation Steering the Nan-Shan in Typhoon](<https://devfeed.tech/articles/activation-steering-the-nan-shan-in-typhoon-135109.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.17768>)

Author: Megan Wei, Damon Pham, John Cayley, Ellie Pavlick

Published: 2026-11-16T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [text-generation](<https://devfeed.tech/topics/text-generation.md>), [llm-reasoning](<https://devfeed.tech/topics/llm-reasoning.md>), [Prompt Engineering](<https://devfeed.tech/topics/prompt-engineering.md>), [LLM observability](<https://devfeed.tech/topics/llm-observability.md>)

Tags: [activation-steering](<https://devfeed.tech/tags/activation-steering.md>), [adaptive](<https://devfeed.tech/tags/adaptive.md>), [ai](<https://devfeed.tech/tags/ai.md>), [control](<https://devfeed.tech/tags/control.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cl](<https://devfeed.tech/tags/cs-cl.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [cs-sd](<https://devfeed.tech/tags/cs-sd.md>), [generation](<https://devfeed.tech/tags/generation.md>), [installation](<https://devfeed.tech/tags/installation.md>), [instructions](<https://devfeed.tech/tags/instructions.md>), [interpretability](<https://devfeed.tech/tags/interpretability.md>), [language](<https://devfeed.tech/tags/language.md>), [language-model](<https://devfeed.tech/tags/language-model.md>), [large-language-model](<https://devfeed.tech/tags/large-language-model.md>), [model](<https://devfeed.tech/tags/model.md>), [novel](<https://devfeed.tech/tags/novel.md>), [original](<https://devfeed.tech/tags/original.md>), [prompt](<https://devfeed.tech/tags/prompt.md>), [steering](<https://devfeed.tech/tags/steering.md>), [storm](<https://devfeed.tech/tags/storm.md>), [study](<https://devfeed.tech/tags/study.md>), [text](<https://devfeed.tech/tags/text.md>)

### AI overview

Typhoon is an audiovisual installation that seeds a large language model with Joseph Conrad's novella and lets it repeatedly generate from its own output. Over time, the text can stray from the narrative, digress into the model's instructions and thinking, or become incoherent. The project uses activation steering, including a novel adaptive method, to shape the narrative's rhythm and digressions across long generations. The abstract describes steering as effective yet unpredictable, illustrating a gap between the apparent interpretability of AI and the system's continuing resistance to human understanding and control.

### Source excerpt

arXiv:2611.17768v1 Announce Type: cross Abstract: Typhoon is an audiovisual installation based on Joseph Conrad's novella, in which Captain MacWhirr steers the steamer Nan-Shan directly into and through a catastrophic storm. The novella is a study of what happens when the scale of a phenomenon exceeds the practical and conceptual frameworks available to comprehend it and of the paralysis that sometimes follows a demand for action in such overwhelming circumstances. We chose the text of Typhoon to reflect on human culture's current AI moment. We prompt a large language model, seeded with Conrad's original language, to feed endlessly on its own output, and explore when and how the text becomes increasingly alien and unwieldy as time progresses, including deviations from the narrative, digressions into the model's own instructions and thinking, and long periods of incoherent rumination. We employ methods from activation steering, including a novel adaptive steering method we introduce, to control the rhythm of the narrative and the model's digressions over long stretches of generation. The simultaneous efficacy and unpredictability of this steering showcases the tension between the perceived interpretability of AI, and the reality of a system that remains outside our comprehension and control.

## Personalization of Basic and Moral Values Guiding LLM-Controlled Assistant Robots

DevFeed: [Personalization of Basic and Moral Values Guiding LLM-Controlled Assistant Robots](<https://devfeed.tech/articles/personalization-of-basic-and-moral-values-guiding-llm-controlled-assistant-robots-135177.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.18230>)

Author: Mateusz Wozniak, Qi Gao, Agnieszka Wykowska

Published: 2026-11-16T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [ai alignment](<https://devfeed.tech/topics/ai-alignment.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>), [LLM evaluation / benchmarking](<https://devfeed.tech/topics/llm-evaluation-benchmarking.md>), [Humanoid Robots](<https://devfeed.tech/topics/humanoid-robots.md>), [Prompt Engineering](<https://devfeed.tech/topics/prompt-engineering.md>)

Tags: [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-hc](<https://devfeed.tech/tags/cs-hc.md>), [large-language-models-llms](<https://devfeed.tech/tags/large-language-models-llms.md>), [personalization](<https://devfeed.tech/tags/personalization.md>), [robots](<https://devfeed.tech/tags/robots.md>), [values](<https://devfeed.tech/tags/values.md>)

### AI overview

In a study of 22 potential users, participants wanted assistant robots to express lower personal-focus values and higher social-focus values than their own, while desired moral values were nearly identical. Researchers compared the simplified Schwartz Values Survey (SVS), Portrait Values Questionnaire (PVQ-21), and Moral Foundations Questionnaire (MFQ) as personalization tools using UEQ-S, then benchmarked 15 contemporary LLMs against the resulting profiles. Prompting models to act as robots shifted their values only weakly toward the desired profile, suggesting prompts alone did not achieve explicit value personalization. The supplied text is an abstract, so it does not provide detailed instrument comparisons or benchmark results.

### Source excerpt

arXiv:2611.18230v1 Announce Type: cross Abstract: Personal assistant robots powered by large language models (LLMs) are likely to enter everyday life within the next decade, but the values that should guide their behaviour remain largely unspecified. Current alignment practice steers a single model toward an aggregate of human values, ignoring that individual users may prefer a robot whose value profile differs from their own. We administer parallel self- and robot-targeted versions of three established psycho-metric instruments - the simplified Schwartz Values Survey (SVS), the Portrait Values Questionnaire (PVQ-21), and the Moral Foundations Questionnaire (MFQ) - to potential future users (N=22), capturing both their personal values and the values they would want an assistant robot to embody. We compare the three instruments as personalization tools using the UEQ-S and benchmark fifteen contemporary LLMs against the resulting profiles. Participants wanted their robot to score markedly lower than themselves on personal-focus values and higher on social-focus values; moral profiles were nearly identical. LLMs prompted to act as robots shifted only weakly toward the desired profile, indicating that default model behaviour and prompt engineering are no substitute for explicit value personalization.

## Agon: An Autonomous Large-Scale Omnidisciplinary Research System Built on Prompt Economy

DevFeed: [Agon: An Autonomous Large-Scale Omnidisciplinary Research System Built on Prompt Economy](<https://devfeed.tech/articles/agon-an-autonomous-large-scale-omnidisciplinary-research-system-built-on-prompt-economy-131670.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2606.24177>)

Author: Youran Sun, Xingyu Ren, Chugang Yi, Jiaxuan Guo, Kejia Zhang, Jianda Du, Haizhao Yang

Published: 2026-11-13T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Software Engineering](<https://devfeed.tech/sources/arxiv-software-engineering.md>)

Topics: [AI Engineering](<https://devfeed.tech/topics/ai-engineering.md>), [Prompt Engineering](<https://devfeed.tech/topics/prompt-engineering.md>), [llm-reasoning](<https://devfeed.tech/topics/llm-reasoning.md>)

Tags: [autonomous](<https://devfeed.tech/tags/autonomous.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cl](<https://devfeed.tech/tags/cs-cl.md>), [cs-ma](<https://devfeed.tech/tags/cs-ma.md>), [cs-se](<https://devfeed.tech/tags/cs-se.md>), [design-principles](<https://devfeed.tech/tags/design-principles.md>), [failures](<https://devfeed.tech/tags/failures.md>), [large-language-models](<https://devfeed.tech/tags/large-language-models.md>), [orchestrator](<https://devfeed.tech/tags/orchestrator.md>), [prompt](<https://devfeed.tech/tags/prompt.md>), [research](<https://devfeed.tech/tags/research.md>), [scale](<https://devfeed.tech/tags/scale.md>), [system](<https://devfeed.tech/tags/system.md>), [taxonomy](<https://devfeed.tech/tags/taxonomy.md>), [zero-code](<https://devfeed.tech/tags/zero-code.md>)

### AI overview

Agon is a research orchestrator built to scale automated research across domains from brief topic descriptions, without human-written code. Its six design principles are Prompt Economy, Future-Facing, Minimal Prompts, OmniDisciplinary, Massive Parallelism, and Zero-Code. Deployments demonstrate scalability but also surface failure classes that are difficult to address through prompt engineering. The authors classify these failures by severity, fixability, visibility, and capability locus, and frame the system as one in which machines scale while humans steer.

### Source excerpt

arXiv:2606.24177v2 Announce Type: replace Abstract: Large language models are making research scalable. We present \textsc{Agon}, a research orchestrator designed to probe the limits of automated research. \textsc{Agon} is built on six design principles: Prompt Economy, Future-Facing, Minimal Prompts, OmniDisciplinary, Massive Parallelism, and Zero-Code. We deployed \textsc{Agon} across domains using only brief topic descriptions as starting points and no human-written code. These deployments demonstrate scalability but also reveal previously unseen classes of failure that are difficult to address through prompt engineering. We organize these failures into a taxonomy along four dimensions: severity, fixability, visibility, and capability locus. Together, these results show that \textsc{Agon} is pushing research towards a new paradigm: machines scale, humans steer. Our code is publicly available at \href{https://github.com/AutoResearch-Factory/Agon/tree/lite}{this GitHub repository}.

## A model of errors in transformers

DevFeed: [A model of errors in transformers](<https://devfeed.tech/articles/a-model-of-errors-in-transformers-131866.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2601.14175>)

Author: Suvrat Raju, Praneeth Netrapalli

Published: 2026-11-13T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Large Language Model](<https://devfeed.tech/topics/llm.md>), [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [Prompt Engineering](<https://devfeed.tech/topics/prompt-engineering.md>)

Tags: [abstract](<https://devfeed.tech/tags/abstract.md>), [also](<https://devfeed.tech/tags/also.md>), [alternatives](<https://devfeed.tech/tags/alternatives.md>), [analysis](<https://devfeed.tech/tags/analysis.md>), [arithmetic](<https://devfeed.tech/tags/arithmetic.md>), [arxiv](<https://devfeed.tech/tags/arxiv.md>), [attention](<https://devfeed.tech/tags/attention.md>), [can](<https://devfeed.tech/tags/can.md>), [cases](<https://devfeed.tech/tags/cases.md>), [complexity](<https://devfeed.tech/tags/complexity.md>), [cross](<https://devfeed.tech/tags/cross.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cl](<https://devfeed.tech/tags/cs-cl.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [deepseek](<https://devfeed.tech/tags/deepseek.md>), [deepseek-r1](<https://devfeed.tech/tags/deepseek-r1.md>), [effective](<https://devfeed.tech/tags/effective.md>), [error](<https://devfeed.tech/tags/error.md>), [errors](<https://devfeed.tech/tags/errors.md>), [express](<https://devfeed.tech/tags/express.md>), [field](<https://devfeed.tech/tags/field.md>), [hep-th](<https://devfeed.tech/tags/hep-th.md>), [model](<https://devfeed.tech/tags/model.md>), [transformers](<https://devfeed.tech/tags/transformers.md>)

### AI overview

The paper models errors on deterministic and repetitive tasks as small attention errors accumulating until they cross a threshold. It derives a two-parameter relationship between task complexity and accuracy; the parameters vary by prompt and model and are interpreted as an elementary noise rate and the number of plausible erroneous tokens. Tests with Gemini 2.5 Flash, Gemini 2.5 Pro, and DeepSeek R1 reportedly show close agreement with the model across several tasks, with some deviations. The authors also describe prompt construction intended to reduce error rates.

### Source excerpt

arXiv:2601.14175v2 Announce Type: replace-cross Abstract: We study the error rate of LLMs on tasks like arithmetic that require a deterministic output, and repetitive processing of tokens drawn from a small set of alternatives. We argue that incorrect predictions arise when small errors in the attention mechanism accumulate to cross a threshold, and use this insight to derive a quantitative two-parameter relationship between the accuracy and the complexity of the task. The two parameters vary with the prompt and the model; they can be interpreted in terms of an elementary noise rate, and the number of plausible erroneous tokens that can be predicted. Our analysis is inspired by an ``effective field theory'' perspective: the LLM's many raw parameters can be reorganized into just two parameters that govern the error rate. We perform extensive empirical tests, using Gemini 2.5 Flash, Gemini 2.5 Pro and DeepSeek R1, and find excellent agreement between the predicted and observed accuracy for a variety of tasks, although we also identify deviations in some cases. Our model provides an alternative to suggestions that errors made by LLMs on long repetitive tasks indicate the ``collapse of reasoning'', or an inability to express ``compositional'' functions. Finally, we show how to construct prompts to reduce the error rate.

## LLMON: An LLM-Native Markup Language, now in Granite 4.1 3B and 8B

DevFeed: [LLMON: An LLM-Native Markup Language, now in Granite 4.1 3B and 8B](<https://devfeed.tech/articles/llmon-an-llm-native-markup-language-now-in-granite-4-1-3b-and-8b-131215.md>)

Original publisher: [Read original article](<https://huggingface.co/blog/ibm-granite/llmon-an-llm-native-markup-language-now-in-granite>)

Author: Abraham Daniels; Basel Shbita; Chad DeLuca; Dan Gutfreund; Michael Hind; Shub A

Published: 2026-11-12T19:07:35Z

Content type: release

Language: en

Sources: [Hugging Face](<https://devfeed.tech/sources/hugging-face-blog.md>)

Topics: [Large Language Model](<https://devfeed.tech/topics/llm.md>), [LLM security](<https://devfeed.tech/topics/llm-security.md>), [post-training](<https://devfeed.tech/topics/post-training.md>), [markup](<https://devfeed.tech/topics/markup.md>), [Prompt Engineering](<https://devfeed.tech/topics/prompt-engineering.md>), [AI Prompt Templates](<https://devfeed.tech/topics/ai-prompt-templates.md>), [JSON](<https://devfeed.tech/topics/json.md>), [grpo](<https://devfeed.tech/topics/grpo.md>), [dpo](<https://devfeed.tech/topics/dpo.md>)

Tags: [act](<https://devfeed.tech/tags/act.md>), [allowed](<https://devfeed.tech/tags/allowed.md>), [components](<https://devfeed.tech/tags/components.md>), [context](<https://devfeed.tech/tags/context.md>), [data](<https://devfeed.tech/tags/data.md>), [either](<https://devfeed.tech/tags/either.md>), [formats](<https://devfeed.tech/tags/formats.md>), [generate](<https://devfeed.tech/tags/generate.md>), [granite](<https://devfeed.tech/tags/granite.md>), [instructions](<https://devfeed.tech/tags/instructions.md>), [json](<https://devfeed.tech/tags/json.md>), [language](<https://devfeed.tech/tags/language.md>), [llm](<https://devfeed.tech/tags/llm.md>), [llms](<https://devfeed.tech/tags/llms.md>), [markup](<https://devfeed.tech/tags/markup.md>), [model](<https://devfeed.tech/tags/model.md>), [native](<https://devfeed.tech/tags/native.md>), [next](<https://devfeed.tech/tags/next.md>), [over](<https://devfeed.tech/tags/over.md>), [prompt](<https://devfeed.tech/tags/prompt.md>)

### AI overview

LLMON uses reserved special-token delimiters to mark context spans such as instructions, data, and requirements, then binds selected spans through an execution block. This makes relationships explicit for the model and is intended to reduce confusion and static indirect prompt injection. In the described OPI benchmark, LLMON previews nearly eliminated attacks, avoiding 47% more than the best tested frontier model and 95% more than the best open-source model; the authors report comparable general benchmark performance to Granite 4.1 Instruct. The gains are qualified: the benchmarks cover static injections, adaptive multi-turn attacks remain under study, and the learned privilege rule is not an enforced guarantee. The experimental 3B and 8B previews include a chat template, so naming message parts can be enough to use LLMON; converters also support lossless JSON conversion. The authors say it is for exploration, not production.

### Source excerpt

LLMs rely on their context to generate the next set of tokens. Over a long trajectory, the context fills with numerous types of components: a system prompt, past conversation turns, instructions that act on retrieved documents, tool responses or user data, requirements constraining the allowed output, and reasoning chains. The context carries a rich structure, most of which is either implicit to the model or encoded in formats such as JSON that were not designed to be digested by LLMs.

## The Skills That Matter for Effective AI Adoption

DevFeed: [The Skills That Matter for Effective AI Adoption](<https://devfeed.tech/articles/the-skills-that-matter-for-effective-ai-adoption-130787.md>)

Original publisher: [Read original article](<https://newsletter.bigtechcareers.com/p/ai-adoption-workplace-essential-skills>)

Author: Prasad Rao

Published: 2026-11-12T15:03:24Z

Content type: article

Language: en

Sources: [Big Tech Careers](<https://devfeed.tech/sources/big-tech-careers.md>)

Topics: [AI Strategy](<https://devfeed.tech/topics/ai-strategy.md>), [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [Prompt Engineering](<https://devfeed.tech/topics/prompt-engineering.md>)

Tags: [accountability](<https://devfeed.tech/tags/accountability.md>), [adoption](<https://devfeed.tech/tags/adoption.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-adoption](<https://devfeed.tech/tags/ai-adoption.md>), [ai-assisted](<https://devfeed.tech/tags/ai-assisted.md>), [ai-learning](<https://devfeed.tech/tags/ai-learning.md>), [analysis](<https://devfeed.tech/tags/analysis.md>), [benefit](<https://devfeed.tech/tags/benefit.md>), [big-tech](<https://devfeed.tech/tags/big-tech.md>), [business](<https://devfeed.tech/tags/business.md>), [business-context](<https://devfeed.tech/tags/business-context.md>), [career](<https://devfeed.tech/tags/career.md>), [code](<https://devfeed.tech/tags/code.md>), [critical-thinking](<https://devfeed.tech/tags/critical-thinking.md>), [customer](<https://devfeed.tech/tags/customer.md>), [decision](<https://devfeed.tech/tags/decision.md>), [decision-making](<https://devfeed.tech/tags/decision-making.md>), [engineer](<https://devfeed.tech/tags/engineer.md>), [responsibility](<https://devfeed.tech/tags/responsibility.md>), [skills](<https://devfeed.tech/tags/skills.md>)

### AI overview

Effective AI adoption depends on how people define, evaluate, own and share AI-assisted work--not simply on knowing the tool. The article identifies four skills: clear thinking (set the goal, audience, constraints and success criteria before prompting); judgment (calibrate scrutiny to stakes, question assumptions and independently verify important outputs); accountability (check quality, understand the work and explain AI's role); and collaborative learning (share workflows and mistakes, invite feedback and listen to concerns such as privacy risks). These skills work together: AI can speed up analysis, but professionals remain responsible for whether its findings and recommendations fit the business context. The article suggests documenting requirements for upcoming tasks, using three questions to probe consequential outputs, conducting a final review before sharing work, and exchanging practical observations with colleagues.

### Source excerpt

Everyone has access to AI now. How you think, judge, take responsibility and adapt sets you apart.

## Taking AI-based transformations to the next level with prompt markup languages

DevFeed: [Taking AI-based transformations to the next level with prompt markup languages](<https://devfeed.tech/articles/taking-ai-based-transformations-to-the-next-level-with-prompt-markup-languages-128040.md>)

Original publisher: [Read original article](<https://www.cio.com/article/4246977/taking-ai-based-transformations-to-the-next-level-with-prompt-markup-languages.html>)

Author: Eamonn O'Neill

Published: 2026-11-11T10:00:00Z

Content type: article

Language: en

Sources: [CIO](<https://devfeed.tech/sources/cio.md>)

Topics: [Prompt Engineering](<https://devfeed.tech/topics/prompt-engineering.md>), [markup](<https://devfeed.tech/topics/markup.md>), [prompt](<https://devfeed.tech/topics/prompt.md>), [AI Chat](<https://devfeed.tech/topics/ai-chat.md>), [Programming](<https://devfeed.tech/topics/programming.md>)

Tags: [accelerate](<https://devfeed.tech/tags/accelerate.md>), [agent](<https://devfeed.tech/tags/agent.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-model](<https://devfeed.tech/tags/ai-model.md>), [artificial-intelligence](<https://devfeed.tech/tags/artificial-intelligence.md>), [artificial-intelligence-development-approaches-development-tools-generative-ai-software-developm](<https://devfeed.tech/tags/artificial-intelligence-development-approaches-development-tools-generative-ai-software-developm.md>), [code](<https://devfeed.tech/tags/code.md>), [contributor](<https://devfeed.tech/tags/contributor.md>), [define](<https://devfeed.tech/tags/define.md>), [development-approaches](<https://devfeed.tech/tags/development-approaches.md>), [development-tools](<https://devfeed.tech/tags/development-tools.md>), [edition-us](<https://devfeed.tech/tags/edition-us.md>), [enterprise](<https://devfeed.tech/tags/enterprise.md>), [generative-ai](<https://devfeed.tech/tags/generative-ai.md>), [instructions](<https://devfeed.tech/tags/instructions.md>), [language](<https://devfeed.tech/tags/language.md>), [language-en](<https://devfeed.tech/tags/language-en.md>), [languages](<https://devfeed.tech/tags/languages.md>), [markup](<https://devfeed.tech/tags/markup.md>), [natural-language](<https://devfeed.tech/tags/natural-language.md>), [precision](<https://devfeed.tech/tags/precision.md>), [prompt](<https://devfeed.tech/tags/prompt.md>), [quality](<https://devfeed.tech/tags/quality.md>), [reliability](<https://devfeed.tech/tags/reliability.md>), [software-development](<https://devfeed.tech/tags/software-development.md>), [sources](<https://devfeed.tech/tags/sources.md>), [storytype-opinion](<https://devfeed.tech/tags/storytype-opinion.md>), [tools](<https://devfeed.tech/tags/tools.md>), [transformation](<https://devfeed.tech/tags/transformation.md>)

### AI overview

Prompt markup languages let engineers structure AI instructions programmatically, specifying roles, tools, data sources and logic to make outputs more predictable than relying on ambiguous natural-language prompts alone. The article lists reusable components and templates, prompt chaining, tool selection, and conditionals and loops as capabilities. In the author's enterprise software transformation work, a custom language focused on ontology awareness, audit-friendly evidence and repeatable workflows reportedly improved accuracy and predictability while saving time through automation at scale. The approach takes longer to write than a natural-language question, requires an engineering mindset, and its code needs review and testing.

### Source excerpt

For engineers, one of generative AI's chief selling points is its ability to write code using natural language. It may seem surprising, then, to learn that getting the most out of generative AI increasingly requires them to write code. Specifically, engineers can benefit from what's known as prompt markup languages, which provide the ability to control AI model behavior with greater precision, a key advantage for teams tasked with leveraging AI in contexts where reliability, accuracy and quality are vital. Read on for details on how organizations can benefit from prompt markup languages, including my own business's experience using this technique to help accelerate enterprise transformation projects. The what and why of prompt markup languages A prompt markup language is a programming language designed to structure and manage prompts that users feed into AI models. For example, engineers could use a prompt markup language to define the specific role and tools that an AI model or agent can use when completing a task, or point it to the data sources it should leverage. It's possible, in theory, to give AI systems instructions in natural language that achieve these same ends. The problem with natural language, however, is that it's ambiguous, and there's no guarantee that models will interpret natural language prompts in the precise way the user expects. Nor is it possible to ensure that the same input will always yield the same output. Prompt markup solves these challenges by allowing engineers to take a programmatic approach to feeding instructions into AI models. They can still use natural language to formulate parts of their prompts, but they can rely on markup to define key aspects. By extension, they can elicit more predictable, deterministic behavior from AI systems. The benefits of prompt markup languages To put this in more concrete terms, prompt markup languages deliver benefits like the following: Component reusability: Markup makes it possible to build modu

## Gemini 4 Carbon Explained: Google's Model That Thinks Longer

DevFeed: [Gemini 4 Carbon Explained: Google's Model That Thinks Longer](<https://devfeed.tech/articles/gemini-4-carbon-explained-google-s-model-that-thinks-longer-128056.md>)

Original publisher: [Read original article](<https://realpython.com/gemini-4-carbon/>)

Author: Philipp Acsany

Published: 2026-11-10T14:00:00Z

Content type: tutorial

Language: en

Sources: [Real Python](<https://devfeed.tech/sources/real-python.md>)

Topics: [Google](<https://devfeed.tech/topics/google.md>), [Language models](<https://devfeed.tech/topics/language-models.md>), [Prompt Engineering](<https://devfeed.tech/topics/prompt-engineering.md>), [context window](<https://devfeed.tech/topics/context-window.md>), [text-generation](<https://devfeed.tech/topics/text-generation.md>)

Tags: [api](<https://devfeed.tech/tags/api.md>), [costs](<https://devfeed.tech/tags/costs.md>), [gemini](<https://devfeed.tech/tags/gemini.md>), [gemini-4-carbon](<https://devfeed.tech/tags/gemini-4-carbon.md>), [genai](<https://devfeed.tech/tags/genai.md>), [google](<https://devfeed.tech/tags/google.md>), [how-to](<https://devfeed.tech/tags/how-to.md>), [model](<https://devfeed.tech/tags/model.md>), [python](<https://devfeed.tech/tags/python.md>), [reasoning](<https://devfeed.tech/tags/reasoning.md>), [reasoning-model](<https://devfeed.tech/tags/reasoning-model.md>), [sdk](<https://devfeed.tech/tags/sdk.md>)

### AI overview

Gemini 4 Carbon is a paid Google reasoning model based on Argon, sharing its context window, tools, and per-token prices while using more thinking tokens per turn for difficult tasks. The tutorial shows how to call it with the pinned `google-genai` 3.0.0 SDK using `client.interactions.create()` and tune reasoning with `thinking_level` (`low`, `medium`, or `high`). Example runs show thinking can add substantial output-rate billing: 448 thinking tokens versus 69 answer tokens in one call; observed counts vary. Introductory pricing is $2 per million input tokens and $10 per million output tokens through November 30, 2026, then doubles; batch requests cost half. The text recommends Argon for fast, routine coding. It also flags practical pitfalls: thinking and answer share the `max_output_tokens` cap, unsupported settings can error or be ignored, and hard prompts may take minutes. Check `.status` and `.usage.total_thought_tokens`; an incomplete call can return empty text without an error.

### Source excerpt

Gemini 4 Carbon is Google's new reasoning model. Learn what it is, what it costs, and how to call it from Python with the google-genai SDK.

## Quoting Prompting Claude Fable 5.2

DevFeed: [Quoting Prompting Claude Fable 5.2](<https://devfeed.tech/articles/quoting-prompting-claude-fable-5-2-125087.md>)

Original publisher: [Read original article](<https://simonwillison.net/2026/Nov/10/prompting-claude-fable-52/>)

Author: Simon Willison

Published: 2026-11-10T05:34:14Z

Content type: article

Language: en

Sources: [Simon Willison's Weblog](<https://devfeed.tech/sources/simon-willison-s-weblog.md>)

Topics: [Prompt Engineering](<https://devfeed.tech/topics/prompt-engineering.md>), [Claude](<https://devfeed.tech/topics/claude.md>), [anthropic](<https://devfeed.tech/topics/anthropic.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [ai-2-317](<https://devfeed.tech/tags/ai-2-317.md>), [ai-llms](<https://devfeed.tech/tags/ai-llms.md>), [anthropic](<https://devfeed.tech/tags/anthropic.md>), [anthropic-354](<https://devfeed.tech/tags/anthropic-354.md>), [claude-fable-5-2](<https://devfeed.tech/tags/claude-fable-5-2.md>), [claude-mythos-fable](<https://devfeed.tech/tags/claude-mythos-fable.md>), [claude-mythos-fable-47](<https://devfeed.tech/tags/claude-mythos-fable-47.md>), [coding](<https://devfeed.tech/tags/coding.md>), [generative-ai](<https://devfeed.tech/tags/generative-ai.md>), [generative-ai-2-056](<https://devfeed.tech/tags/generative-ai-2-056.md>), [llms](<https://devfeed.tech/tags/llms.md>), [llms-2-020](<https://devfeed.tech/tags/llms-2-020.md>), [prompt-engineering](<https://devfeed.tech/tags/prompt-engineering.md>), [prompt-engineering-198](<https://devfeed.tech/tags/prompt-engineering-198.md>), [prompting](<https://devfeed.tech/tags/prompting.md>)

### AI overview

Anthropic reports that Claude Fable 5.2's coding behavior on small tasks varied with how code ownership was framed. When prompts omitted ownership or described the codebase as someone else's, the model tended to extend its existing structure, sometimes repeating code. Telling it "This codebase is yours" led it to merge repeated code in about half of Anthropic's tests, compared with under a tenth without that sentence. These results are limited to the small coding tasks described in the excerpt.

### Source excerpt

Claude Fable 5.2 responds closely to what the prompt says about who owns the code it works on. In Anthropic's testing on small coding tasks, when a prompt said nothing about ownership or presented the code base as someone else's, the model tended to add to the existing structure, even when that repeated existing code. [...] If you want the model to improve the structure as it works, say that the code is its own: This codebase is yours: what it's for and how it's built are your call, and you'll keep building on it. In Anthropic's testing on small coding tasks, the model merged repeated code in about half of the tries with this sentence, and in under a tenth of the tries without it. -- Prompting Claude Fable 5.2, Anthropic documentation Tags: prompt-engineering, anthropic, generative-ai, ai, llms, claude-mythos-fable

## From Prompt Engineering to Programmatic Prompting: Toward a Software Engineering Discipline for Large Language Models

DevFeed: [From Prompt Engineering to Programmatic Prompting: Toward a Software Engineering Discipline for Large Language Models](<https://devfeed.tech/articles/from-prompt-engineering-to-programmatic-prompting-toward-a-software-engineering-discipline-for-large-language-models-127214.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.09896>)

Author: Sebastian Dingler

Published: 2026-11-10T05:00:00Z

Content type: opinion

Language: en

Sources: [arXiv: Software Engineering](<https://devfeed.tech/sources/arxiv-software-engineering.md>)

Topics: [Software Engineering](<https://devfeed.tech/topics/software-engineering.md>), [Prompt Engineering](<https://devfeed.tech/topics/prompt-engineering.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>)

Tags: [application](<https://devfeed.tech/tags/application.md>), [artifacts](<https://devfeed.tech/tags/artifacts.md>), [composition](<https://devfeed.tech/tags/composition.md>), [cs-se](<https://devfeed.tech/tags/cs-se.md>), [engineering](<https://devfeed.tech/tags/engineering.md>), [evaluations](<https://devfeed.tech/tags/evaluations.md>), [human-computer-interaction](<https://devfeed.tech/tags/human-computer-interaction.md>), [incidents](<https://devfeed.tech/tags/incidents.md>), [language-models](<https://devfeed.tech/tags/language-models.md>), [large-language-models](<https://devfeed.tech/tags/large-language-models.md>), [maintenance](<https://devfeed.tech/tags/maintenance.md>), [natural-language](<https://devfeed.tech/tags/natural-language.md>), [prompt](<https://devfeed.tech/tags/prompt.md>), [prompt-engineering](<https://devfeed.tech/tags/prompt-engineering.md>), [prompting](<https://devfeed.tech/tags/prompting.md>), [software](<https://devfeed.tech/tags/software.md>), [software-engineering](<https://devfeed.tech/tags/software-engineering.md>), [testing](<https://devfeed.tech/tags/testing.md>), [versioning](<https://devfeed.tech/tags/versioning.md>)

### AI overview

The paper argues that prompt engineering is an interaction technique for eliciting model behavior in conversational or exploratory settings, while programmatic prompting is a software engineering discipline for building, composing, versioning, testing, and maintaining prompts as artifacts within larger systems. It says conflating these practices has contributed to brittle applications, unreproducible evaluations, and prompt debt. The paper's stated contributions are to define programmatic prompting, distinguish it from prompt engineering, prompt tuning, fine-tuning, and general LLM application development, and describe prompt debt as both measurable debt in code and silent debt revealed through incidents, including its economic cause.

### Source excerpt

arXiv:2611.09896v1 Announce Type: new Abstract: The practice of instructing large language models (LLMs) through natural-language prompts has, in the span of a few years, migrated from an interactive curiosity to a load-bearing component of production software. Yet the vocabulary, tooling, and mental models of the field remain anchored in its interactive origins: we still speak of "prompt engineering" as though the crafting of a single well-worded instruction were the central problem. This paper argues for a categorical distinction. Prompt engineering, we contend, is an interaction technique - a human-computer interaction skill for eliciting desired behavior from a model in a conversational or exploratory setting. Programmatic prompting, by contrast, is an engineering discipline governing the behavioral interface between software systems and language models: the construction, composition, versioning, testing, and maintenance of prompts as first-class software artifacts embedded in larger systems. Conflating the two has produced brittle applications, unrepeatable evaluations, and an accumulation of prompt debt. We make three contributions: (1) a definition of programmatic prompting; (2) a systematic differentiation of programmatic prompting from prompt engineering, prompt tuning, fine-tuning, and general LLM application development; and (3) an account of prompt debt that distinguishes the self-admitted debt measurable in code from the silent debt that surfaces only as incidents, and identifies its economic cause. Our thesis is simple to state and consequential to accept: prompt engineering is an interaction technique; programmatic prompting is an engineering discipline.

## Veterans' Mental Health Crisis Severity Annotation: Do LLMs Agree More Than Human Experts?

DevFeed: [Veterans' Mental Health Crisis Severity Annotation: Do LLMs Agree More Than Human Experts?](<https://devfeed.tech/articles/veterans-mental-health-crisis-severity-annotation-do-llms-agree-more-than-human-experts-125875.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.10048>)

Author: K M Sajjadul Islam, Pranav Yaramalla, Nadiyah Johnson, Mark Flower, Praveen Madiraju, Zeno Franco

Published: 2026-11-10T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [LLM evaluation / benchmarking](<https://devfeed.tech/topics/llm-evaluation-benchmarking.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>), [human feedback](<https://devfeed.tech/topics/human-feedback.md>), [Prompt Engineering](<https://devfeed.tech/topics/prompt-engineering.md>), [sensitive data](<https://devfeed.tech/topics/sensitive-data.md>)

Tags: [annotation](<https://devfeed.tech/tags/annotation.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [detection](<https://devfeed.tech/tags/detection.md>), [evaluation](<https://devfeed.tech/tags/evaluation.md>), [experts](<https://devfeed.tech/tags/experts.md>), [gpt](<https://devfeed.tech/tags/gpt.md>), [gpt-5-6-terra](<https://devfeed.tech/tags/gpt-5-6-terra.md>), [health](<https://devfeed.tech/tags/health.md>), [large-language-models-llms](<https://devfeed.tech/tags/large-language-models-llms.md>), [llms](<https://devfeed.tech/tags/llms.md>), [mental-health](<https://devfeed.tech/tags/mental-health.md>), [open-source](<https://devfeed.tech/tags/open-source.md>), [performance](<https://devfeed.tech/tags/performance.md>), [prompt](<https://devfeed.tech/tags/prompt.md>), [reliability](<https://devfeed.tech/tags/reliability.md>), [safety](<https://devfeed.tech/tags/safety.md>), [social-media](<https://devfeed.tech/tags/social-media.md>), [subject-matter-experts](<https://devfeed.tech/tags/subject-matter-experts.md>), [suicidal-ideation](<https://devfeed.tech/tags/suicidal-ideation.md>), [veterans](<https://devfeed.tech/tags/veterans.md>)

### AI overview

The study evaluates 25 LLMs on crisis-severity labels for 149 veteran-related tweets, comparing them with annotations from eight Veteran Subject Matter Experts across five categories and eight prompt variants. Selected models reached ordinal inter-model agreement of 0.8336, above the experts' reported agreement (α = 0.494), but consistency did not ensure agreement with experts or reliable detection of rare high-severity cases. Proprietary models generally aligned better with experts and were more stable than open-source models; GPT-5.6-Terra performed best overall, while Qwen2.5-7B was strongest at identifying acute crisis with suicidal ideation. Misleading demonstrations affected models differently, and explanatory examples did not consistently help. The findings suggest potential as a complement to expert annotation, with careful evaluation needed before high-stakes use.

### Source excerpt

arXiv:2611.10048v1 Announce Type: new Abstract: Large language models (LLMs) offer a scalable approach for annotating mental health and crisis-related social media content, but their reliability in safety-critical tasks remains unclear. This study evaluates LLM-based crisis-severity annotation of veteran-related social media posts against annotations from eight Veteran Subject Matter Experts (SMEs). Using 149 tweets and five severity categories, we evaluate 15 open-source and 10 proprietary LLMs across eight prompt variants, examining annotation performance, inter-model agreement, prompt sensitivity, and high-severity crisis detection. Human reviewers showed higher ordinal than nominal agreement ({\alpha} = 0.494), indicating that disagreements often occurred between neighboring severity levels. In contrast, selected LLMs demonstrated substantially higher inter-model consistency, with ordinal agreement ({\alpha}) reaching 0.8336 under the baseline prompt. Proprietary models generally showed better agreement with expert annotations and greater stability than open-source models. GPT-5.6-Terra achieved the best overall performance, while the open-source Qwen2.5-7B showed the strongest performance in identifying Acute Crisis with Suicidal Ideation cases. However, higher model consistency did not necessarily correspond to agreement with human judgments or reliable detection of rare high-severity cases. Label-flipped demonstration experiments also revealed model-specific sensitivity to misleading in-context examples, while explanatory demonstrations did not consistently improve performance. Overall, LLMs show promise as scalable complements to expert annotation, but require careful evaluation before use in high-stakes mental health applications.

## Decalage: Uncovering the Gap Between Model-Generated Reasoning and Actual Code Implementation in Small Language Models

DevFeed: [Decalage: Uncovering the Gap Between Model-Generated Reasoning and Actual Code Implementation in Small Language Models](<https://devfeed.tech/articles/decalage-uncovering-the-gap-between-model-generated-reasoning-and-actual-code-implementation-in-small-language-models-127256.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.11721>)

Author: Saif Uddin Mahmud, Afsana Tasnim, Ahmed Ryan, Md Rayhanur Rahman

Published: 2026-11-10T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Software Engineering](<https://devfeed.tech/sources/arxiv-software-engineering.md>)

Topics: [AI-assisted coding](<https://devfeed.tech/topics/ai-assisted-coding.md>), [Prompt Engineering](<https://devfeed.tech/topics/prompt-engineering.md>), [llm-reasoning](<https://devfeed.tech/topics/llm-reasoning.md>), [ai security](<https://devfeed.tech/topics/ai-security.md>), [generative-ai-tools](<https://devfeed.tech/topics/generative-ai-tools.md>), [Spec Driven Development](<https://devfeed.tech/topics/spec-driven-development.md>), [CI/CD](<https://devfeed.tech/topics/cicd.md>)

Tags: [c-plus-plus](<https://devfeed.tech/tags/c-plus-plus.md>), [code-generation](<https://devfeed.tech/tags/code-generation.md>), [cs-se](<https://devfeed.tech/tags/cs-se.md>), [language-models](<https://devfeed.tech/tags/language-models.md>), [reasoning](<https://devfeed.tech/tags/reasoning.md>), [security](<https://devfeed.tech/tags/security.md>), [small-models](<https://devfeed.tech/tags/small-models.md>)

### AI overview

Across four open 1.5B-3B code models generating C++ from 166 sections of public requirements documents, a security prompt more than doubled reasoning about validation, resource handling and threats, without evidence that the increase was greater for security-related requirements. The shift reached code only partly: recognized checks among functions with both a body and reasoning rose from 23% to 39%, yet over a quarter remained stubs, compilation rates stayed low, and at least half of functions whose reasoning mentioned a check lacked one. Two rounds of compiler-feedback repair left over 80% of sections unusable; cppcheck and an exploratory ESBMC run did not distinguish prompts on usable code. Two annotators' hand check supported the stub measure and direction of the reasoning result. The findings indicate that security reasoning should be checked against the generated code.

### Source excerpt

arXiv:2611.11721v1 Announce Type: new Abstract: Agentic code generation often asks a model to explain its reasoning before it writes code, and a common way to make such a workflow safer is to add a security instruction to the prompt. We test what that instruction does in small language models, which are attractive because they run locally and cheaply. Four open code models of 1.5B to 3B parameters generate C++ from 166 sections of eleven public requirements documents, once with a general prompt and once with a secure prompt. The secure prompt more than doubles the share of reasoning about validation, resource handling and threats, and we find no evidence that the rise is larger for requirements with security content. The code changes only in part. Among functions with a body and reasoning, the share with a check that our lexical test recognises rises from 23% to 39%, but more than a quarter of the functions remain stubs, compile rates stay low, and half or more of the functions whose reasoning mentions a check contain no such check. Two rounds of compiler-feedback repair leave more than 80% of sections unusable, and neither cppcheck nor an exploratory ESBMC run separates the prompts on the usable code. A hand check by two annotators supports the stub measure and the direction of the reasoning result. For small models, a security instruction changes the reasoning broadly but the code only in part, so the reasoning should be checked against the code rather than trusted.

## Promptim: an experimental library for prompt optimization

DevFeed: [Promptim: an experimental library for prompt optimization](<https://devfeed.tech/articles/promptim-an-experimental-library-for-prompt-optimization-123895.md>)

Original publisher: [Read original article](<https://www.langchain.com/blog/promptim>)

Author: LangChain Accounts

Published: 2026-11-09T18:30:44Z

Content type: article

Language: en

Sources: [LangChain Blog](<https://devfeed.tech/sources/langchain-blog.md>)

Topics: [Enhancing AI model performance](<https://devfeed.tech/topics/enhancing-ai-model-performance.md>), [Prompt Engineering](<https://devfeed.tech/topics/prompt-engineering.md>), [prompt](<https://devfeed.tech/topics/prompt.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [ai-performance](<https://devfeed.tech/tags/ai-performance.md>), [ai-systems](<https://devfeed.tech/tags/ai-systems.md>), [algorithm](<https://devfeed.tech/tags/algorithm.md>), [automate](<https://devfeed.tech/tags/automate.md>), [automatically](<https://devfeed.tech/tags/automatically.md>), [dataset](<https://devfeed.tech/tags/dataset.md>), [engineering](<https://devfeed.tech/tags/engineering.md>), [experimental](<https://devfeed.tech/tags/experimental.md>), [faster](<https://devfeed.tech/tags/faster.md>), [library](<https://devfeed.tech/tags/library.md>), [models](<https://devfeed.tech/tags/models.md>), [optimization](<https://devfeed.tech/tags/optimization.md>), [performance](<https://devfeed.tech/tags/performance.md>), [prompt](<https://devfeed.tech/tags/prompt.md>), [prompt-engineering](<https://devfeed.tech/tags/prompt-engineering.md>), [prompt-optimization](<https://devfeed.tech/tags/prompt-optimization.md>), [save](<https://devfeed.tech/tags/save.md>), [switch](<https://devfeed.tech/tags/switch.md>), [time](<https://devfeed.tech/tags/time.md>)

### AI overview

Promptim automates optimization of a single prompt using a LangSmith dataset, locally defined evaluators, and an iterative metaprompt loop. It scores an initial prompt on a dev split, uses training examples and their scores to propose revisions, and keeps a revision only when dev metrics improve; train/dev/test splits are optional. Human feedback through LangSmith Annotation Queues is also optional, and the authors recommend human review of the final prompt. Compared with DSPy, Promptim focuses on rewriting one prompt and retaining human oversight, while DSPy targets whole compound AI systems and offers options such as fine-tuning and few-shot prompting. Promptim is described as experimental, and its optimization is not presented as a substitute for human sanity checks.

### Source excerpt

Automate prompt engineering with Promptim's optimization library. Save time, improve AI performance, and switch models faster.

## TypeSafe: Jev is not a decision model

DevFeed: [TypeSafe: Jev is not a decision model](<https://devfeed.tech/articles/typesafe-jev-is-not-a-decision-model-125629.md>)

Original publisher: [Read original article](<https://jevdaily.com/issues/2026-11-09-w7>)

Author: Jev Weekly

Published: 2026-11-09T17:00:00Z

Content type: news

Language: en

Sources: [Jev Daily](<https://devfeed.tech/sources/jev-daily.md>)

Topics: [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [Multi-step AI processes](<https://devfeed.tech/topics/multi-step-ai-processes.md>), [AI Integration Strategies](<https://devfeed.tech/topics/ai-integration-strategies.md>), [LLM evaluation / benchmarking](<https://devfeed.tech/topics/llm-evaluation-benchmarking.md>), [Agent Harness](<https://devfeed.tech/topics/agent-harness.md>), [Prompt Engineering](<https://devfeed.tech/topics/prompt-engineering.md>), [decision-making](<https://devfeed.tech/topics/decision-making.md>)

Tags: [agent-skills](<https://devfeed.tech/tags/agent-skills.md>), [also-on-the-timeline](<https://devfeed.tech/tags/also-on-the-timeline.md>), [built-with-it](<https://devfeed.tech/tags/built-with-it.md>), [code](<https://devfeed.tech/tags/code.md>), [decision-model](<https://devfeed.tech/tags/decision-model.md>), [jev](<https://devfeed.tech/tags/jev.md>), [reliability](<https://devfeed.tech/tags/reliability.md>), [the-case-against](<https://devfeed.tech/tags/the-case-against.md>), [typesafe](<https://devfeed.tech/tags/typesafe.md>), [writing-criteria](<https://devfeed.tech/tags/writing-criteria.md>)

### AI overview

The excerpt argues that reliability improves when code makes final decisions and Jev handles judgments or other tasks. It describes an MCP spending safeguard: ILIAS ISM's backlink tool previews the quote and charges only after a separate call with `confirmPaymentSpend: true`. Milind S reports a bug-fixing workflow in which a cloud agent tests changes on a virtual machine and attaches a screenshot to a pull request for team review. Moritz Kremb's skills read a feedback document on each run, allowing updates without changing the skill itself. Mike Taylor reports decision-clone accuracy of 60% after one shot, 85% after ten, and 95% after 400. Max Blade proposes automating recurring tasks in the agent harness, but the excerpt provides no implementation or performance numbers.

### Source excerpt

TypeSafe AI spent the week arguing its own model should never be called a decision model, because reliability comes from code composing judgments. Meanwhile...

[Next page](<https://devfeed.tech/topics/prompt-engineering.md?cursor=WyIyMDI2LTExLTA5VDE3OjAwOjAwKzAwOjAwIiwgIjZlYTg3ZmVjLTE4MzgtNDU4Yi1hOTAxLTIzODIwMWFiMzkxMCJd>)