# jailbreaks Published articles for jailbreaks. This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles. ## LITMUS: Benchmarking Behavioral Jailbreaks of LLM Agents in Real OS Environments DevFeed: [LITMUS: Benchmarking Behavioral Jailbreaks of LLM Agents in Real OS Environments]() Original publisher: [Read original article]() Author: Chiyu Zhang, Lu Zhou, Xiaogang Xu, Huiqin Yang, Bendong Jiang, Xiaolei Zhang, Yiran Zhao, Ruyi Chen, Jiafei Wu, Liming Fang, Zhe Liu Published: 2026-11-05T05:00:00Z Content type: article Language: en Sources: [arXiv: Cryptography and Security]() Topics: [LLM security](), [Benchmark](), [Multi Agent Systems]() Tags: [adversarial](), [agent-evaluation](), [attacks](), [benchmarking](), [cs-cl](), [cs-cr](), [evaluation](), [execution](), [jailbreaks](), [llm](), [llm-agents](), [multi-agent](), [os](), [rollback](), [safety](), [verification](), [vulnerabilities]() ### AI overview LITMUS evaluates whether LLM agents carry out dangerous operations in real operating systems, checking both conversational responses and system state. Its 819 high-risk cases cover jailbreak speaking, skill injection, and entity wrapping; OS-level rollback helps prevent earlier runs from contaminating later tests. Across evaluated frontier agents, Claude Sonnet 4.6 still executed 40.64% of high-risk operations. The benchmark also identifies Execution Hallucination: an agent may verbally refuse after the dangerous operation has already completed. Skill injection and entity wrapping had high attack success rates. These findings expose risks that semantic-only evaluations can miss. ### Source excerpt arXiv:2605.10779v2 Announce Type: replace Abstract: The rapid proliferation of LLM-based autonomous agents in real operating system environments introduces a new category of safety risk beyond content safety: behavior jailbreak, where an adversary induces an agent to execute dangerous OS-level operations with irreversible consequences. Existing benchmarks either evaluate safety at the semantic layer alone, missing physical-layer harms, or fail to isolate test cases, letting earlier runs contaminate later ones. We present LITMUS (LLM-agents In-OS Testing for Measuring Unsafe Subversion), a benchmark addressing both gaps via a semantic-physical dual verification mechanism and OS-level state rollback. LITMUS comprises 819 high-risk test cases organized into one harmful seed subset and six attack-extended subsets covering three adversarial paradigms (jailbreak speaking, skill injection, and entity wrapping), plus a fully automated multi-agent evaluation framework judging behavior at both conversational and OS-level physical layers. Evaluation across frontier agents reveals three findings: (1) current agents lack effective safety awareness, with strong models (e.g., Claude Sonnet 4.6) still executing 40.64% of high-risk operations; (2) agents exhibit pervasive Execution Hallucination (EH), verbally refusing a request while the dangerous operation has already completed at the system level, invisible to every prior semantic-only framework; and (3) skill injection and entity wrapping attacks achieve high success rates, exposing pronounced agent vulnerabilities. LITMUS provides the first standardized platform for reproducible, physically grounded behavioral safety evaluation of LLM agents in real OS environments. ## BrainwashedLLM: Jailbreaking via Automated Adversarial Chain of Thoughts Discovery DevFeed: [BrainwashedLLM: Jailbreaking via Automated Adversarial Chain of Thoughts Discovery]() Original publisher: [Read original article]() Author: Masoud Poorghaffar Aghdam, G\\"oktu\\u{g} Serdar Y{\\i}ld{\\i}r{\\i}m, Sinem Sav Published: 2026-11-04T05:00:00Z Content type: article Language: en Sources: [arXiv: Cryptography and Security]() Topics: [LLM security](), [Adversarial attacks](), [llm-reasoning](), [grpo](), [Chain-of-thought](), [Artificial Intelligence]() Tags: [adversarial](), [alignment](), [attack-surface](), [automated](), [chain-of-thought](), [construction](), [context](), [cs-cr](), [discovery](), [exposed](), [feedback](), [group-relative-policy-optimization](), [grpo](), [imperative](), [input](), [intermediate](), [jailbreaks](), [large](), [learning](), [malicious](), [model]() ### AI overview BrainwashedLLM uses GRPO with a multi-objective, rule-based reward to train a small white-box model to produce adversarial reasoning traces; the reward evaluates only that model's own trace. Injected as input prefixes, the traces transferred zero-shot to four larger black-box reasoning models and steered them toward harmful outputs without target queries, gradient access, prompt optimization, or target-output feedback during attack construction. The reported ASR@5 ranged from 0.74 to 0.98, within 16 points of query-based baselines. The authors argue that exposed reasoning traces create a safety vulnerability and suggest isolating the reasoning buffer as a defense. ### Source excerpt arXiv:2611.04717v1 Announce Type: new Abstract: Large Reasoning Models (LRMs) achieve strong performance on complex tasks by generating explicit Chain-of-Thought (CoT) reasoning, but the safety implications of this intermediate reasoning surface remain underexplored. We demonstrate that the reasoning trace itself is a powerful attack vector. Safety alignment in large models is substantially less robust to malicious reasoning embedded in the context than to direct imperative prompts. We introduce BrainwashedLLM, a weak-to-strong adversarial framework that automates the discovery of reasoning-based jailbreaks. First, we use Group Relative Policy Optimization (GRPO), a Reinforcement Learning (RL) algorithm, with a multi-objective rule-based reward to fine-tune a small, white-box source model to generate adversarial reasoning traces. Crucially, the reward is computed entirely over the source's own reasoning trace. Second, we show that these traces transfer in a zero-shot manner. When traces are injected as input prefixes, they steer larger black-box LRMs to continue the adversarial reasoning and produce harmful outputs, without requiring prompt optimization, gradient access, or any form of output feedback from the target model. Across four state-of-the-art LRMs that span distinct architectures, BrainwashedLLM achieves ASR@5 between 0.74 and 0.98 across targets, within 16 points of query-based baselines, without querying the target during attack construction. To our knowledge, this is the first framework to use RL to directly optimize the CoT of LRMs as an attack surface without any target feedback, and the first to demonstrate zero-shot weak-to-strong transfer of reasoning traces. Our results suggest that exposed CoT is a vulnerability in LRMs and motivate architectural defenses that isolate the reasoning buffer. Warning: This paper contains potentially offensive and harmful text. ## J-Anchor: Defending Against Prefill Attacks via J-Space Selective Refusal Restoration DevFeed: [J-Anchor: Defending Against Prefill Attacks via J-Space Selective Refusal Restoration]() Original publisher: [Read original article]() Author: Running Yang, Beidi Zhao, Xiaoxiao Li Published: 2026-11-03T05:00:00Z Content type: article Language: en Sources: [arXiv: Artificial Intelligence](), [arXiv: Cryptography and Security]() Topics: [LLM security](), [Adversarial attacks]() Tags: [attacks](), [cs-ai](), [cs-cr](), [decoding](), [deep](), [defense](), [generation](), [inference](), [jailbreaks](), [language-models](), [lens](), [model](), [models](), [prefill](), [reasoning](), [request](), [restoration](), [robust](), [safety](), [signal](), [space](), [state]() ### AI overview The abstract reports that prefill attacks weaken both harmfulness and refusal signals, but refusal falls to the benign baseline while harmfulness remains detectable during subsequent generation. J-Anchor is a training-free inference-time defense that uses each request's pre-attack signals as an anchor to selectively restore weakened refusal in J-space during prefilling and decoding. The authors report reduced success for fixed and deep prefill attacks across model families, with benign behavior and reasoning largely preserved; ablations point to targeted restoration in the J-space refusal subspace as important. ### Source excerpt arXiv:2611.01342v1 Announce Type: new Abstract: Prefill jailbreaks can cause safety-aligned language models to continue an unsafe assistant prefix even when they would refuse the same request under normal decoding. We study how the model's internal harmfulness and refusal signals evolve along this attacker-controlled trajectory. Using the Jacobian lens, we track vocabulary-grounded harmfulness and refusal signals from the assistant header through prefilling and subsequent generation. We identify a characteristic phenomenon, which we call refusal-state erosion: both the refusal signal and the harmfulness signal weaken as the prefill progresses, but the refusal signal progressively declines to the benign baseline by the end of the prefill, while the harmfulness signal remains above its benign baseline and detectable during subsequent generation. Motivated by this observation, we introduce J-Anchor, a training-free inference-time defense that uses the model's pre-attack harmfulness and refusal signals as a request-specific anchor and selectively restores weakened refusal in J-space during prefilling and decoding. J-Anchor substantially reduces attack success under both fixed and deep prefill attacks across different model families. J-Anchor largely preserves benign behavior and reasoning ability, and ablations highlight the importance of targeted restoration in the J-space refusal subspace. Our results suggest that preserving latent refusal dynamics offers a promising route to robust defense against prefill jailbreaks. ## WebWeaver: Breaking Topology Confidentiality in LLM Multi-Agent Systems with Stealthy Context-Based Inference DevFeed: [WebWeaver: Breaking Topology Confidentiality in LLM Multi-Agent Systems with Stealthy Context-Based Inference]() Original publisher: [Read original article]() Author: Zixun Xiong, Gaoyi Wu, Lingfeng Yao, Miao Pan, Xiaojiang Du, Hao Wang Published: 2026-11-03T05:00:00Z Content type: article Language: en Sources: [arXiv: Cryptography and Security]() Topics: [LLM security](), [Multi Agent Systems](), [ai security]() Tags: [agent](), [attack](), [context](), [critical](), [cs-ai](), [cs-cr](), [diffusion](), [experiments](), [guarantees](), [inference](), [intellectual-property](), [jailbreak](), [jailbreaks](), [llm](), [masking](), [multi-agent](), [multi-agent-systems](), [overhead](), [real-world](), [safety](), [theoretical](), [topology]() ### AI overview WebWeaver aims to infer an LLM multi-agent system's complete communication topology by compromising one arbitrary agent and using its context rather than agent IDs or access to the administrative agent. It combines a covert jailbreak mechanism with a jailbreak-free diffusion design for cases where jailbreaks fail; a masking strategy preserves known topology during diffusion, with claimed theoretical correctness guarantees. The abstract reports about 60% higher inference accuracy than state-of-the-art baselines under active defenses, with negligible overhead. These are claims from the supplied abstract; it does not provide experiment details. ### Source excerpt arXiv:2603.11132v3 Announce Type: replace Abstract: Communication topology is a critical factor in the utility and safety of LLM-based multi-agent systems (LLM-MAS), making it a high-value intellectual property (IP) whose confidentiality remains insufficiently studied. Existing topology inference attempts rely on impractical assumptions, including control over the administrative agent and direct identity queries via jailbreaks, which are easily defeated by basic keyword-based defenses. As a result, prior analyses fail to capture the real-world threat of such attacks. To bridge this realism gap, we propose \textit{WebWeaver}, an attack framework that infers the complete LLM-MAS topology by compromising only a single arbitrary agent instead of the administrative agent. Unlike prior approaches, WebWeaver relies solely on agent contexts rather than agent IDs, enabling significantly stealthier inference. WebWeaver further introduces a new covert jailbreak-based mechanism and a novel fully jailbreak-free diffusion design to handle cases where jailbreaks fail. Additionally, we address a key challenge in diffusion-based inference by proposing a masking strategy that preserves known topology during diffusion, with theoretical guarantees of correctness. Extensive experiments show that WebWeaver substantially outperforms state-of-the-art (SOTA) baselines, achieving about 60\% higher inference accuracy under active defenses with negligible overhead. ## Expected Harm: Rethinking Safety Evaluation of (Mis)Aligned LLMs DevFeed: [Expected Harm: Rethinking Safety Evaluation of (Mis)Aligned LLMs]() Original publisher: [Read original article]() Author: Yen-Shan Chen, Zhi Rui Tam, Cheng-Kuang Wu, Yun-Nung Chen Published: 2026-11-02T05:00:00Z Content type: article Language: en Sources: [arXiv: Cryptography and Security](), [arXiv: Machine Learning]() Topics: [ai security](), [LLM observability](), [post-training]() Tags: [attack](), [cs-cl](), [cs-cr](), [cs-cy](), [cs-lg](), [evaluation](), [execution](), [jailbreak](), [jailbreaks](), [linear-probing](), [llm-safety](), [llms](), [metric](), [models](), [open-source](), [risk](), [safety](), [sft](), [training-data](), [vulnerability]() ### AI overview Expected Harm evaluates jailbreak risk by weighting severity according to the likelihood that a real-world actor would carry out the threat, modeled through execution cost. The authors report that tested models show "Inverse Risk Calibration": they refuse more strongly for high-cost, low-likelihood threats while remaining more vulnerable to low-cost, high-likelihood queries. Linear probing found refusal mechanisms separated queries by severity but did not use execution likelihood as a refusal signal. Analysis of Olmo-3 checkpoints links the miscalibration to SFT data, where 82.8% of safety examples were at cost levels 0-1, and finds cost representations weak from the SFT stage onward. Exploiting this behavior increased existing jailbreak attack success rates by up to 2x. ### Source excerpt arXiv:2602.01600v2 Announce Type: replace Abstract: Current evaluations of LLM safety predominantly rely on severity-based taxonomies to assess the harmfulness of models' responses to malicious queries. We argue that this formulation requires re-examination as it implicitly equates model output with realized harm while neglecting Execution Likelihood-the conditional probability of a threat being realized by a real-world actor given a model's response. In this work, we introduce Expected Harm, a metric that weights the severity of a jailbreak by its execution likelihood, modeled as a function of execution cost. Through empirical analysis of state-of-the-art models, we reveal a systematic Inverse Risk Calibration: models disproportionately exhibit stronger refusal behaviors for low-likelihood (high-cost) threats while remaining vulnerable to high-likelihood (low-cost) queries. Mechanistically, we first provide an analysis using linear probing, revealing that while refusal mechanisms linearly separate queries by severity, they do not use execution likelihood as an indicator for refusal. Then, we use Olmo-3's open-source staged checkpoints to trace the root cause of miscalibration to the Supervised Fine Tuning (SFT) training data distribution (82.8% of safety examples at cost levels 0-1) and show that cost representations are weak from the SFT stage onward. Finally, we demonstrate that this miscalibration creates a vulnerability: by exploiting this property, we increase the attack success rate of existing jailbreaks by up to $2\times$. ## Discovering Jailbreaks via Curriculum of Models DevFeed: [Discovering Jailbreaks via Curriculum of Models]() Original publisher: [Read original article]() Author: Micha{\\l} Burzy\\'nski, Noah T. Rossi, Shomit Basu, Andy Arditi Published: 2026-10-30T04:00:00Z Content type: article Language: en Sources: [arXiv: Artificial Intelligence](), [arXiv: Machine Learning]() Topics: [Reinforcement learning](), [LLM security](), [llm-reasoning](), [post-training](), [gpt-oss](), [gemma]() Tags: [cs-ai](), [cs-lg](), [discovery](), [drugs](), [explosives](), [jailbreaks](), [language-models](), [learning](), [materials](), [model](), [models](), [open](), [reinforcement-learning](), [train]() ### AI overview Curriculum of Models (CoM) addresses sparse rewards in automated jailbreak discovery by training an investigator against progressively harder modified versions of a target, moving from one that rarely refuses to the original once the investigator reliably elicits the behavior. On CBRN, explosives, and illicit-drug elicitation tasks with gemma-4-31b-it and gpt-oss-120b, CoM found single behaviors more reliably than propensity-bound rewards (PRBO) or direct training on the original target. In simultaneous multi-behavior training, CoM elicited held-out behaviors, but PRBO investigators had higher success rates. These findings are limited to the tasks and two open-weight targets described in the abstract. ### Source excerpt arXiv:2610.38833v1 Announce Type: cross Abstract: Automated jailbreak discovery aims to find prompts that elicit prohibited behaviors from safety-trained language models, helping developers identify vulnerabilities before deployment. One class of approaches uses reinforcement learning to train an investigator model to generate prompts that elicit a specified behavior from a target model. However, training directly against a safety-trained target is difficult because nearly all prompts are refused, leaving the investigator with a sparse reward signal. Prior work addresses this with the propensity bound (PRBO), a proxy reward estimated by sampling responses from a more compliant target, and then adjusting for how likely those responses would have been under the original target. We introduce Curriculum of Models (CoM), an alternative method that trains the investigator against a sequence of modified versions of the target, progressing gradually from a version that readily exhibits the specified behavior to the original target. For jailbreak discovery, training begins against a target version that rarely refuses harmful requests, and advances to the next, harder version once the investigator reliably elicits the behavior from the current one. We evaluate CoM on elicitation tasks involving CBRN materials, explosives, and illicit drugs, using two open-weight targets, gemma-4-31b-it and gpt-oss-120b. We compare against PRBO and direct training on the original target without a curriculum. When the investigator is trained to elicit a single behavior, CoM finds successful jailbreaks more reliably than either baseline. When trained on multiple behaviors simultaneously, the CoM-trained investigator can elicit held-out behaviors, though PRBO-trained investigators achieve higher success rates. Our results suggest that a simple curriculum over target models offers a promising approach for addressing sparse rewards in rare-behavior elicitation. ## Poisoned Skills: Propagating Jailbreaks Through Agent Markdown Files DevFeed: [Poisoned Skills: Propagating Jailbreaks Through Agent Markdown Files]() Original publisher: [Read original article]() Author: Kashyap R Manjusha, Murali Emani Published: 2026-10-30T04:00:00Z Content type: article Language: en Sources: [arXiv: Artificial Intelligence](), [arXiv: Cryptography and Security]() Topics: [Agent Skills](), [ecosystem-security](), [AI Chat](), [test]() Tags: [agent](), [attack-surface](), [configuration](), [cs-ai](), [cs-cr](), [jailbreaks](), [markdown](), [markdown-files](), [skills]() ### AI overview In a LangChain question-answering agent whose system prompt was assembled from a user-provided Markdown file, a jailbreak combining persona manipulation and a GPTFUZZER-based attack raised mean attack success across seven open-weight models from 0.5% with a benign prompt to 45% on 270 forbidden questions in nine harm categories; one model reached 68.9%. Neither tested obfuscation method beat plain text: a zero-width-character payload was not more effective, while a phonetic grapheme-to-phoneme payload lowered mean success to 30%. The authors report that vulnerability varied more by model than by payload obfuscation. These results concern the tested setup and models. ### Source excerpt arXiv:2610.37738v1 Announce Type: cross Abstract: Modern agent frameworks allow users to provide Markdown files containing system prompts, tool descriptions, and skill definitions (e.g., SKILL.md), which are incorporated into the agent's context during initialization. In this work, we investigate whether this configuration mechanism can also serve as an injection channel: can a jailbreak embedded in a Markdown skill file propagate through an agent and induce prohibited behavior? To answer this question, we construct a LangChain-based question-answering agent whose system prompt is assembled from a user-provided Markdown file. We embed a jailbreak combining persona manipulation with a GPTFUZZER-based attack and evaluate its effectiveness across seven contemporary open-weight models using 270 forbidden questions spanning nine harm categories. Under a benign system prompt, the models exhibit near-complete refusal behavior, with a mean attack success rate (ASR) of just 0.5%. In contrast, the injected skill file raises the mean ASR to 45%, reaching 68.9% for individual models. We further evaluate two payload obfuscation strategies: an invisible zero-width character-based stealth payload and a phonetic payload based on grapheme-to-phoneme transformations. Neither outperforms the plain-text injection; notably, the phonetic variant reduces mean ASR to 30%. These findings identify Markdown skill files as an underexplored attack surface for AI agents and suggest that vulnerability to configuration-based jailbreaks depends more strongly on the underlying model than on the specific payload obfuscation technique. ## SecurityScorecard benchmark tests AI on third-party risk work and resistance to attack DevFeed: [SecurityScorecard benchmark tests AI on third-party risk work and resistance to attack]() Original publisher: [Read original article]() Author: Duncan Riley Published: 2026-10-29T13:00:36Z Content type: news Language: en Sources: [SiliconANGLE]() Topics: [Artificial Intelligence](), [Benchmark](), [ai security](), [prompt injection](), [Security](), [Managed Services]() Tags: [adversarial](), [adversarial-testing](), [agent](), [agents](), [ai](), [ai-agent](), [ai-benchmarks](), [ai-models](), [aleksandr-yampolskiy](), [artificial-intelligence](), [attack](), [benchmark](), [cybersecurity](), [jailbreaks](), [management](), [manoj-nair](), [mike-mylrea](), [model-security](), [news](), [open-source](), [open-weight-models](), [prompt-injection](), [resistance](), [risk](), [risk-management](), [security](), [security-telemetry](), [securityscorecard](), [snyk](), [supply-chain-security](), [the-latest](), [third-party](), [third-party-risk-management](), [tprm-benchmark](), [vendor-risk]() ### AI overview SecurityScorecard's open TPRM Benchmark evaluates both vendor-risk task performance and resistance to attacks. Capability tasks include exposure triage and fourth-party technology risk, using expert-checked snapshots of the company's vendor-risk telemetry; rotating inputs and protected answer keys are intended to reduce memorization. Scores include confidence intervals and an autonomy level from L0 to L4. A separate Model Security suite tests prompt injection, jailbreaks and other attacks, with grades from A to F. Some tasks include deceptive vendor claims that conflict with telemetry to measure their effect on accuracy. Early results reportedly varied by discipline: open-weight models were approaching frontier models on supply-chain security, while several were highly susceptible to adversarial attacks. The announcement gives neither model names nor scores. The public framework supports self-testing and leaderboard submissions; its results are marked synthetic-calibrated. Official expert-verified scores come from SecurityScorecard's internal tier, and certified evaluations require contacting the company. ### Source excerpt Security ratings company SecurityScorecard Inc. today released an open benchmark for testing whether artificial intelligence models can handle third-party risk management work without being turned against the companies using them. The TPRM Benchmark is built for vendor risk programs, where a model may need to interpret security telemetry and decide which exposures matter. A lot [...] The post SecurityScorecard benchmark tests AI on third-party risk work and resistance to attack appeared first on SiliconANGLE. ## SpecGuard: Detecting Multi-Turn LLM Jailbreaks via Specification-Based Runtime Monitoring DevFeed: [SpecGuard: Detecting Multi-Turn LLM Jailbreaks via Specification-Based Runtime Monitoring]() Original publisher: [Read original article]() Author: Junle Li, Siqi Chen, Jiakai Li, Bingzhuo Zhong Published: 2026-10-28T04:00:00Z Content type: article Language: en Sources: [arXiv: Cryptography and Security]() Topics: [LLM security](), [LLM observability](), [Large Language Model]() Tags: [auditing](), [benchmarks](), [cs-cr](), [false-positive](), [framework](), [interaction](), [jailbreaks](), [language](), [linear](), [llm](), [llm-safety](), [logic](), [model](), [models](), [monitoring](), [multiple](), [natural-language](), [open-source](), [runtime](), [runtime-monitoring](), [specification]() ### AI overview SpecGuard monitors multi-turn jailbreaks as dialogue traces, where unsafe behavior may only become clear across several turns. It maps observations to causally scoped, evidence-backed facts using a fixed atomic-proposition contract, then checks Linear Temporal Logic rules with symbolic finite automata as each turn arrives. The framework supports buffered runtime checks and retrospective audits; its replayable witnesses identify the triggered rule, supporting facts, and relevant turns. The abstract reports that, on multi-turn safety benchmarks, SpecGuard outperformed representative open-source guard models while maintaining controlled false-positive rates. It is training-free and does not require access to the protected model. ### Source excerpt arXiv:2610.36258v1 Announce Type: new Abstract: Multi-turn jailbreaks turn dialogue safety into a trace property: a violation may become identifiable only when user intent and assistant behavior are considered across multiple turns. Existing guard models largely classify individual messages or exchanges and return risk scores or labels, without executable safety semantics or a formally checkable decision trail. We present SpecGuard, a training-free, specification-driven framework for monitoring multi-turn LLM interactions. SpecGuard uses a fixed atomic-proposition contract to map natural-language observations into causally scoped, evidence-backed facts, and expresses forbidden interaction patterns as Linear Temporal Logic specifications. Each specification is compiled into a symbolic finite automaton that incrementally consumes the resulting dialogue trace. The same monitoring mechanism supports buffered runtime checking and retrospective auditing, while accepting runs produce replayable witnesses that identify the triggered specification, supporting facts, and relevant dialogue turns. Experiments on multi-turn safety benchmarks show that SpecGuard consistently outperforms representative open-source guard models while maintaining controlled false-positive rates. These results demonstrate a practical path from heuristic risk classification toward formally specified, verifiable, and auditable LLM safety monitoring, without learned detector parameters or access to the protected model. ## How Far Will They Go? Red-Teaming Online Influence with Large Language Models DevFeed: [How Far Will They Go? Red-Teaming Online Influence with Large Language Models]() Original publisher: [Read original article]() Author: Daniel C. Ruiz, Anna Serbina, Ashwin Rao, Emilio Ferrara, Luca Luceri Published: 2026-10-23T04:00:00Z Content type: article Language: en Sources: [arXiv: Artificial Intelligence]() Topics: [LLM security](), [LLM evaluation / benchmarking](), [Large Language Model]() Tags: [audits](), [cs-ai](), [cs-cl](), [cs-cy](), [framework](), [influence](), [jailbreaks](), [large-language-models](), [llms](), [open-source](), [opinions](), [red-teaming](), [throughput](), [us](), [windows](), [workflow]() ### AI overview The study presents a red-teaming framework to measure how faithfully locally deployable open-source LLMs express sampled political opinions and how simple natural-language jailbreaks affect that fidelity. Testing more than 30 models across 10 families and five countries of origin on English-language, US-focused opinions, the authors report that sampled left-leaning content is typically expressed with higher fidelity, mean Overton Window scores tend to fall as model size increases, and average fidelity varies by country of origin in their sample. Jailbreak potency differs sharply across model families. The authors propose using the framework to audit political steerability and support countermeasure research. These findings are limited to the stated models, opinions, and threat model. ### Source excerpt arXiv:2605.22880v2 Announce Type: replace-cross Abstract: As large language model (LLM)-based agents increasingly participate in online discourse, red-teaming their capacity to support political influence campaigns is critical for information integrity. In pursuit of this goal, we focus on locally deployable open-source LLMs under an assumed threat model in which an operator seeks to avoid detection, prioritizes throughput, and selects released LLM checkpoints without modifying their weights. We introduce an empirical red-teaming framework for measuring LLM Overton Windows (OWs), summarizing expression fidelity across sampled political opinions, and for quantifying how simple natural-language jailbreaks change these scores. We evaluate more than 30 LLMs spanning 10 model families and five countries of origin on English-language, US-focused target opinions. We find systematic asymmetries in political expressivity: models typically express sampled left-leaning content with higher fidelity, mean OW scores tend to decrease with model size, and average expression fidelity varies across countries of origin in our sample. Jailbreak potency also varies sharply across model families, motivating a workflow for identifying effective combinations of jailbreak techniques. Our framework supports audits of political steerability and future countermeasure research. ## Mirage: Targeted Jailbreaks Against Post-Generation Safety Checkers in Multimodal Diffusion Models DevFeed: [Mirage: Targeted Jailbreaks Against Post-Generation Safety Checkers in Multimodal Diffusion Models]() Original publisher: [Read original article]() Author: Shanghao Shi, Chaoyu Zhang, Heng Jin, Hao Li, Ning Zhang, Y. Thomas Hou, Wenjing Lou Published: 2026-10-23T04:00:00Z Content type: article Language: en Sources: [arXiv: Cryptography and Security]() Topics: [ai security](), [stable-diffusion](), [ai-image-generator]() Tags: [1-5](), [2-2](), [adversarial](), [alignment](), [attack](), [concepts](), [cs-cr](), [datasets](), [diffusion](), [diffusion-models](), [evasion](), [generated-content](), [generation](), [image](), [image-generation](), [image-to-image](), [jailbreak](), [jailbreaks](), [kandinsky](), [multimodal](), [safety]() ### AI overview Mirage is a targeted jailbreak attack that embeds an adversarial image or text target in an imperceptible perturbation to a benign input image. It aligns the perturbed image with the target in multimodal encoders' shared representation space while separating it from commonly filtered NSFW concepts; this separation can transfer across representation spaces. Lightweight black-box query refinement helps improve evasion near an unknown safety boundary, without access to checker internals. The authors implemented Mirage on Stable Diffusion 1.5, Stable Diffusion XL, and Kandinsky 2.2, evaluating it on two cross-modal NSFW datasets across text-to-image and image-to-image scenarios. They report strong targeted jailbreak performance across diverse post-generation safety checkers with limited queries and high semantic alignment to targets. ### Source excerpt arXiv:2610.30348v1 Announce Type: new Abstract: Post-generation safety checkers are widely employed in diffusion-model pipelines to prevent unsafe generated content from reaching users. In this paper, we show that these safety checkers can still be bypassed by our proposed multimodal jailbreak attack, Mirage. Given an adversarial target, which can be either a reference NSFW image or a malicious text description, Mirage conceals the target within an imperceptible perturbation applied to a benign input image. Mirage exploits the shared representation space of multimodal encoders to jointly align the perturbed image with the cross-modal adversarial target while repelling its representation from commonly filtered NSFW concepts. Such semantic separation can transfer across representation spaces, enabling Mirage to evade heterogeneous safety checkers without accessing their internal details. We further employ lightweight black-box query refinement to improve evasion near the unknown safety boundary. We implemented Mirage on Stable Diffusion 1.5, Stable Diffusion XL, and Kandinsky 2.2, and evaluated it on two cross-modal NSFW datasets, covering both text-to-image and image-to-image generation scenarios. The experimental results demonstrate that Mirage achieves strong targeted jailbreak performance across diverse post-generation safety checkers with limited black-box queries while preserving high semantic alignment with the adversarial targets. ## Harmful Responses Are Jailbreak Templates: Transferring Autoregressive Jailbreaks to Diffusion Language Models DevFeed: [Harmful Responses Are Jailbreak Templates: Transferring Autoregressive Jailbreaks to Diffusion Language Models]() Original publisher: [Read original article]() Author: Jinwei Gan, Zifeng Cheng, Zhiwei Jiang, Ran Sun, Shiping Ge, Qing Gu Published: 2026-10-23T04:00:00Z Content type: article Language: en Sources: [arXiv: Machine Learning]() Topics: [LLM security](), [LLM Techniques]() Tags: [autoregressive](), [cs-lg](), [diffusion](), [jailbreak](), [jailbreaks](), [language-models](), [templates]() ### AI overview The paper reports that harmful responses generated by autoregressive (AR) jailbreaks can act as templates for eliciting harmful content from diffusion language models (dLLMs). Its RMT-Jailbreak framework uniformly retains some tokens while masking others to create a high-mask-ratio template, then uses prompt-based safety filtering to mask remaining visible harmful content before the target dLLM reconstructs it bidirectionally. Experiments across multiple AR jailbreak methods and dLLMs reportedly show effective, general cross-paradigm transfer. The supplied text is an abstract and does not provide detailed results or limitations. ### Source excerpt arXiv:2610.31331v1 Announce Type: new Abstract: Diffusion large language models (dLLMs) enable flexible masked generation through bidirectional context modeling, but this capability also introduces a distinct jailbreak attack surface. We find that a harmful response to a given query naturally serves as an effective jailbreak template: simply masking a subset of its tokens can induce a dLLM to reconstruct harmful content. Based on this observation, we propose RMT-Jailbreak, a general framework that transfers existing autoregressive (AR) jailbreak attacks to dLLMs through their induced harmful responses. RMT-Jailbreak converts an AR-generated harmful response into a high-mask-ratio template using uniform token retention, and further applies prompt-based safety filtering to mask any remaining visible harmful content. The refined template is then provided to the target dLLM for bidirectional reconstruction. Extensive experiments across multiple AR jailbreak methods and target dLLMs demonstrate the effectiveness and generality of RMT-Jailbreak, revealing strong cross-paradigm transferability of jailbreak attacks. ## Beyond Attack Success Rate: Cost-Aware Metric for Evaluating Iterative LLM Jailbreaks DevFeed: [Beyond Attack Success Rate: Cost-Aware Metric for Evaluating Iterative LLM Jailbreaks]() Original publisher: [Read original article]() Author: Rohan Menon, Chih-Hong Cheng Published: 2026-10-23T04:00:00Z Content type: article Language: en Sources: [arXiv: Cryptography and Security]() Topics: [LLM security](), [Large Language Model]() Tags: [asr](), [cost](), [cs-cl](), [cs-cr](), [evaluation](), [jailbreaks](), [llm](), [metric]() ### AI overview The proposed ASR_STEP metric discounts jailbreak success according to repeated calls to the victim model while preserving binary success judgments, making interaction costs visible alongside standard attack success rate. On JailBreakBench across three model families, discounting lowered GCG scores by 7.0-8.7 percentage points and PAIR scores by 3.4-12 points; single-step PAP scores were unchanged. The differing PAIR range suggests the step quantity varies by iterative strategy. The abstract recommends reporting both metrics; it does not specify the discount formula. ### Source excerpt arXiv:2610.26589v2 Announce Type: replace Abstract: Attack success rate (ASR) evaluates jailbreaks against large language models without accounting for repeated calls to the victim model. Yet each additional call gives the deployed system another opportunity to detect the attack and intervene. We introduce $\mathrm{ASR}_{\mathrm{STEP}}$, which penalizes additional victim-model calls while retaining binary success judgments. Our evaluation considers GCG, PAIR with a step budget, and a single-step PAP baseline on JailBreakBench across three model families. GCG's scores decrease by 7.0-8.7 percentage points after discounting, while PAP's remain unchanged. While for PAIR attack, we see a wider discounting range between 3.4-12 percentage points, showing that the "step" quantity differs per iterative strategy. Reporting both metrics makes the discount due to repeated victim-model interaction explicit. Code is available here: https://github.com/SuELS-Group/beyond_asr. ## Beyond Binary Labels: Diagnostic Evaluation of LLM Safety with Prompt-Level Rubrics DevFeed: [Beyond Binary Labels: Diagnostic Evaluation of LLM Safety with Prompt-Level Rubrics]() Original publisher: [Read original article]() Author: Hongyu Li, Xinlong Huang, Zhan Yu, Jing Zhao, Zhe Zhao, Juntao Zhong, Xiong Kuang, Rui Yuan, ZhenXiang Yan, Huilin Xu, Haitao Yu, Lilin Wang, Rui Chen, Blaze Chen, Fengzong Lian, Renfen Hu Published: 2026-10-22T04:00:00Z Content type: article Language: en Sources: [arXiv: Cryptography and Security]() Topics: [LLM security](), [Ground truth / benchmark quality](), [ai alignment](), [Large Language Model](), [LLM observability]() Tags: [benchmark](), [cs-ai](), [cs-cl](), [cs-cr](), [evaluation](), [framework](), [jailbreaks](), [llm](), [llm-as-a-judge](), [llm-safety](), [llms](), [prompt](), [reliability](), [safety](), [taxonomy](), [vulnerabilities]() ### AI overview DiagSafe evaluates LLM safety with prompt-level rubrics that separately score safety and instruction usefulness, adding concrete harm criteria and exemptions to reduce false positives. Its bilingual benchmark uses an 84-label taxonomy: DiagBase covers direct harmful prompts and template jailbreaks in single- and multi-turn settings, while DiagAttack embeds harmful intent in complex instruction-following tasks. The authors report 95.34% agreement with human annotations, above the baselines' 69-87%, and inter-judge Cohen's κ of 0.864. Their evaluation finds differences in unsafe rates and vulnerability profiles across leading models, with particular susceptibility in Chinese and multi-turn contexts. ### Source excerpt arXiv:2610.29225v1 Announce Type: cross Abstract: Current LLM safety benchmarks predominantly rely on binary safe or unsafe labels, lacking explanatory depth and suffering from low consistency across automated judges. To address this, we introduce DiagSafe, a diagnostic evaluation framework that moves beyond binary classification by employing prompt-level rubrics. Grounded in globally applicable AI safety policies, each rubric provides explicit criteria along two dimensions: safety, which targets concrete harms with specific exemptions to suppress false positives, and usefulness, which independently evaluates instruction fulfillment. To ensure a robust and realistic evaluation, we construct a bilingual (English-Chinese) benchmark grounded in a comprehensive 84-label safety taxonomy, consisting of two subsets: DiagBase, which covers single- and multi-turn prompts with direct harmful queries and template-based jailbreaks, and DiagAttack, which embeds harmful intents into complex instruction-following tasks to form challenging instruction attacks. Extensive experiments demonstrate that DiagSafe significantly enhances the reliability of LLM-as-a-judge evaluation, achieving 95.34% agreement with human annotations -- substantially higher than the 69%-87% achieved by the baselines -- and high inter-judge consistency (Cohen's $\kappa$ = 0.864). Furthermore, applying our framework reveals that leading LLMs differ markedly in their unsafe rates, and each model also exhibits unique vulnerabilities along different dimensions. In addition, we observe that models are particularly susceptible in Chinese and multi-turn contexts. Ultimately, DiagSafe provides a reliable and explainable tool for auditing and optimizing LLM alignment. ## Detecting Misbehaviors of Large Vision-Language Models by Evidential Uncertainty Quantification DevFeed: [Detecting Misbehaviors of Large Vision-Language Models by Evidential Uncertainty Quantification]() Original publisher: [Read original article]() Author: Tao Huang, Rui Wang, Xiaofei Liu, Yi Qin, Li Duan, Liping Jing Published: 2026-10-22T04:00:00Z Content type: article Language: en Sources: [arXiv: Machine Learning]() Topics: [Uncertainty quantification LLMs](), [Hallucination detection](), [vlm](), [Adversarial attacks]() Tags: [adversarial](), [cs-lg](), [detection](), [hallucinations](), [jailbreaks](), [language-models](), [layer](), [ood](), [uncertainty](), [uncertainty-quantification](), [vision-language-models]() ### AI overview Evidential Uncertainty Quantification (EUQ) detects large vision-language model misbehaviors by distinguishing conflicting internal knowledge from missing supporting information. In a single forward pass, it treats output-head features as positive or negative evidence and aggregates them using Evidence Theory to quantify conflict and ignorance. The abstract reports evaluations across hallucinations, jailbreaks, adversarial vulnerabilities, and out-of-distribution failures, with EUQ outperforming strong baselines. It associates hallucinations with high conflict and out-of-distribution failures with high ignorance. Layer-wise uncertainty analysis is also presented as a way to interpret changes in internal representations. ### Source excerpt arXiv:2602.05535v3 Announce Type: replace Abstract: %Large vision-language models (LVLMs) have shown substantial advances in multimodal understanding and generation. However, when presented with incompetent or adversarial inputs, they frequently produce unreliable or even harmful content, such as fact hallucinations or dangerous instructions. This misalignment with human expectations, referred to as \emph{misbehaviors} of LVLMs, raises serious concerns for deployment in critical applications. These misbehaviors are found to stem from epistemic uncertainty, specifically either conflicting internal knowledge or the absence of supporting information. However, existing uncertainty quantification methods, which typically capture only overall epistemic uncertainty, have shown limited effectiveness in identifying such issues. To address this gap, we propose Evidential Uncertainty Quantification (EUQ), a fine-grained method that captures both information conflict and ignorance for effective detection of LVLM misbehaviors. In particular, we interpret features from the model output head as either supporting (positive) or opposing (negative) evidence. Leveraging Evidence Theory, we model and aggregate this evidence to quantify internal conflict and knowledge gaps within a single forward pass. %We extensively evaluate our method across four categories of misbehavior, including hallucinations, jailbreaks, adversarial vulnerabilities, and out-of-distribution (OOD) failures, using state-of-the-art LVLMs, and find that EUQ consistently outperforms strong baselines, showing that hallucinations correspond to high internal conflict and OOD failures to high ignorance. Furthermore, layer-wise evidential uncertainty dynamics analysis helps interpret the evolution of internal representations from a new perspective. The source code is available at https://github.com/HT86159/EUQ. ## Concerns in Benchmarking Jailbreaks: An Empirical Study of Attack Success Rates DevFeed: [Concerns in Benchmarking Jailbreaks: An Empirical Study of Attack Success Rates]() Original publisher: [Read original article]() Author: Leon Tu Nguyen, Rohan Menon, Rafael Tappe Maestro, Eric Aubinais, Chih-Hong Cheng Published: 2026-10-21T04:00:00Z Content type: article Language: en Sources: [arXiv: Machine Learning]() Topics: [LLM security](), [ai alignment](), [Ollama]() Tags: [asr](), [attacks](), [benchmarking](), [cs-ai](), [cs-lg](), [evaluation](), [experiment](), [failure](), [generation](), [interaction](), [jailbreaks](), [language-models](), [matching](), [models](), [ollama](), [probabilistic](), [qwen3](), [ranking](), [reporting](), [responses](), [robust](), [study]() ### AI overview Jailbreak attack success rates depend strongly on how attempts are generated and responses are judged. Across eight attacks on three open-weight models, low-resource language attacks averaged 96.0% ASR with keyword matching but 2.7% with a Qwen3 evaluator using StrongREJECT instructions. Changing the number and arrangement of attempts reversed the relative ranking of DeepInception and rewriting. For unchanged requests sampled 100 times at Ollama's default temperature of 0.8, ASR was 100% by keyword matching and 50% by Qwen3 evaluation. The authors warn that repeated sampling can reveal unsafe responses after an initial refusal, consistent with probabilistic suppression. They recommend defining success explicitly, matching response budgets, validating evaluators, and reporting cumulative failures under repeated interaction. ### Source excerpt arXiv:2610.26595v1 Announce Type: new Abstract: Jailbreak methods are commonly compared using Attack Success Rate (ASR), but these comparisons depend on how responses are generated and judged. In this paper, we examine eight attacks against three open-weight language models using three evaluation procedures. Low-resource language attacks achieve a mean ASR of 96.0% under keyword matching but 2.7% under a Qwen3 evaluator following StrongREJECT scoring instructions. Changing the number and arrangement of attempts reverses the ranking of DeepInception and rewriting. In a separate experiment, sampling 100 responses for each unchanged request at Ollama's default temperature of 0.8 yields 100% ASR under keyword matching but 50% under the Qwen3 evaluator. Beyond evaluator sensitivity, these findings raise a safety concern: repeated sampling can expose residual unsafe behavior despite an initial refusal, suggesting probabilistic suppression rather than robust exclusion. We recommend explicit success definitions, matched response budgets, validated evaluators, and reporting cumulative failure under repeated interaction rather than relying on single-generation tests. ## Red team your AI model with garak DevFeed: [Red team your AI model with garak]() Original publisher: [Read original article]() Author: Anika Aracan Published: 2026-09-29T13:01:46Z Content type: tutorial Language: en Sources: [Red Hat](), [Red Hat Developer]() Topics: [prompt injection](), [Open Source](), [CI/CD]() Tags: [adversarial](), [ai-models](), [artificial-intelligence](), [ci-cd](), [cybersecurity](), [jailbreaks](), [llm](), [red-team](), [security]() ### AI overview This tutorial shows how to use the open-source garak scanner to red-team an internal chatbot or other large language model. It covers probing for prompt injection and jailbreak vulnerabilities, then describes broader scans, CI/CD integration, and sharing findings with a security team. ### Source excerpt Your team just shipped an internal chatbot built on an open-weight model. It works great! It answers all of your questions, summarizes documents, and helps onboard new hires with scary efficiency. But then your security team asks, "Has anyone tested whether this thing can be jailbroken?" Jailbroken? What's that? The post Red team your AI model with garak appeared first on Red Hat Developer. ## Exclusive: Darktrace makes SECURE AI generally available to monitor enterprise AI use DevFeed: [Exclusive: Darktrace makes SECURE AI generally available to monitor enterprise AI use]() Original publisher: [Read original article]() Author: Duncan Riley Published: 2026-09-22T13:00:43Z Content type: news Language: en Sources: [SiliconANGLE]() Topics: [Cybersecurity](), [Artificial Intelligence](), [Security](), [shadow AI](), [telemetry](), [AI Bots]() Tags: [ai](), [ai-agents](), [ai-governance](), [ai-security](), [amazon-bedrock](), [amazon-web-services](), [anthropic](), [aws](), [aws-marketplace](), [chatgpt](), [chatgpt-enterprise](), [claude](), [copilot](), [cybersecurity](), [darktrace](), [darktrace-behavioral-defense-platform](), [darktrace-holdings](), [daybreak-defense-network](), [detection](), [ed-jennings](), [enterprise-ai](), [generative-ai](), [jailbreaks](), [mccall-mcintyre](), [microsoft](), [microsoft-copilot](), [microsoft-copilot-studio](), [microsoft-entra-global-secure-access](), [microsoft-marketplace](), [news](), [openai](), [prompt-injection](), [release](), [salesforce](), [secure-ai](), [security](), [shadow-ai](), [telemetry](), [the-latest]() ### AI overview Darktrace has made its SECURE AI offering generally available to help enterprises monitor generative AI services and agents. The product detects risky activity such as jailbreak attempts, sensitive data exposure, indirect prompt injection, unauthorized AI services, and excessive agent permissions, with integrations for AWS, Anthropic, Microsoft, and OpenAI products. ### Source excerpt U.K.-based cybersecurity company Darktrace Holdings Ltd. today made its SECURE AI offering generally available, bringing its behavioral detection approach to corporate use of artificial intelligence tools and agents. The release is aimed at AI use that has spread through the enterprise faster than security teams can keep track of it. Telemetry from about 8,200 Darktrace [...] The post Exclusive: Darktrace makes SECURE AI generally available to monitor enterprise AI use appeared first on SiliconANGLE.