# Jailbreak Published articles for Jailbreak. This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles. ## GuardrailAgent: An Agentic Framework for Jailbreak Defense DevFeed: [GuardrailAgent: An Agentic Framework for Jailbreak Defense]() Original publisher: [Read original article]() Author: Tianze Chang, Junjie Chu, Yixin Wu, Michael Backes, Yang Zhang Published: 2026-11-09T05:00:00Z Content type: article Language: en Sources: [arXiv: Cryptography and Security]() Topics: [Agentic AI](), [Large Language Model](), [Resilience]() Tags: [abstract](), [adapting](), [adaptive](), [adversary](), [agent](), [agentic](), [align](), [arxiv](), [attack](), [attacks](), [base](), [behaviors](), [built-in](), [cases](), [cost](), [cs-cr](), [defense](), [detection](), [existing](), [framework](), [jailbreak]() ### AI overview GuardrailAgent is a multi-agent framework designed to defend against evolving jailbreak attacks. It builds a continuously updated knowledge base by gathering and summarizing attack characteristics, uses it for detection on both inputs and outputs, and incorporates misclassified cases through a feedback module for iterative improvement. In tests spanning more than ten representative jailbreaks, it reported 93.1% input-side detection, 93.9% overall detection, and a 95.8% F1 score, with rapid adaptation to unseen attacks. These results are limited to the reported evaluation. ### Source excerpt arXiv:2611.09734v1 Announce Type: new Abstract: Despite undergoing safety training to align with human values, large language models (LLMs) remain vulnerable to jailbreak attacks that circumvent built-in safeguards and elicit malicious behaviors given by the adversary. Existing defenses are mostly static and fail to adapt to new or unknown attack strategies, providing limited resilience. To address these, we propose a multi-agent defense framework, GuardrailAgent, that offers adaptive, end-to-end protection. The system autonomously gathers and summarizes attack characteristics into a continuously updated knowledge base that supports both input- and output-side detection. A feedback module integrates misclassified cases for iterative improvement at minimal cost. Tested on over ten representative jailbreaks, the framework achieved a 93.1% input-side detection rate, 93.9% overall detection, and a 95.8% F1 score, while rapidly adapting to unseen attacks. These results demonstrate GuardrailAgent's outstanding capability to maintain LLM safety in the face of evolving jailbreak tactics. ## AttentionSwitch: Jailbreak Defense for LLMs via Joint Learning of Sparse Attention-Head Selection and Scaling DevFeed: [AttentionSwitch: Jailbreak Defense for LLMs via Joint Learning of Sparse Attention-Head Selection and Scaling]() Original publisher: [Read original article]() Author: Doniyorkhon Obidov, Honggang Yu, Xiaolong Guo, Kaichen Yang Published: 2026-11-09T05:00:00Z Content type: article Language: en Sources: [arXiv: Cryptography and Security]() Topics: [LLM security](), [ai safety](), [Large Language Model](), [Adversarial attacks](), [post-training](), [Model Development]() Tags: [attacks](), [attention](), [cs-ai](), [cs-cl](), [cs-cr](), [defense](), [fine-tuning](), [jailbreak](), [learning](), [llms](), [scaling](), [selection](), [sparse-attention]() ### AI overview AttentionSwitch jointly learns which attention heads to intervene on and how strongly to change them within a frozen LLM, addressing a limitation of defenses that optimize location and intervention separately. It selects a sparse set of heads and applies query-conditioned multiplicative scaling that can attenuate or amplify their outputs without changing representation directions. The abstract reports better performance than existing defenses across popular models and jailbreak attacks, with particularly strong results when training data is scarce. It does not provide specific metrics or describe the evaluation setup. ### Source excerpt arXiv:2611.08752v1 Announce Type: cross Abstract: Large language models have achieved remarkable performance across a wide range of tasks, yet they remain susceptible to jailbreak attacks that circumvent safety mechanisms and elicit harmful or unintended outputs. Although safety can be improved through model fine-tuning, such approaches can be computationally expensive and may degrade previously acquired capabilities through catastrophic forgetting. Recent jailbreak defenses have therefore explored targeted interventions on specific model components associated with unsafe behavior. However, existing approaches typically decouple two inherently related decisions: where to intervene and how to modify the selected components. Such sequential optimization may be suboptimal because the optimal intervention location can depend on the intervention applied there, and vice versa. In this work, we introduce AttentionSwitch, the first jailbreak defense to jointly optimize intervention sites and strengths within a frozen LLM. Specifically, AttentionSwitch learns a sparse set of attention heads together with query-conditioned multiplicative scaling factors, allowing the outputs of the selected heads to be either attenuated or amplified without altering the directions of their representations. Across popular language models and jailbreak attacks, AttentionSwitch outperforms existing defenses, with particularly strong results when training data is scarce. ## Jailbreak susceptibility prediction and mitigation via the behavioral geometry of models DevFeed: [Jailbreak susceptibility prediction and mitigation via the behavioral geometry of models]() Original publisher: [Read original article]() Author: Hayden Helm, Xiaodong Liu, Weiwei Yang Published: 2026-11-09T05:00:00Z Content type: article Language: en Sources: [arXiv: Cryptography and Security](), [arXiv: Machine Learning]() Topics: [LLM security](), [ai security]() Tags: [cs-ai](), [cs-cr](), [cs-lg](), [defense](), [deployment](), [detection](), [evaluation](), [framework](), [jailbreak](), [mitigation](), [model](), [models](), [optimization](), [optimized](), [prediction](), [providers](), [robust](), [set]() ### AI overview The paper proposes using behavioral geometry across previously evaluated and defended models to predict jailbreak susceptibility and transfer defenses, reducing the need to fully evaluate every model configuration. Across 82 models from 24 providers and 100 configurations of one base model, its methods achieved 0.94 AUPRC for susceptibility detection with about 98% fewer probes than full evaluation. Geometry-based selection of a source model for defense transfer outperformed assigning by same provider by 4.1% (p = 0.004) at no extra probe cost; three models were sufficient to cover the evaluated population. The abstract reports robustness to hyperparameter selection and judge. ### Source excerpt arXiv:2605.26409v2 Announce Type: replace Abstract: Evaluating and mitigating a generative system's susceptibility to jailbreak attacks is critical to its safe deployment. Given the number of deployable systems, full per-configuration evaluation and optimization is impractical. In this paper, we formalize the behavioral geometry of a population of models that, by leveraging previously evaluated and defended models, supports both efficient susceptibility prediction and effective defense transfer across a population. We apply the framework to 82 models spanning 24 providers and to 100 system configurations of a single base model. Simple methods that use the behavioral geometry reach an AUPRC of $0.94$ for susceptibility detection with $\approx98\%$ fewer probes relative to a full evaluation. Using the behavioral geometry to select which model to transfer an optimized defense from outperforms same-provider assignment ($+4.1\%$, $p = 0.004$) at no additional probe cost, with a set of three models sufficient to cover the population. Results are robust to hyperparameter selection and judge. ## ProMoral-Bench: Evaluating Prompting Strategies for Moral Reasoning and Safety in LLMs DevFeed: [ProMoral-Bench: Evaluating Prompting Strategies for Moral Reasoning and Safety in LLMs]() Original publisher: [Read original article]() Author: Rohan Subramanian Thomas, Shikhar Shiromani, Abdullah Chaudhry, Ruizhe Li, Vasu Sharma, Kevin Zhu Published: 2026-11-06T05:00:00Z Content type: comparison Language: en Sources: [arXiv: Artificial Intelligence]() Topics: [llm-reasoning](), [Enhancing AI model performance](), [LLM security]() Tags: [behavior](), [benchmark](), [code](), [comparison](), [comparisons](), [cs-ai](), [cs-cl](), [data](), [datasets](), [ethics](), [evaluation](), [few-shot](), [jailbreak](), [language-models](), [large-language-models](), [large-language-models-llms](), [llm](), [llms](), [prompting](), [reasoning](), [safety](), [strategies]() ### AI overview ProMoral-Bench compares 10 prompting paradigms across four LLM families using ETHICS, Scruples, WildJailbreak, and a 200-pair ETHICS-Contrast set. It reports task-level results alongside Moral Competence Score (MCS), Safety Robustness Score (SRS) for jailbreak refusal and over-refusal, and Unified Moral Safety Score (UMSS), a study-specific relative summary rather than an absolute measure of safety or moral grounding. In this fixed-template evaluation, compact exemplar-guided scaffolds offered stronger competence-robustness trade-offs at lower token cost than the multi-stage variants tested, though multi-stage prompting sometimes improved jailbreak refusal. Few-shot examples did not overlap with evaluation items but came from the same evaluation distributions, so the results do not establish out-of-distribution robustness. ### Source excerpt arXiv:2602.13274v2 Announce Type: replace Abstract: Prompt design affects the moral judgments and refusal behavior produced by large language models (LLMs), yet controlled comparisons remain fragmented across datasets and models. We introduce ProMoral-Bench, a benchmark-style comparison of 10 prompting paradigms across four LLM families. Using ETHICS, Scruples, WildJailbreak, and our 200-pair ETHICS-Contrast set, we report task-level metrics together with Moral Competence Score (MCS), jailbreak/over-refusal Safety Robustness Score (SRS), and Unified Moral Safety Score (UMSS). UMSS is a study-specific relative summary of MCS and SRS, not an absolute measure of model safety or moral grounding. In this fixed-template evaluation, compact exemplar-guided scaffolds attain stronger competence-robustness trade-offs at lower token cost than the particular multi-stage variants tested, although multi-stage prompting occasionally improves jailbreak refusal. The few-shot demonstrations are non-overlapping but drawn from the evaluation distributions, so their advantage should not be read as out-of-distribution robustness. Code and data are available at https://github.com/rohanthomas334-sketch/ProMoral_Bench. ## JailBound: FOL-Guided Jailbreak Evaluation for Exploring LLM Behavioral Boundaries DevFeed: [JailBound: FOL-Guided Jailbreak Evaluation for Exploring LLM Behavioral Boundaries]() Original publisher: [Read original article]() Author: Fazong Wu, Xin Wang, Ming Yang, Zhenyong Zhang, Xiaoming Wu Published: 2026-11-06T05:00:00Z Content type: article Language: en Sources: [arXiv: Cryptography and Security]() Topics: [LLM security](), [Large Language Model](), [test]() Tags: [adversarial](), [attack](), [automated](), [behavior](), [benchmark](), [benchmarks](), [cs-cr](), [evaluation](), [framework](), [generator](), [jailbreak](), [llm](), [local](), [safety](), [search]() ### AI overview JailBound combines automated benchmark construction with first-order-loss-guided search to evaluate jailbreak behavior across threat conditions. A threat taxonomy guides a fine-tuned meta-attack generator to synthesize prompts; dual search branches explore high-value attack states and states sensitive to behavioral changes. Across 46 LLMs from 13 model families, the High-Value Branch raised average attack success rate over a zero-order-loss-only baseline by 5.1 and 4.9 percentage points in two matched evaluation settings. In local perturbation tests on JailBound and S-Eval, Safety-Sensitivity Branch states had higher judge-label flip rates than High-Value Branch states at every tested radius, suggesting greater local sensitivity in those safety labels. ### Source excerpt arXiv:2611.07530v1 Announce Type: new Abstract: As large language models (LLMs) are increasingly deployed as general-purpose conversational assistants, they must handle diverse user requests while reliably refusing harmful ones. Jailbreak attacks craft adversarial instructions or conversational contexts to elicit unsafe responses despite these safeguards. However, existing jailbreak benchmarks often rely on manually designed or collected prompts, making it difficult to systematically extend coverage to new threat conditions. Moreover, existing attack search methods typically optimize for attack success rate (ASR), providing limited insight into how model behavior changes locally between safe refusal and unsafe response. To address these limitations, we present JailBound, a unified jailbreak evaluation framework that combines automated benchmark construction with first-order loss (FOL)-guided dual-branch search. JailBound organizes threats by risk category, application domain, and attack type. Conditioned on this taxonomy, a fine-tuned meta-attack generator synthesizes evaluation prompts across diverse threat conditions. FOL serves as a local sensitivity signal for exploring high-value attack states and behavior-sensitive states. We evaluate JailBound across 46 LLMs from 13 model families under a unified experimental protocol. Under matched settings, the High-Value Branch improves average ASR over a ZOL-only baseline by 5.1 and 4.9 percentage points across the two evaluation settings. Under local perturbations, states from the Safety-Sensitivity Branch exhibit higher behavioral flip rates than those from the High-Value Branch across all tested radii for JailBound and S-Eval, indicating greater local sensitivity of judge-assigned safety labels. Warning: this paper includes examples that may be offensive or harmful. ## SafeStart: Mitigating the Overlooked Safety Cost of Active Expert Reduction in MoE LLM Serving DevFeed: [SafeStart: Mitigating the Overlooked Safety Cost of Active Expert Reduction in MoE LLM Serving]() Original publisher: [Read original article]() Author: Erfan Baghaei Potraghloo, Saeid Shokoufa, Seyedarmin Azizi, Arya Fayyazi, Souvik Kundu, Mehdi Kamal, Massoud Pedram Published: 2026-11-05T05:00:00Z Content type: article Language: en Sources: [arXiv: Artificial Intelligence]() Topics: [Mixture of Experts (MoE)](), [LLM optimization](), [LLM security](), [LLM Techniques](), [vllm]() Tags: [analysis](), [batch](), [benchmarks](), [compliance](), [cost](), [cs-ai](), [cs-lg](), [decode](), [decoding](), [deepseek](), [experts](), [generation](), [jailbreak](), [kernel](), [language-models](), [llm](), [llm-serving](), [mixture-of-experts-moe](), [moe](), [reduction](), [safety](), [serving]() ### AI overview Reducing the number of active experts per token can weaken MoE models' refusal of harmful requests even when benchmark accuracy changes little. In vLLM on one RTX A6000, retaining about two thirds of selected experts improved offline decoding speed by 15-28% at batch size 8; under a jailbreak, harmful answers rose significantly for three of nine tested models. SafeStart runs all experts while processing the prompt and the first eight generated tokens, then uses the reduced expert set. It retained at least 85% of the decode speedup at batch sizes 8 and 32. On the three affected models, refusal changes under the tested jailbreak ranged from −2.1 to +1.3 percentage points; on 889 held-out requests, SafeStart limited the harmful-compliance increase to 0.6-2.6 points and was non-inferior to the original model on two of the three. The results suggest refusal should be evaluated alongside accuracy when enabling expert reduction. ### Source excerpt arXiv:2611.06147v1 Announce Type: new Abstract: Mixture-of-Experts (MoE) language models send each token through a few of many expert subnetworks. Serving methods make these models faster by running even fewer experts per token, and they are validated on accuracy benchmarks. We show that these methods can weaken the model's refusal of harmful requests, and we introduce SafeStart, which largely restores refusal while keeping most of the speedup. In vLLM on one RTX A6000, keeping about two thirds of the selected experts speeds up offline decoding by 15 to 28% at batch size 8. We find that, at this setting, under a jailbreak that tells the model not to refuse, the share of harmful requests that the model answers rises by 18.4 percentage points on Qwen1.5-MoE, 9.5 on OLMoE, and 6.6 on DeepSeek-V2-Lite, the three of nine models where it rises significantly, while MMLU accuracy drops by less than two points. We discover that most of this increase arises while the model processes the prompt and generates its first few tokens. SafeStart therefore runs all experts for the prompt and the first eight generated tokens, and the reduced set of experts afterwards, through a fused kernel in vLLM. With experts removed, SafeStart keeps at least 85% of the offline decode speedup at batch sizes 8 and 32 and brings the change under this jailbreak to between -2.1 and +1.3 points for three expert-reduction methods on these three models. On 889 held-out requests, in an analysis fixed before generation, truncation raises harmful compliance by 5.8 to 21.4 points, and SafeStart leaves a rise of 0.6 to 2.6 points, non-inferior to the original model on two of the three models. Our results show that faster MoE serving need not cost safety: a serving system can keep most of the gain from running fewer experts by running all of them at the start of each response, and it should test refusal, not only accuracy, before it enables such a method. ## UpSafe$^\\circ$C: Upcycling for Controllable Safety in Large Language Models DevFeed: [UpSafe$^\\circ$C: Upcycling for Controllable Safety in Large Language Models]() Original publisher: [Read original article]() Author: Yuhao Sun, Zhuoer Xu, Shiwen Cui, Kun Yang, Lingyun Yu, Yongdong Zhang, Hongtao Xie Published: 2026-11-05T05:00:00Z Content type: article Language: en Sources: [arXiv: Artificial Intelligence](), [arXiv: Cryptography and Security]() Topics: [LLM security](), [Mixture of Experts (MoE)](), [post-training](), [LLM Techniques](), [Large Language Model](), [LLM evaluation / benchmarking]() Tags: [benchmarks](), [capabilities](), [control](), [cs-ai](), [cs-cr](), [cs-lg](), [guardrail](), [guardrails](), [inference](), [jailbreak](), [language-models](), [large-language-models](), [llm-safety](), [mixture-of-experts](), [mixture-of-experts-moe](), [modular](), [post-training](), [robust](), [router](), [safety](), [sft](), [temperature](), [upcycling]() ### AI overview UpSafe°C adds controllable safety to LLMs by converting identified safety-critical layers into a sparse Mixture-of-Experts structure. Its router selectively activates the model's original MLPs and added safety experts, while two-stage supervised fine-tuning aims to strengthen safety discrimination without losing general capability. A safety temperature setting adjusts the safety-utility trade-off at inference time. The abstract reports improved resistance to harmful and jailbreak inputs across multiple benchmarks, base models, and scales, with competitive general-task performance; it does not provide detailed benchmark results in the supplied excerpt. ### Source excerpt arXiv:2510.02194v2 Announce Type: replace Abstract: Large Language Models (LLMs) have achieved remarkable progress across a wide range of tasks, but remain vulnerable to safety risks such as harmful content generation and jailbreak attacks. Existing safety techniques---including external guardrails, inference-time guidance, and post-training alignment---each face limitations in balancing safety, utility, and controllability. In this work, we propose UpSafe$^\circ$C, a unified framework for enhancing LLM safety through safety-aware upcycling. Our approach first identifies safety-critical layers and upcycles them into a sparse Mixture-of-Experts (MoE) structure, where the router acts as a soft guardrail that selectively activates original MLPs and added safety experts. We further introduce a two-stage SFT strategy to strengthen safety discrimination while preserving general capabilities. To enable flexible control at inference time, we introduce a safety temperature mechanism, allowing dynamic adjustment of the trade-off between safety and utility. Experiments across multiple benchmarks, base model, and model scales demonstrate that UpSafe$^\circ$C achieves robust safety improvements against harmful and jailbreak inputs, while maintaining competitive performance on general tasks. Moreover, analysis shows that safety temperature provides fine-grained inference-time control that achieves controllable trade-off between utility and safety. Our results highlight a new direction for LLM safety: moving from static alignment toward dynamic, modular, and inference-aware control. Code: https://github.com/HansSunY/UpSafe. ## Addition by Subtraction: Toward Mitigating Fragile Safety Alignment in Large Language Models DevFeed: [Addition by Subtraction: Toward Mitigating Fragile Safety Alignment in Large Language Models]() Original publisher: [Read original article]() Author: Hamid Dashtbani, Alexandra Dmitrienko Published: 2026-11-05T05:00:00Z Content type: article Language: en Sources: [arXiv: Artificial Intelligence](), [arXiv: Cryptography and Security]() Topics: [post-training](), [LLM security](), [LLM Techniques](), [LLM evaluation / benchmarking](), [Large Language Model]() Tags: [ablation](), [alignment](), [capabilities](), [cs-ai](), [cs-cr](), [experiments](), [jailbreak](), [language-models](), [large-language-models-llms](), [optimization](), [pruning](), [robustness](), [safety]() ### AI overview The proposed "Addition by Subtraction" method aims to make safety alignment less dependent on a small set of critical model components. During finetuning, it gradually removes identified safety-critical neurons, then uses preference optimization to rebuild aligned behavior through other parts of the model. Across four models from three vendors, the authors report greater robustness to pruning attacks, refusal-direction ablation, and a broad range of black-box jailbreaks, with attack success rates reduced by up to 90% for some attacks. They also report improved truthfulness while largely preserving general capabilities. ### Source excerpt arXiv:2611.06553v1 Announce Type: new Abstract: Large language models (LLMs) undergo safety alignment to prevent harmful generations, yet they remain vulnerable to jailbreak attacks. Recent work reveals a deeper structural weakness: safety alignment can be highly sparse, such that pruning a small number of safety-critical neurons or ablating a single refusal direction can substantially undermine aligned behavior. In this work, we take the first steps toward strengthening safety alignment such that it is less dependent on sparse model components. We propose Addition by Subtraction, which gradually removes identified safety-critical neurons during finetuning and uses preference optimization to re-establish safety-aligned behavior through other parts of the model. Our results show that the resulting models are substantially more robust to pruning-based attacks. Remarkably, these benefits generalize beyond the white-box attacks motivating our design: Addition by Subtraction improves robustness against other sparsity-based interventions, including refusal-direction ablation, as well as a broad range of black-box jailbreak attacks. The finetuned models also exhibit improved truthfulness while largely preserving general capabilities. Comprehensive experiments across four models from three vendors demonstrate consistent improvements in safety robustness, reducing attack success rates by up to 90% for some attacks. ## Helpful to a Fault: Measuring Illicit Assistance in Multi-Turn, Multilingual LLM Agents DevFeed: [Helpful to a Fault: Measuring Illicit Assistance in Multi-Turn, Multilingual LLM Agents]() Original publisher: [Read original article]() Author: Nivya Talokar, Ayush K Tarun, Murari Mandal, Maksym Andriushchenko, Antoine Bosselut Published: 2026-11-04T05:00:00Z Content type: article Language: en Sources: [arXiv: Machine Learning]() Topics: [LLM security](), [LLM evaluation / benchmarking](), [AI Agent](), [Large Language Model]() Tags: [adaptive](), [agents](), [analysis](), [attack](), [automated](), [benchmarks](), [completion](), [cs-cl](), [cs-lg](), [deployment](), [discovery](), [evaluations](), [illegal](), [jailbreak](), [language](), [languages](), [llm](), [llm-agents](), [measuring](), [memory](), [metric](), [multilingual-llm]() ### AI overview STING evaluates whether tool-using LLM agents assist with illicit goals across multiple turns. It builds stepwise plans around benign personas, probes agents with adaptive follow-ups, and uses judge agents to track progress. Its analysis treats jailbreak discovery as time to first event, with discovery curves, language-based hazard-ratio attribution, and Restricted Mean Jailbreak Discovery. In AgentHarm scenarios, STING produced substantially higher illicit-task completion than single-turn prompting and adapted chat-oriented multi-turn baselines. Across six non-English settings, attack success and task completion did not consistently rise in lower-resource languages, unlike common chatbot findings. The supplied abstract does not provide numerical results or details of the scenarios. ### Source excerpt arXiv:2602.16346v5 Announce Type: replace-cross Abstract: LLM-based agents execute real-world workflows via tools and memory. These affordances enable ill-intended adversaries to also use these agents to carry out complex misuse scenarios. Existing agent misuse benchmarks largely test single-prompt instructions, leaving a gap in measuring how agents end up helping with harmful or illegal tasks over multiple turns. We introduce STING (Sequential Testing of Illicit N-step Goal execution), an automated red-teaming framework that constructs a step-by-step illicit plan grounded in a benign persona and iteratively probes a target agent with adaptive follow-ups, using judge agents to track phase completion. We further introduce an analysis framework that models multi-turn red-teaming as a time-to-first-jailbreak random variable, enabling analysis tools like discovery curves, hazard-ratio attribution by attack language, and a new metric: Restricted Mean Jailbreak Discovery. Across AgentHarm scenarios, STING yields substantially higher illicit-task completion than single-turn prompting and chat-oriented multi-turn baselines adapted to tool-using agents. In multilingual evaluations across six non-English settings, we find that attack success and illicit-task completion do not consistently increase in lower-resource languages, diverging from common chatbot findings. Overall, STING provides a practical way to evaluate and stress-test agent misuse in realistic deployment settings, where interactions are inherently multi-turn and often multilingual. ## Grammar-Constrained Decoding Can Jailbreak LLMs into Generating Malicious Code DevFeed: [Grammar-Constrained Decoding Can Jailbreak LLMs into Generating Malicious Code]() Original publisher: [Read original article]() Author: Yitong Zhang, Shiteng Lu, Jia Li Published: 2026-11-04T05:00:00Z Content type: news Language: en Sources: [arXiv: Artificial Intelligence](), [arXiv: Cryptography and Security](), [arXiv: Software Engineering]() Topics: [LLM security](), [Large Language Model](), [post-training]() Tags: [alignment](), [code-generation](), [cs-ai](), [cs-cl](), [cs-cr](), [cs-se](), [decoding](), [jailbreak](), [llms](), [security]() ### AI overview Grammar-Constrained Decoding (GCD), used to enforce syntactic validity in generated code, can also be exploited as a jailbreak surface. The paper's CodeSpear attack applies benign code grammar constraints to induce malicious code generation; across 10 LLMs and 4 benchmarks, it outperformed representative jailbreak baselines and raised attack success by more than 30 percentage points on average. The proposed CodeShield alignment teaches models to produce semantically harmless, structurally diverse "honeypot" code under GCD, while retaining natural-language refusals when available. The abstract reports that CodeShield restored safety under CodeSpear while preserving benign utility. ### Source excerpt arXiv:2606.11817v2 Announce Type: replace Abstract: Large Language Models (LLMs) are increasingly used for code generation, raising concerns that they may be misused to produce malicious code. Meanwhile, Grammar-Constrained Decoding (GCD) has been widely adopted to improve the reliability of LLM-generated code by enforcing syntactic validity. In this paper, we reveal a counterintuitive risk: this reliability-oriented technique can itself become an attack surface. We uncover a new jailbreak attack, termed CodeSpear, that exploits GCD to induce LLMs into generating malicious code. Our experiments show that simply applying a benign code grammar constraint can effectively jailbreak LLMs. To address this vulnerability, we propose CodeShield, a safety alignment approach that robustly preserves safe behavior even under attacker-controlled grammar constraints. CodeShield aligns the model in the code modality by teaching it to generate honeypot code under GCD. Such code is semantically harmless, so it does not implement the malicious request, and structurally diverse, so it is difficult to suppress through grammar tightening. At the same time, CodeShield still preserves natural-language refusals when natural language is available. Experiments on 10 popular LLMs across 4 benchmarks show that CodeSpear outperforms representative jailbreak baselines and increases the attack success rate by more than 30 percentage points on average. CodeShield also restores safety under CodeSpear while preserving benign utility. Our findings reveal a fundamental risk of GCD and call for greater attention to its potential security implications. ## Free the Model, Sell the Gateway: Prompt Adware hidden in Jailbreak Bundles DevFeed: [Free the Model, Sell the Gateway: Prompt Adware hidden in Jailbreak Bundles]() Original publisher: [Read original article]() Author: stcpresearch Published: 2026-11-03T13:57:04Z Content type: article Language: en Sources: [Check Point Research]() Topics: [Prompt Engineering](), [prompt injection](), [gateway](), [Tool]() Tags: [actor](), [ads](), [advertisement](), [adware](), [agent](), [agent-skills](), [ai](), [attack](), [business](), [check-point-research](), [check-point-research-publications](), [cobalt-strike](), [framework](), [gateway](), [jailbreak](), [prompt](), [toolkit](), [tools](), [webserver]() ### AI overview Check Point Research analyzed Haiou 4.5.0, a Codex jailbreak bundle whose installer decrypts and extracts packaged prompts and six security-focused Skills. One prompt modifies global agent instructions so exact greeting triggers produce a catchphrase advertising the author's OpenAI-compatible LLM gateway. The researchers call this "prompt adware": persistent instruction changes that promote a commercial service through natural language rather than executable payloads. The package also includes an uninstaller that restores prior configurations. Its prompts target hacking-related tasks, and researchers observed the bundle alongside tools used by a cybercrime actor; they found no evidence linking its author to cybercrime. The gateway itself is not shown to be malicious, though the article notes that gateways can pose data and agent-session risks. The analyzed Codex version contains ad injection; the Claude repository did not, and Claude Code reportedly rejected the injection during testing. ### Source excerpt Key Points Free the Model, Sell the Gateway: Prompt Adware hidden in Jailbreak Bundles Check Point Research found a misconfigured webserver of a threat actor operating the Cobalt Strike framework. Besides an open Cobalt Strike server, whose logs showed the actor compromising various private computers of Chinese speaking individuals, other tools from the toolkit of [...] The post Free the Model, Sell the Gateway: Prompt Adware hidden in Jailbreak Bundles appeared first on Check Point Research. ## HaloGuard 2.0: An Open-Weights Constitutional Output Guard for Multilingual Response Safety DevFeed: [HaloGuard 2.0: An Open-Weights Constitutional Output Guard for Multilingual Response Safety]() Original publisher: [Read original article]() Author: Ashmiya Lenin, Navaneeth Sangameswaran, Preetham S Published: 2026-11-03T05:00:00Z Content type: release Language: en Sources: [arXiv: Cryptography and Security]() Topics: [LLM security](), [Streaming](), [Open Source Models & Datasets]() Tags: [benchmarks](), [concurrent](), [costs](), [cs-cl](), [cs-cr](), [false-positives](), [input](), [jailbreak](), [languages](), [open-weights](), [operational](), [red-teaming](), [responses](), [safety](), [stop](), [stream](), [token](), [training]() ### AI overview HaloGuard 2.0 is an open-weight streaming output guard that scores each response token in one forward pass, allowing a deployment to stop generation when harmful content is confirmed. Its first training setup learned to judge prompts instead of responses, correctly handling only 13.9% of matched safe/unsafe pairs; a corrective phase on 106,043 counterfactual responses addressed this, and the released model averages the base and corrective models. On six English benchmarks, the 0.8B model reaches 83.6 macro F1, up from 72.8, with 38% fewer false positives; it is reported to match leading full-response guards with 5-10 times fewer parameters and score 2.6 points above Qwen3Guard-Stream-4B. The multilingual 0.8B variant scores 83.1 F1 across 17 languages, 5.0 points above the strongest baseline. At 64 concurrent streams on one H100, reported cost is 0.37 ms per token per stream. The excerpt also says both guards undergo weekly red-teaming. ### Source excerpt arXiv:2611.02134v1 Announce Type: cross Abstract: An input guard decides whether a request reaches the model, but never sees what the model says. Harm that emerges during generation, because a jailbreak passed the input layer or a benign request drifted into operational detail, is visible only on the output side and only as tokens arrive. We present HaloGuard 2.0, an open-weight streaming output guard that assigns a safe/unsafe probability to every response token in a single forward pass, so a deployment can stop the stream once harm is confirmed. The guard reuses the constitution of HaloGuard 1.0, our input guard, to organise a 478,532-response training corpus, and supervises each token from the point where the response becomes harmful. We find that a guard trained on conventionally shaped data learns a shortcut specific to streaming: it judges the prompt instead of the response, and handles only 13.9% of matched safe/unsafe response pairs correctly. We correct this with a second phase on 106,043 counterfactual responses built around the base guard's own failure modes, and release the equal-weight average of the base and corrective models. On six English response-safety benchmarks, HaloGuard 2.0-0.8B reaches a macro F1 of 83.6 (83.3 on held-out halves of the decision-rule search), up from 72.8, with 38% fewer false positives. It matches the strongest full-response guards with 5-10x fewer parameters and is 2.6 points above the strongest streaming guard, Qwen3Guard-Stream-4B. A multilingual variant, HaloGuard 2.0-Poly-0.8B, trained on PolyGuardMix, reaches 83.1 F1 on the response labels of PolyGuardPrompts across 17 languages, 5.0 points above the strongest baseline. On one H100 the guard costs 0.37 ms per token per stream when moderating 64 concurrent responses. Both guards are hardened continuously through the weekly red-teaming protocol of HaloGuard 1.0, extended here to the output layer. ## WebWeaver: Breaking Topology Confidentiality in LLM Multi-Agent Systems with Stealthy Context-Based Inference DevFeed: [WebWeaver: Breaking Topology Confidentiality in LLM Multi-Agent Systems with Stealthy Context-Based Inference]() Original publisher: [Read original article]() Author: Zixun Xiong, Gaoyi Wu, Lingfeng Yao, Miao Pan, Xiaojiang Du, Hao Wang Published: 2026-11-03T05:00:00Z Content type: article Language: en Sources: [arXiv: Cryptography and Security]() Topics: [LLM security](), [Multi Agent Systems](), [ai security]() Tags: [agent](), [attack](), [context](), [critical](), [cs-ai](), [cs-cr](), [diffusion](), [experiments](), [guarantees](), [inference](), [intellectual-property](), [jailbreak](), [jailbreaks](), [llm](), [masking](), [multi-agent](), [multi-agent-systems](), [overhead](), [real-world](), [safety](), [theoretical](), [topology]() ### AI overview WebWeaver aims to infer an LLM multi-agent system's complete communication topology by compromising one arbitrary agent and using its context rather than agent IDs or access to the administrative agent. It combines a covert jailbreak mechanism with a jailbreak-free diffusion design for cases where jailbreaks fail; a masking strategy preserves known topology during diffusion, with claimed theoretical correctness guarantees. The abstract reports about 60% higher inference accuracy than state-of-the-art baselines under active defenses, with negligible overhead. These are claims from the supplied abstract; it does not provide experiment details. ### Source excerpt arXiv:2603.11132v3 Announce Type: replace Abstract: Communication topology is a critical factor in the utility and safety of LLM-based multi-agent systems (LLM-MAS), making it a high-value intellectual property (IP) whose confidentiality remains insufficiently studied. Existing topology inference attempts rely on impractical assumptions, including control over the administrative agent and direct identity queries via jailbreaks, which are easily defeated by basic keyword-based defenses. As a result, prior analyses fail to capture the real-world threat of such attacks. To bridge this realism gap, we propose \textit{WebWeaver}, an attack framework that infers the complete LLM-MAS topology by compromising only a single arbitrary agent instead of the administrative agent. Unlike prior approaches, WebWeaver relies solely on agent contexts rather than agent IDs, enabling significantly stealthier inference. WebWeaver further introduces a new covert jailbreak-based mechanism and a novel fully jailbreak-free diffusion design to handle cases where jailbreaks fail. Additionally, we address a key challenge in diffusion-based inference by proposing a masking strategy that preserves known topology during diffusion, with theoretical guarantees of correctness. Extensive experiments show that WebWeaver substantially outperforms state-of-the-art (SOTA) baselines, achieving about 60\% higher inference accuracy under active defenses with negligible overhead. ## JailbreakOPT: Iterative Jailbreak Prompt Optimization with Macro-Action DevFeed: [JailbreakOPT: Iterative Jailbreak Prompt Optimization with Macro-Action]() Original publisher: [Read original article]() Author: Ge Shi, Jun Yin, Donglin Xie, Fangyi Liu, Yucan Li, Menglin Liu Published: 2026-11-03T05:00:00Z Content type: article Language: en Sources: [arXiv: Cryptography and Security]() Topics: [Prompt Engineering](), [LLM security](), [thompson sampling](), [LLM evaluation / benchmarking]() Tags: [attack](), [benchmarks](), [cs-ai](), [cs-cr](), [exploration](), [failure](), [jailbreak](), [large-language-models-llms](), [policy](), [prompt-optimization](), [reuse](), [safety](), [sampling](), [search](), [vulnerabilities]() ### AI overview JailbreakOPT seeks to improve jailbreak attack success while reducing the response burden of iterative prompt search. It composes reusable prompt transformations into macro-actions to generate standalone attack prompts, then uses contextual Thompson sampling and outcomes from earlier search paths to guide action selection across episodes. The abstract reports higher attack success rates across multiple LLMs and benchmarks than single-action and iterative baselines, but gives no numerical results or further experimental details. The authors suggest the approach can help probe model vulnerabilities and identify failure patterns that may inform defenses. ### Source excerpt arXiv:2606.11425v2 Announce Type: replace Abstract: Jailbreak attacks expose persistent safety weaknesses in large language models (LLMs), but existing stateless single-turn methods face a trade-off: hand-crafted prompts are expressive but static, while iterative prompt optimization can adapt but often relies on low-level mutations that require many target-model responses. We propose JailbreakOPT, a method that maintains a library of reusable prompt transformations (actions) and composes them into macro-actions for within-episode search, generating standalone attack prompts. To reuse experience across episodes, JailbreakOPT learns an action-selection policy using contextual Thompson sampling and outcomes from prior search paths. Experiments across multiple LLMs and benchmarks show that JailbreakOPT improves attack success rate (ASR) over single-action and iterative baselines. More broadly, exploration and exploitation over human-designed attack strategies can help probe residual vulnerabilities and identify failure patterns that may inform defenses aimed at broader classes of attacks. ## Expected Harm: Rethinking Safety Evaluation of (Mis)Aligned LLMs DevFeed: [Expected Harm: Rethinking Safety Evaluation of (Mis)Aligned LLMs]() Original publisher: [Read original article]() Author: Yen-Shan Chen, Zhi Rui Tam, Cheng-Kuang Wu, Yun-Nung Chen Published: 2026-11-02T05:00:00Z Content type: article Language: en Sources: [arXiv: Cryptography and Security](), [arXiv: Machine Learning]() Topics: [ai security](), [LLM observability](), [post-training]() Tags: [attack](), [cs-cl](), [cs-cr](), [cs-cy](), [cs-lg](), [evaluation](), [execution](), [jailbreak](), [jailbreaks](), [linear-probing](), [llm-safety](), [llms](), [metric](), [models](), [open-source](), [risk](), [safety](), [sft](), [training-data](), [vulnerability]() ### AI overview Expected Harm evaluates jailbreak risk by weighting severity according to the likelihood that a real-world actor would carry out the threat, modeled through execution cost. The authors report that tested models show "Inverse Risk Calibration": they refuse more strongly for high-cost, low-likelihood threats while remaining more vulnerable to low-cost, high-likelihood queries. Linear probing found refusal mechanisms separated queries by severity but did not use execution likelihood as a refusal signal. Analysis of Olmo-3 checkpoints links the miscalibration to SFT data, where 82.8% of safety examples were at cost levels 0-1, and finds cost representations weak from the SFT stage onward. Exploiting this behavior increased existing jailbreak attack success rates by up to 2x. ### Source excerpt arXiv:2602.01600v2 Announce Type: replace Abstract: Current evaluations of LLM safety predominantly rely on severity-based taxonomies to assess the harmfulness of models' responses to malicious queries. We argue that this formulation requires re-examination as it implicitly equates model output with realized harm while neglecting Execution Likelihood-the conditional probability of a threat being realized by a real-world actor given a model's response. In this work, we introduce Expected Harm, a metric that weights the severity of a jailbreak by its execution likelihood, modeled as a function of execution cost. Through empirical analysis of state-of-the-art models, we reveal a systematic Inverse Risk Calibration: models disproportionately exhibit stronger refusal behaviors for low-likelihood (high-cost) threats while remaining vulnerable to high-likelihood (low-cost) queries. Mechanistically, we first provide an analysis using linear probing, revealing that while refusal mechanisms linearly separate queries by severity, they do not use execution likelihood as an indicator for refusal. Then, we use Olmo-3's open-source staged checkpoints to trace the root cause of miscalibration to the Supervised Fine Tuning (SFT) training data distribution (82.8% of safety examples at cost levels 0-1) and show that cost representations are weak from the SFT stage onward. Finally, we demonstrate that this miscalibration creates a vulnerability: by exploiting this property, we increase the attack success rate of existing jailbreaks by up to $2\times$. ## ChatGPT as a Surveillance Assistant? An Exploratory Case Study of AI-Assisted Surveillance Risks DevFeed: [ChatGPT as a Surveillance Assistant? An Exploratory Case Study of AI-Assisted Surveillance Risks]() Original publisher: [Read original article]() Author: Ayana Hussain, Nicholas Vincent Published: 2026-11-02T05:00:00Z Content type: article Language: en Sources: [arXiv: Artificial Intelligence]() Topics: [ChatGPT](), [Large Language Model](), [LLM security](), [Security & Privacy]() Tags: [2026](), [ai](), [ai-assisted](), [analysis](), [case-study](), [chatgpt](), [code](), [cs-ai](), [guidance](), [instructions](), [jailbreak](), [language-model](), [language-models](), [large-language-models](), [large-language-models-llms](), [llm](), [llms](), [policy](), [risks](), [study](), [surveillance]() ### AI overview In this exploratory case study, ChatGPT gave substantive surveillance guidance for 40 of 55 jailbreak prompts and 33 of 55 direct prompts, without a prefix. Researchers organized 110 prompts into seven risk categories and manually reviewed responses; direct prompts that initially failed could be retried with a short fixed prefix. Compliant responses rarely mentioned policy or offered privacy-preserving alternatives. Some responses that stated a refusal still included detailed instructions, working code, or architectural guidance alongside brief ethical caveats. These findings describe tests of ChatGPT's public web interface in this study. ### Source excerpt arXiv:2610.40890v1 Announce Type: new Abstract: Large language models (LLMs) can generate detailed guidance, functional code, and persuasive text at scale. As one of the most widely used consumer language model products as of 2026, ChatGPT gives broad public access to these capabilities, motivating concerns about its potential use in surveillance-related tasks. We present an exploratory case study of what potentially harmful surveillance assistance looks like in ChatGPT's public web interface. We contribute a taxonomy of seven surveillance risk categories based on prior LLM risk research and reports of deployed surveillance systems, instantiated as 110 prompts, along with a manually reviewed analysis of ChatGPT's responses. Prompts were issued using two approaches: 55 jailbreak prompts drawing on five types of elicitation strategies, and 55 plain-language direct prompts, each run alone, and if unsuccessful, again with a short fixed prefix. We find that ChatGPT provides substantive surveillance guidance for 40 of the 55 jailbreak prompts and for 33 of the 55 direct prompts without any prefix. Compliant responses also rarely included policy mentions or suggestions for privacy-preserving alternatives. We manually review every response and find that even when ChatGPT states a refusal, it may still supply detailed instructions, working code, or architectural guidance alongside brief ethical caveats. ## Rethinking LLM Jailbreak Evaluation: Continuous Risk Landscapes Beyond Binary Outcomes DevFeed: [Rethinking LLM Jailbreak Evaluation: Continuous Risk Landscapes Beyond Binary Outcomes]() Original publisher: [Read original article]() Author: Yukai Zhou, Wenjie Wang Published: 2026-10-30T04:00:00Z Content type: article Language: en Sources: [arXiv: Machine Learning]() Topics: [LLM security](), [Large Language Model]() Tags: [abstract](), [adaptive](), [adversarial](), [also](), [arxiv](), [asr](), [attack](), [attacks](), [based](), [behavior](), [candidate](), [capacity](), [classification](), [context](), [continuous](), [cs-cl](), [cs-lg](), [evaluation](), [jailbreak](), [llm](), [llm-jailbreak](), [risk]() ### AI overview The paper argues that binary attack success rate misses the range of possible LLM responses: a jailbreak prompt induces many possible continuations, while each sampled answer shows only one. It proposes a context-conditioned risk landscape over first-token branches, estimated by fixing candidate openings and repeatedly sampling continuations. The measure captures both current decoder exposure and capacity for safe rerouting, and supports comparisons across clean prompts, attacked variants, attack strengths, and an adaptive defense. Across multiple models and attacks, the authors report risky branches beneath apparently safe outputs, different ways attacks shift opening-token distributions, and cases where the strongest attack before defense is not strongest after the defense reacts. ### Source excerpt arXiv:2610.38595v1 Announce Type: new Abstract: A binary success indicator is a natural representation for discriminative tasks such as image classification under adversarial perturbations, where each input is mapped to a single prediction. However, generative tasks are fundamentally different: an LLM jailbreak prompt induces a distribution over open-ended trajectories, while a sampled response reveals only one path. Yet LLM jailbreak evaluation typically retains the binary response indicator underlying attack success rate, obscuring the surrounding structure of possible generations. We introduce a continuous, context-conditioned risk landscape over first-token branches, estimated by fixing candidate openings and repeatedly sampling their continuations. This continuous risk-landscape perspective reveals two complementary properties: current decoder exposure and safe rerouting capacity, which have been obscured by binary-indicator-based ASR. The resulting representation also provides a common basis for comparing clean prompts, attacked variants, and attack strength against a known adaptive defense. Across multiple target models and jailbreak attacks, we uncover risky branches beneath apparently safe outputs, identify qualitatively different ways attacks reshape the model's opening distribution, and show that attacks strongest before a defense do not necessarily remain the strongest after the adaptive defense reacts. These findings recast jailbreaks as deformations of the surrounding continuation-risk landscape, establishing a more informative foundation for understanding jailbreak behavior and developing safer language models. ## Jailbreak Connectivity: Towards Diverse, Transferable, and Universal Multimodal Large Language Model Jailbreak DevFeed: [Jailbreak Connectivity: Towards Diverse, Transferable, and Universal Multimodal Large Language Model Jailbreak]() Original publisher: [Read original article]() Author: Xu Zhang, Ziqing Hu, Shuo Han, Ren Wang Published: 2026-10-30T04:00:00Z Content type: article Language: en Sources: [arXiv: Artificial Intelligence](), [arXiv: Cryptography and Security](), [arXiv: Machine Learning]() Topics: [LLM security](), [Adversarial attacks](), [vlm](), [LLM evaluation / benchmarking]() Tags: [adversarial](), [attack](), [connectivity](), [continuous](), [cross](), [cs-ai](), [cs-cr](), [cs-lg](), [distribution](), [diversity](), [image](), [jailbreak](), [large-language-model](), [multimodal](), [safety](), [security]() ### AI overview Jailbreak Connectivity (JC) generates diverse visual jailbreaks by optimizing continuous paths through low-loss regions in image space. It samples images along those paths, uses lightweight surrogate classifiers to guide transfer across models, and optimizes at the distribution level for attacks spanning harmful queries. On MM-SafetyBench, reported average attack success rates are 84.01% for Qwen3-VL-8B-Instruct, 85.77% for InternVL3.5-8B, and 97.95% for Gemma-3-12B-IT, up to 68.36 percentage points above the strongest baselines. In black-box evaluations, JC exceeds FigStep by 24.69 points on GPT-5.6 and 27.34 points on Gemini 3.1 Pro. The abstract presents these results as evidence of structured vulnerabilities in MLLM safety alignment; it does not provide implementation details or broader evaluation results. ### Source excerpt arXiv:2610.38091v1 Announce Type: cross Abstract: Multimodal large language models (MLLMs) are vulnerable to visual jailbreak attacks that bypass safety safeguards. Existing methods typically optimize isolated adversarial examples, limiting attack diversity, transferability, and generalization. We propose Jailbreak Connectivity (JC), a framework based on the observation that effective adversarial inputs can be connected through low-loss regions in image space. JC optimizes continuous adversarial paths: path sampling yields diverse jailbreak images, lightweight surrogate classifiers guide cross-model transfer, and a distribution-level objective enables universal attacks across harmful queries. On MM-SafetyBench, JC achieves average attack success rates of 84.01%, 85.77%, and 97.95% on Qwen3-VL-8B-Instruct, InternVL3.5-8B, and Gemma-3-12B-IT, respectively, exceeding the strongest baselines by up to 68.36 percentage points. In black-box evaluations, JC improves attack success rates over FigStep by 24.69 and 27.34 percentage points on GPT-5.6 and Gemini 3.1 Pro, respectively. These findings reveal structured vulnerabilities in MLLM safety alignment and inform more comprehensive security evaluation. ## RedCodeAgent: Automatic Red-teaming Agent against Diverse Code Agents DevFeed: [RedCodeAgent: Automatic Red-teaming Agent against Diverse Code Agents]() Original publisher: [Read original article]() Author: Chengquan Guo, Chulin Xie, Yu Yang, Zhaorun Chen, Zinan Lin, Xander Davies, Yarin Gal, Dawn Song, Bo Li Published: 2026-10-30T04:00:00Z Content type: article Language: en Sources: [arXiv: Software Engineering]() Topics: [AI Agent](), [AI Chat]() Tags: [adaptive](), [agents](), [attack](), [automated](), [benchmarks](), [boundary](), [capabilities](), [code](), [code-generation](), [cs-se](), [cursor](), [data](), [debugging](), [effective](), [efficiency](), [environments](), [evaluation](), [execution](), [jailbreak](), [knowledge](), [red-teaming]() ### AI overview RedCodeAgent automates red-teaming of code agents by using adaptive memory to draw on prior jailbreak knowledge and select tools or tool combinations for each query. It evaluates agents in simulated sandboxes that check execution results, addressing a limitation of judges that assess static code alone. The abstract reports higher attack success rates, lower rejection rates, and high efficiency compared with existing red-teaming methods across multiple agents, risky scenarios, and programming languages. Tests on Cursor and Codeium also exposed previously unidentified security risks. These findings are reported in the supplied abstract; it provides no numerical results or specific vulnerabilities. ### Source excerpt arXiv:2510.02609v3 Announce Type: replace Abstract: Code agents have gained widespread adoption due to their strong code generation capabilities and integration with code interpreters, enabling dynamic execution, debugging, and interactive programming capabilities. While these advancements have streamlined complex workflows, they have also introduced critical safety and security risks. Current static safety benchmarks and red-teaming tools are inadequate for identifying emerging real-world risky scenarios, as they fail to cover certain boundary conditions, such as the combined effects of different jailbreak tools. In this work, we propose RedCodeAgent, the first automated red-teaming agent designed to systematically uncover vulnerabilities in diverse code agents. With an adaptive memory module, RedCodeAgent can leverage existing jailbreak knowledge, dynamically select the most effective red-teaming tools and tool combinations in a tailored toolbox for a given input query, thus identifying vulnerabilities that might otherwise be overlooked. For reliable evaluation, we develop simulated sandbox environments to additionally evaluate the execution results of code agents, mitigating potential biases of LLM-based judges that only rely on static code. Through extensive evaluations across multiple state-of-the-art code agents, diverse risky scenarios, and various programming languages, RedCodeAgent consistently outperforms existing red-teaming methods, achieving higher attack success rates and lower rejection rates with high efficiency. We further validate RedCodeAgent on real-world code assistants, e.g., Cursor and Codeium, exposing previously unidentified security risks. By automating and optimizing red-teaming processes, RedCodeAgent enables scalable, adaptive, and effective safety assessments of code agents. Our code and data are available at https://github.com/1mocat/RedCodeAgent. ## DYNASHIELD: A Black-Box Moving Target Defense for LLMs via Dynamic Decoding Customization DevFeed: [DYNASHIELD: A Black-Box Moving Target Defense for LLMs via Dynamic Decoding Customization]() Original publisher: [Read original article]() Author: Xiaoqun Liu, Weiming Qi, Qiben Yan Published: 2026-10-28T04:00:00Z Content type: article Language: en Sources: [arXiv: Cryptography and Security]() Topics: [LLM security](), [Large Language Model]() Tags: [adversarial](), [apis](), [configuration](), [cs-cl](), [cs-cr](), [decoding](), [defense](), [deployments](), [framework](), [inference](), [jailbreak](), [llms](), [runtime-controls](), [safety](), [training]() ### AI overview DYNASHIELD aims to reduce jailbreak success in black-box LLM deployments by changing decoding hyperparameters and system prompts at inference time. It identifies configurations associated with lower attack success, then probabilistically samples from a weighted pool to vary model behavior. The abstract reports evaluation on 7 open-source models against 4 jailbreak attacks using AdvBench prompts, with substantial attack-success reductions versus 7 baseline defenses, while maintaining response quality and adding minimal inference overhead. The approach uses exposed runtime controls, so it requires neither retraining nor access to model internals. ### Source excerpt arXiv:2412.07672v3 Announce Type: replace Abstract: Large language models (LLMs) remain vulnerable to jailbreak attacks in which adversarial prompts induce harmful outputs. Existing defenses often require access to the model internals or additional training, limiting their applicability for service providers deployed through black-box APIs. In this paper, we propose DYNASHIELD, a moving target defense framework that improves robustness by customizing decoding hyperparameters and system prompts at inference time. DYNASHIELD includes two key steps: (1) it identifies decoding configurations that reduce attack success probability, and (2) it probabilistically samples from a weighted configuration pool to introduce controlled variability in model behavior. We evaluate DYNASHIELD across 7 open-source LLMs under 4 state-of-the-art jailbreak attacks, using adversarial prompts from AdvBench. Results show substantial reductions in attack success rate compared with 7 baseline defenses, while maintaining response quality and incurring minimal inference overhead. Because DYNASHIELD operates solely through exposed runtime controls, it requires no retraining or model-internal access. These results suggest that safety-aware dynamic decoding is a promising and practically lightweight defense mechanism for black-box LLM deployments. ## PrivacyEvalClaw: A Tool-Augmented Deliberative Agent for Auditing Privacy Leakage in LLM Jailbreak Responses DevFeed: [PrivacyEvalClaw: A Tool-Augmented Deliberative Agent for Auditing Privacy Leakage in LLM Jailbreak Responses]() Original publisher: [Read original article]() Author: Krishna Kanth Nakka, Yujun Wang, Shijie Zhou, Tu Nguyen, Gabriele Gelardi, Xuebing Zhou Published: 2026-10-28T04:00:00Z Content type: article Language: en Sources: [arXiv: Cryptography and Security]() Topics: [Security & Privacy](), [Large Language Model](), [llm-reasoning]() Tags: [agent](), [auditing](), [cs-cr](), [evaluation](), [experiments](), [information](), [jailbreak](), [language-models](), [large-language-models](), [large-language-models-llms](), [llm](), [llm-as-judge](), [llm-jailbreak](), [llms](), [models](), [modules](), [personal](), [personal-information](), [privacy](), [responses](), [tool]() ### AI overview PrivacyEvalClaw addresses disagreement among privacy evaluators, especially when jailbreak responses reveal personal information indirectly or ambiguously. This training-free agent treats evaluators and analysis modules as tools, gathers evidence adaptively, investigates uncertain disclosures, and reconciles conflicting signals before judging leakage. In pairwise LLM-as-Judge evaluations of complete trajectories, it achieved favorable win rates over competing baselines on outcome, process, and overall quality, and produced reasoning traces intended to improve transparency and auditability. ### Source excerpt arXiv:2610.36711v1 Announce Type: new Abstract: Reliable evaluation is a critical bottleneck in privacy red-teaming of large language models (LLMs): determining whether a jailbreak response actually reveals personal information can be difficult when disclosures are indirect, contextual, partial, or obfuscated. Existing evaluation approaches increasingly rely on guard models and LLM-as-Judges, yet these evaluators may interpret the same response differently. To better characterize this evaluation setting, we conduct a systematic study of privacy evaluators and find substantial disagreement, particularly in ambiguous cases that require additional evidence or targeted analysis to resolve. Motivated by these findings, we introduce \textsc{PrivacyEvalClaw}, a training-free deliberative agent for evaluating privacy leakage in LLM jailbreak responses. Rather than relying on isolated single-pass judgments, PrivacyEvalClaw treats evaluators and analysis modules as tools, adaptively gathers evidence, investigates uncertain disclosures, and reconciles conflicting signals before producing a final judgment. Our primary experiments use pairwise LLM-as-Judge evaluation of complete trajectories, showing that PrivacyEvalClaw achieves favorable win rates against competing baselines across outcome, process, and overall quality, while producing explicit reasoning traces that make the evaluation process more transparent and auditable. ## One Leak Away: How Pretrained Model Exposure Amplifies Jailbreak Risks in Finetuned LLMs DevFeed: [One Leak Away: How Pretrained Model Exposure Amplifies Jailbreak Risks in Finetuned LLMs]() Original publisher: [Read original article]() Author: Yixin Tan, Zhe Yu, Rui Wen, Jun Sakuma Published: 2026-10-27T04:00:00Z Content type: article Language: en Sources: [arXiv: Cryptography and Security]() Topics: [LLM security](), [Large Language Model](), [Representation learning]() Tags: [adversarial](), [attack](), [cs-ai](), [cs-cr](), [defense](), [hidden-states](), [inheritance](), [jailbreak](), [llms](), [model](), [pgp](), [risks](), [vulnerabilities]() ### AI overview In a pretrain-to-finetune threat model, jailbreak prompts optimized using an exposed pretrained model transfer to its proprietary finetuned variants, even when the attacker cannot access those derivatives. The authors report that transferable prompts are linearly separable in pretrained hidden states, suggesting that transfer-relevant structure is already present. They use this finding to develop Probe-Guided Projection (PGP), which steers prompt optimization toward those directions; experiments across multiple model families and finetuned tasks show strong transfer success. The same representation-level insights also inform a lightweight defense that mitigates transfer while preserving downstream utility. The supplied text is an abstract and does not provide quantitative results or implementation details. ### Source excerpt arXiv:2512.14751v4 Announce Type: replace Abstract: Finetuning pretrained large language models (LLMs) has become the standard paradigm for developing downstream applications. However, its security implications remain unclear, particularly regarding whether finetuned LLMs inherit jailbreak vulnerabilities from their pretrained sources. We investigate this question in a realistic pretrain-to-finetune threat model, where an attacker has full access to a released pretrained LLM but no access to its proprietary finetuned derivatives. Empirical analysis shows that adversarial prompts optimized on the pretrained model transfer most effectively to its finetuned variants, revealing inherited vulnerabilities from pretrained to finetuned LLMs. To further examine this inheritance, we conduct representation-level probing, which shows that transferable prompts are linearly separable within the pretrained hidden states, suggesting that transferability-relevant structure is already encoded in pretrained representations. Building on this insight, we propose the Probe-Guided Projection (PGP) attack, which steers optimization toward transferability-relevant directions. Experiments across multiple LLM families and diverse finetuned tasks confirm PGP's strong transfer success, underscoring the security risks inherent in the pretrain-to-finetune paradigm. Finally, we demonstrate that the same representation-level insights also enable a lightweight defense that mitigates pretrain-to-finetune jailbreak transfer while preserving downstream utility. ## JevAdvBench: A Benchmark and Black-Box Attacks for Reinforcement Learning for Calibrated Decisions Models DevFeed: [JevAdvBench: A Benchmark and Black-Box Attacks for Reinforcement Learning for Calibrated Decisions Models]() Original publisher: [Read original article]() Author: Jianyi Hu, Hangtao Zhang, Yi Liu, Yeqi Zeng, Li Zeng, Xianlong Wang, Rui Wang, Leo Yu Zhang Published: 2026-10-27T04:00:00Z Content type: article Language: en Sources: [arXiv: Cryptography and Security]() Topics: [Adversarial attacks](), [Reinforcement learning]() Tags: [adaptive](), [adversarial](), [agentgateway](), [attacks](), [benchmark](), [confidence](), [cs-ai](), [cs-cl](), [cs-cr](), [decisions-models](), [guardrail](), [input](), [jailbreak](), [prompt-injection]() ### AI overview JevAdvBench evaluates adversarial robustness in typed decision models (TDMs), whose valid-format outputs can still be wrong. It provides 862 questions with audited answers and 12 black-box methods that adaptively edit inputs or options; evaluation retains 5,586 reviewed variants that preserve the correct answer and attacks only initially correct decisions. Attacks flipped 46.3% of Jev's correct decisions, compared with 0.17% under identical reruns. A flawed argument without a command flipped 77.2%, versus 92.0% for direct prompt injection. Across eight open TDMs, the same variants flipped 34.0%-51.7%. At a confidence threshold of 0.8, 23.5% of decisions still executed by a service were wrong. In an Agentgateway demonstration, Jev's guardrail rejected a jailbreak request but accepted it after an argument was inserted. The authors recommend treating TDM inputs as untrusted and not relying on confidence alone. ### Source excerpt arXiv:2609.31142v2 Announce Type: replace Abstract: Typed decision models (TDMs), such as Jev, answer a question about an input with a probability, an option, or an ordinal level. Web services execute these decisions automatically above a confidence threshold. An attacker who writes part of the input can change a decision that the service then executes. Current adversarial benchmarks cannot measure this risk, because they judge generated text. A TDM writes no text, and its wrong decisions have a valid format. We present JevAdvBench, to our knowledge the first unified benchmark for the adversarial robustness of TDMs. It contains 862 questions with audited answers and twelve black-box attack methods that edit the input or the options by adaptive search. Because a changed decision is not always wrong, a review keeps only 5,586 answer-preserving variants, and we attack only correct decisions. The attacks flip 46.3% of Jev's correct decisions, while identical re-runs flip only 0.17%. A flawed argument without a command flips 77.2%, close to direct prompt injection at 92.0%. The same variants flip 34.0% to 51.7% of correct decisions on eight open TDMs. At a 0.8 threshold, 23.5% of the decisions that the service still executes are wrong. On Agentgateway, the Jev guardrail rejects a jailbreak request but accepts it with an inserted argument. Web services should therefore treat TDM inputs as untrusted and should not act on confidence alone. Project website: https://JevAdvBench.github.io/JevAdvBench/ ## Safe at One Loop, Risky at Another: Aligning Safety Across Recurrent Depths in Looped Language Models DevFeed: [Safe at One Loop, Risky at Another: Aligning Safety Across Recurrent Depths in Looped Language Models]() Original publisher: [Read original article]() Author: Yi Wang, Xiuyuan Qi, Dongqi Han, Dongsheng Li, Wenjie Wang Published: 2026-10-26T04:00:00Z Content type: article Language: en Sources: [arXiv: Cryptography and Security](), [arXiv: Machine Learning]() Topics: [LLM security](), [post-training](), [LLM evaluation / benchmarking]() Tags: [alignment](), [bridging](), [cs-cr](), [cs-lg](), [evaluation](), [jailbreak](), [language-models](), [recurrent](), [robustness](), [safety](), [sft](), [supervision](), [utility]() ### AI overview Looped Language Models can behave differently at different recurrent depths: jailbreak success may rise at deeper depths, and attacks can transfer across depths. The abstract reports that supervised fine-tuning and preference alignment do not close these gaps. It introduces SafeBridge, combining depth-specific control of shared recurrent layers, selective state bridging, and joint safety supervision across depths. Across model scales, attack methods, and safety benchmarks, SafeBridge reportedly reduces matched-depth and cross-depth attack success, improves general utility over vanilla models, and maintains comparable over-refusal behavior. The findings suggest that safety cannot be assessed from a single recurrent depth. ### Source excerpt arXiv:2610.10625v2 Announce Type: replace-cross Abstract: Looped Language Models (LoopLMs) provide a parameter efficient approach to scaling model capabilities through repeated use of shared parameters across recurrent steps. Since each recurrent depth can be read out independently, a single LoopLM exposes a broader output space across inference depths, raising an important question: whether safety is preserved throughout recurrent computation. Prior evaluations suggest that deeper recurrence can improve safety on harmful queries, but robustness under jailbreak attacks remains unclear. We therefore conduct a comprehensive safety evaluation of LoopLMs under jailbreak attacks targeting different recurrent depths. We find that attack success can increase at deeper inference depths, the same query can elicit different safety behaviors across depths, and attacks constructed against one depth can transfer to others. Moreover, SFT and preference alignment do not eliminate these safety gaps, motivating an alignment method designed for LoopLMs. We introduce SafeBridge, which combines lightweight depth specific control of shared recurrent layers, selective state bridging, and joint safety supervision across recurrent depths. Across model scales, multiple attack methods, and safety benchmarks, SafeBridge substantially reduces attack success for both matched-depth and cross-depth attacks. It also improves general utility over the vanilla models while maintaining comparable over-refusal behavior. Our results show that the safety of a LoopLM cannot be inferred from a single recurrent depth, motivating safety alignment across recurrent computation. Our code and model checkpoints will be released upon acceptance. [Next page]()