# Adversarial attacks

Adversarial attacks are attacks that exploit vulnerabilities in AI and machine-learning systems, including by manipulating inputs or training data.

This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles.

## On the Vulnerability of Lane Detectors to Physical Shadow Attack

DevFeed: [On the Vulnerability of Lane Detectors to Physical Shadow Attack](<https://devfeed.tech/articles/on-the-vulnerability-of-lane-detectors-to-physical-shadow-attack-145991.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2409.18248>)

Author: Pedram MohajerAnsari (Clemson University, Clemson, SC, USA), Amir Salarpour (Clemson University, Clemson, SC, USA), Jan de Voor (Clemson University, Clemson, SC, USA), Arkajyoti Mitra (University of T

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Cryptography and Security](<https://devfeed.tech/sources/arxiv-cryptography-and-security.md>)

Topics: [Adversarial attacks](<https://devfeed.tech/topics/adversarial-attacks.md>), [autonomous vehicles](<https://devfeed.tech/topics/autonomous-vehicles.md>), [Security](<https://devfeed.tech/topics/security.md>), [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>)

Tags: [adversarial](<https://devfeed.tech/tags/adversarial.md>), [algorithm](<https://devfeed.tech/tags/algorithm.md>), [attack](<https://devfeed.tech/tags/attack.md>), [autonomous](<https://devfeed.tech/tags/autonomous.md>), [autonomous-vehicles](<https://devfeed.tech/tags/autonomous-vehicles.md>), [bias](<https://devfeed.tech/tags/bias.md>), [cs-cr](<https://devfeed.tech/tags/cs-cr.md>), [deep-neural-networks](<https://devfeed.tech/tags/deep-neural-networks.md>), [detection](<https://devfeed.tech/tags/detection.md>), [exploits](<https://devfeed.tech/tags/exploits.md>), [optimize](<https://devfeed.tech/tags/optimize.md>), [patterns](<https://devfeed.tech/tags/patterns.md>), [private](<https://devfeed.tech/tags/private.md>), [property](<https://devfeed.tech/tags/property.md>), [real-world](<https://devfeed.tech/tags/real-world.md>), [security](<https://devfeed.tech/tags/security.md>), [security-vulnerabilities](<https://devfeed.tech/tags/security-vulnerabilities.md>), [simulations](<https://devfeed.tech/tags/simulations.md>), [software](<https://devfeed.tech/tags/software.md>), [study](<https://devfeed.tech/tags/study.md>), [vulnerability](<https://devfeed.tech/tags/vulnerability.md>)

### AI overview

The Negative Shadow attack exploits a shadow-removal intensity bias: bright patterns cast by sunlight through shaped occluders can be mistaken for lane markings by neural lane detectors. A genetic algorithm optimizes these patterns against CLRerNet, HybridNets, and TwinLiteNet. In CARLA-OpenPilot simulations, the attack produced hazard-scenario violation rates of up to 100%; the authors also report a miniature-road testbed and a real-world Comma 3X deployment that caused a lane departure. In a human-subject study, 83.6% of participants failed to detect the patterns, while tested input-preprocessing defenses offered only limited protection.

### Source excerpt

arXiv:2409.18248v3 Announce Type: replace Abstract: Automated Lane Centering (ALC) systems are critical components of Level 2 autonomous vehicles, yet their reliance on Deep Neural Networks (DNNs) for lane detection introduces security vulnerabilities. This work introduces the Negative Shadow (NS) Attack, a novel physical-world adversarial technique that exploits intensity bias in shadow-removal preprocessing, where bright patches within shadowed regions are misclassified as genuine lane markings. Unlike previous attacks requiring illegal road markings, the NS attack is passive and stealthy, utilizing sunlight filtered through shaped occluders placed on private property to cast patterns onto the roadway. We utilize a genetic algorithm to optimize the geometry of these shadows against three state-of-the-art lane detection systems: CLRerNet, HybridNets, and TwinLiteNet. The attack is evaluated across three hazard scenarios -- left-turn deviation, head-on collision, and off-road deviation -- using software-in-the-loop simulations with CARLA-OpenPilot, which demonstrated violation rates of up to 100%. Further validation is provided through a miniature road testbed and a real-world deployment where the attack successfully caused a Comma 3X-equipped vehicle to depart its lane. Additionally, a human-subject study indicates high stealth, with 83.6% of participants failing to detect the adversarial patterns, and an assessment of input-preprocessing defenses shows they offer only limited protection against this threat.

## On the Impact of Feature Reduction Techniques on Adversarial Attacks

DevFeed: [On the Impact of Feature Reduction Techniques on Adversarial Attacks](<https://devfeed.tech/articles/on-the-impact-of-feature-reduction-techniques-on-adversarial-attacks-145002.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.25004>)

Author: Mohammad Meymani, Roozbeh Razavi-Far

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Adversarial attacks](<https://devfeed.tech/topics/adversarial-attacks.md>), [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [Feature Engineering](<https://devfeed.tech/topics/feature-engineering.md>), [ai security](<https://devfeed.tech/topics/ai-security.md>)

Tags: [adversarial](<https://devfeed.tech/tags/adversarial.md>), [adversarial-machine-learning](<https://devfeed.tech/tags/adversarial-machine-learning.md>), [apt](<https://devfeed.tech/tags/apt.md>), [attacks](<https://devfeed.tech/tags/attacks.md>), [backdoor](<https://devfeed.tech/tags/backdoor.md>), [classifiers](<https://devfeed.tech/tags/classifiers.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [defense](<https://devfeed.tech/tags/defense.md>), [effect](<https://devfeed.tech/tags/effect.md>), [f1](<https://devfeed.tech/tags/f1.md>), [feature](<https://devfeed.tech/tags/feature.md>), [features](<https://devfeed.tech/tags/features.md>), [impact](<https://devfeed.tech/tags/impact.md>), [improvements](<https://devfeed.tech/tags/improvements.md>), [information](<https://devfeed.tech/tags/information.md>), [learning](<https://devfeed.tech/tags/learning.md>), [machine-learning](<https://devfeed.tech/tags/machine-learning.md>), [machine-learning-models](<https://devfeed.tech/tags/machine-learning-models.md>), [metrics](<https://devfeed.tech/tags/metrics.md>), [reduction](<https://devfeed.tech/tags/reduction.md>), [techniques](<https://devfeed.tech/tags/techniques.md>)

### AI overview

On the APT dataset, permutation importance ranking and univariate mutual information generally yielded the most consistent robustness improvements across the evaluated classifiers and attacks. The study compares PCA, permutation importance ranking, recursive feature elimination, and univariate mutual information across SVM, decision tree, random forest, MLP, and k-nearest neighbors models, under poisoning, backdoor, and universal adversarial perturbation attacks. It evaluates accuracy, F1-score, and mitigation score; the abstract says results for the other reduction methods vary by classifier and attack scenario.

### Source excerpt

arXiv:2611.25004v1 Announce Type: new Abstract: Adversarial attacks have posed significant threats to machine learning models by exploiting their inherent vulnerabilities. Researchers have established an interdisciplinary field called adversarial machine learning to explore these vulnerabilities and develop corresponding defense strategies. Feature reduction strategies are used to decrease the dimensionality of the data in order to reduce training time, avoid the curse of dimensionality, prevent overfitting, and mitigate redundant features. However, their effect on adversarial robustness across different classifiers and attack scenarios remains insufficiently explored. This study aims to analyze the effect of feature reduction techniques on the adversarial robustness of a diverse set of machine learning models under poisoning, backdoor, and universal adversarial perturbation attacks on the APT dataset. These models include support vector machine, decision tree, random forest, multi-layer perceptron, and k-nearest neighbors. The attained results are compared with scenarios where different feature reduction techniques are applied, including principal component analysis, permutation importance ranking, recursive feature elimination, and univariate mutual information. Metrics such as accuracy, F1-score, and mitigation score are employed to evaluate the effect of these techniques on adversarial robustness. The results demonstrate that permutation importance ranking and univariate mutual information generally provide the most consistent robustness improvements, while other methods exhibit more variable effects depending on the classifier and attack scenario.

## Learning to Verify: Signature-Guided Training for Adaptive Adversarial Detection

DevFeed: [Learning to Verify: Signature-Guided Training for Adaptive Adversarial Detection](<https://devfeed.tech/articles/learning-to-verify-signature-guided-training-for-adaptive-adversarial-detection-145031.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.25271>)

Author: Arash Vashagh, Roozbeh Razavi-Far

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Adversarial attacks](<https://devfeed.tech/topics/adversarial-attacks.md>), [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [Neural Network](<https://devfeed.tech/topics/neural-network.md>)

Tags: [ablation](<https://devfeed.tech/tags/ablation.md>), [adaptive](<https://devfeed.tech/tags/adaptive.md>), [adversarial](<https://devfeed.tech/tags/adversarial.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [detection](<https://devfeed.tech/tags/detection.md>), [learning](<https://devfeed.tech/tags/learning.md>), [training](<https://devfeed.tech/tags/training.md>), [verify](<https://devfeed.tech/tags/verify.md>)

### AI overview

The approach keeps the classifier fixed and trains a separate verifier to assess its predictions. During training, class-specific activation signatures guide the creation of difficult adversarial examples; knowledge distillation helps the verifier retain performance on clean inputs. At inference, it checks predictions without using the signatures. On the evaluated datasets, it substantially outperformed confidence-based and feature-based detectors against the tested adaptive L-infinity attacks, including attacks unseen in training. Ablations attribute most of the robustness gain to adaptive verifier training, while signature guidance mainly helps clean accuracy. Additional L2 evaluations expose limits to robustness across perturbation types.

### Source excerpt

arXiv:2611.25271v1 Announce Type: new Abstract: Adversarial example detectors often fail when attackers explicitly attempt to bypass them. We propose a dual-model approach that keeps the original classifier fixed and trains a separate verifier to assess its predictions. During training, we identify patterns of internal neural network activations that distinguish genuine examples of each class from adversarial examples targeting that class. These patterns, called activation signatures, guide the generation of challenging adversarial examples that mislead the classifier, challenge the verifier, and internally resemble the target class. The verifier learns to recognize these examples while preserving its performance on clean inputs through knowledge distillation. At inference, the verifier evaluates the classifier's prediction without requiring activation signatures. We evaluate the approach on publicly available datasets against several adaptive attacks, including attacks unseen during training. The proposed verifier substantially outperforms directly attacked confidence-based and feature-based detectors under the evaluated L-infinity attacks. Ablation studies show that adaptive verifier training is the main source of this improvement, while signature guidance primarily benefits clean accuracy. Additional L2 evaluations reveal limitations in robustness across perturbation types.

## Security of Deep Reinforcement Learning for Autonomous Driving: A Survey

DevFeed: [Security of Deep Reinforcement Learning for Autonomous Driving: A Survey](<https://devfeed.tech/articles/security-of-deep-reinforcement-learning-for-autonomous-driving-a-survey-144846.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2212.06123>)

Author: Ambra Demontis, Srishti Gupta, Maura Pintor, Luca Demetrio, Kathrin Grosse, Hsiao-Ying Lin, Chengfang Fang, Battista Biggio, Fabio Roli

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Reinforcement learning](<https://devfeed.tech/topics/reinforcement-learning.md>), [ai security](<https://devfeed.tech/topics/ai-security.md>), [autonomous vehicles](<https://devfeed.tech/topics/autonomous-vehicles.md>), [Adversarial attacks](<https://devfeed.tech/topics/adversarial-attacks.md>)

Tags: [autonomous-driving](<https://devfeed.tech/tags/autonomous-driving.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [cs-ro](<https://devfeed.tech/tags/cs-ro.md>), [reinforcement-learning](<https://devfeed.tech/tags/reinforcement-learning.md>), [security](<https://devfeed.tech/tags/security.md>), [survey](<https://devfeed.tech/tags/survey.md>)

### AI overview

The survey examines how attacks can undermine reinforcement learning by compromising policy learning or causing trained agents to make erroneous decisions, a concern for safety-critical uses such as autonomous driving. It reviews 86 recent studies and organizes attacks and defenses by threat model and single- versus multi-agent setting, aiming to help select defenses for particular systems. It also assesses how state-of-the-art attacks and defenses apply to autonomous driving. The supplied text does not describe specific attack or defense methods or their comparative effectiveness.

### Source excerpt

arXiv:2212.06123v5 Announce Type: replace Abstract: Reinforcement learning (RL) enables agents to learn optimal behaviors through interaction with their environment and has been increasingly deployed in safety-critical applications, including autonomous driving. Despite its promise, RL is susceptible to attacks designed either to compromise policy learning or to induce erroneous decisions by trained agents. Although the literature on RL security has grown rapidly and several surveys exist, existing categorizations often fall short in guiding the selection of appropriate defenses for specific systems. In this work, we present a comprehensive survey of 86 recent studies on RL security, addressing these limitations by systematically categorizing attacks and defenses according to defined threat models and single- versus multi-agent settings. Furthermore, we examine the relevance and applicability of state-of-the-art attacks and defense mechanisms within the context of autonomous driving, providing insights to inform the design of robust RL systems.

## Ask in the Crowd: Differentially Private LLM Inference via Dummy-Augmented Shuffling

DevFeed: [Ask in the Crowd: Differentially Private LLM Inference via Dummy-Augmented Shuffling](<https://devfeed.tech/articles/ask-in-the-crowd-differentially-private-llm-inference-via-dummy-augmented-shuffling-145566.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.25669>)

Author: Zhihao Liu, Zixiong Guo, Shuo Shao, Yu He, Wenli Wang, Meihui Chen, Di Wang, Yuke Hu

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Cryptography and Security](<https://devfeed.tech/sources/arxiv-cryptography-and-security.md>)

Topics: [LLM security](<https://devfeed.tech/topics/llm-security.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>), [LLM optimization](<https://devfeed.tech/topics/llm-optimization.md>), [Adversarial attacks](<https://devfeed.tech/topics/adversarial-attacks.md>)

Tags: [abstract](<https://devfeed.tech/tags/abstract.md>), [attacks](<https://devfeed.tech/tags/attacks.md>), [balance](<https://devfeed.tech/tags/balance.md>), [cache](<https://devfeed.tech/tags/cache.md>), [classification](<https://devfeed.tech/tags/classification.md>), [client](<https://devfeed.tech/tags/client.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cr](<https://devfeed.tech/tags/cs-cr.md>), [decoding](<https://devfeed.tech/tags/decoding.md>), [efficiency](<https://devfeed.tech/tags/efficiency.md>), [embeddings](<https://devfeed.tech/tags/embeddings.md>), [framework](<https://devfeed.tech/tags/framework.md>), [generation](<https://devfeed.tech/tags/generation.md>), [gpt](<https://devfeed.tech/tags/gpt.md>), [inference](<https://devfeed.tech/tags/inference.md>), [inference-efficiency](<https://devfeed.tech/tags/inference-efficiency.md>), [inference-framework](<https://devfeed.tech/tags/inference-framework.md>), [input](<https://devfeed.tech/tags/input.md>), [kv-cache](<https://devfeed.tech/tags/kv-cache.md>), [llm](<https://devfeed.tech/tags/llm.md>), [llm-inference](<https://devfeed.tech/tags/llm-inference.md>), [private](<https://devfeed.tech/tags/private.md>)

### AI overview

DP-SPD hides a user's real query among client-generated dummy queries, shuffles the queries before sending them to the LLM server, and reuses transformer key-value caches for queries with shared prefixes during decoding. The paper analyzes its privacy guarantee and evaluates classification and generation utility and protection against QIA, EIA, and GPT-based attacks. The abstract reports strong inference-time privacy, near-plaintext utility, and improved efficiency, but gives no quantitative results or implementation details.

### Source excerpt

arXiv:2611.25669v1 Announce Type: new Abstract: Large language models are increasingly deployed as LLM-as-a-Service (LLMaaS), where user queries are sent to remote LLMs for inference, raising severe privacy concerns about the exposure of sensitive user inputs. Differentially private inference mitigates these risks by injecting noise into input embeddings. However, existing methods struggle to balance privacy and utility, as stronger privacy protection often leads to substantial accuracy degradation in LLM inference. In this work, we propose DP-SPD, a differentially private LLM inference framework that protects user queries by hiding the real query among client-generated dummy queries. All queries are randomly shuffled before being sent to the server, preventing direct identification of the user's true intent. To improve inference efficiency, we exploit the shared-prefix structure among queries and reuse transformer key--value (KV) caches during decoding. We analyze the privacy guarantee of DP-SPD and evaluate its utility on classification and generation tasks, as well as its privacy protection under QIA, EIA, and GPT-based attacks. Experimental results show that our method achieves strong inference-time privacy protection and near-plaintext utility while improving inference efficiency through shared-prefix KV-cache reuse.

## Density Feedback Control for Transferability of Generative Sparse Adversarial Attacks

DevFeed: [Density Feedback Control for Transferability of Generative Sparse Adversarial Attacks](<https://devfeed.tech/articles/density-feedback-control-for-transferability-of-generative-sparse-adversarial-attacks-145000.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.25001>)

Author: Dongsu Song, Jay Hoon Jung

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Adversarial attacks](<https://devfeed.tech/topics/adversarial-attacks.md>), [Generative Adversarial Network](<https://devfeed.tech/topics/generative-adversarial-network.md>)

Tags: [abstract](<https://devfeed.tech/tags/abstract.md>), [adversarial](<https://devfeed.tech/tags/adversarial.md>), [architecture](<https://devfeed.tech/tags/architecture.md>), [asr](<https://devfeed.tech/tags/asr.md>), [attack](<https://devfeed.tech/tags/attack.md>), [attacks](<https://devfeed.tech/tags/attacks.md>), [black-box](<https://devfeed.tech/tags/black-box.md>), [cnn](<https://devfeed.tech/tags/cnn.md>), [control](<https://devfeed.tech/tags/control.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [decoder](<https://devfeed.tech/tags/decoder.md>), [encoder](<https://devfeed.tech/tags/encoder.md>), [encoder-decoder](<https://devfeed.tech/tags/encoder-decoder.md>), [evaluation](<https://devfeed.tech/tags/evaluation.md>), [feature](<https://devfeed.tech/tags/feature.md>), [feature-importance](<https://devfeed.tech/tags/feature-importance.md>), [feedback](<https://devfeed.tech/tags/feedback.md>), [fixed](<https://devfeed.tech/tags/fixed.md>), [gap](<https://devfeed.tech/tags/gap.md>), [generative](<https://devfeed.tech/tags/generative.md>), [generator](<https://devfeed.tech/tags/generator.md>)

### AI overview

Density Feedback Control (DFC) adjusts the sparsity coefficient after each training epoch based on the difference between observed hard-mask density and a target, without changing the generator architecture or loss. At a 10% target, it brought three base attacks' mean density within one percentage point in a single run. Two attacks improved transfer attack success rates on CNN and Transformer targets, including under per-image ℓ₀ budgets, while perturbing fewer pixels. In GPG, transfer success recovered as DFC relaxed the coefficient and surpassed the fixed-coefficient baseline with fewer pixels. DFC showed no consistent gain for EGS-TSSA, which limits perturbations to source-model feature-importance regions.

### Source excerpt

arXiv:2611.25001v1 Announce Type: new Abstract: Generator-based sparse adversarial attacks train encoder--decoder networks to produce transferable sparse perturbations for black-box robustness evaluation. Their perturbation density, however, is set only indirectly by a fixed sparsity coefficient whose mapping to density is unknown and shifts during training, so reaching a target density requires re-tuning; this fixed pressure also yields little transferability gain once density reduction begins. We propose Density Feedback Control (DFC), which tightens or relaxes this coefficient at the end of each epoch according to the gap between the observed hard-mask density and the target, without changing the generator architecture or loss. At a 10\% target density, DFC brings the mean density of three base attacks within one percentage point of the target in a single run and improves the transfer attack success rate (ASR) of two of them on CNN and Transformer targets, including under per-image $\ell_0$ budgets, while perturbing fewer pixels. Tracing GPG's training, we observe that as DFC relaxes the coefficient, the transfer ASR recovers with the density and overtakes the fixed-coefficient baseline with fewer pixels. For EGS-TSSA, which restricts perturbations to source-model feature-importance regions, DFC yields no consistent gain; we analyze how this restriction interacts with density feedback.

## VisualJevBench: A Benchmark and Black-Box Attacks for Visual Typed Decision Models

DevFeed: [VisualJevBench: A Benchmark and Black-Box Attacks for Visual Typed Decision Models](<https://devfeed.tech/articles/visualjevbench-a-benchmark-and-black-box-attacks-for-visual-typed-decision-models-146031.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.25435>)

Author: Li Zeng, Yeqi Zeng, Hengkai Liang, Shiguo Wang, Meng Xie, Zeyu Ye, Jinheng Zhang, Xianlong Wang, Ziqi Zhou, Hangtao Zhang

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Cryptography and Security](<https://devfeed.tech/sources/arxiv-cryptography-and-security.md>)

Topics: [Adversarial attacks](<https://devfeed.tech/topics/adversarial-attacks.md>), [LLM security](<https://devfeed.tech/topics/llm-security.md>), [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [Image](<https://devfeed.tech/topics/image.md>)

Tags: [adversarial](<https://devfeed.tech/tags/adversarial.md>), [asr](<https://devfeed.tech/tags/asr.md>), [attacks](<https://devfeed.tech/tags/attacks.md>), [benchmark](<https://devfeed.tech/tags/benchmark.md>), [black-box](<https://devfeed.tech/tags/black-box.md>), [cs-cr](<https://devfeed.tech/tags/cs-cr.md>), [decision-models](<https://devfeed.tech/tags/decision-models.md>), [evaluation](<https://devfeed.tech/tags/evaluation.md>), [framework](<https://devfeed.tech/tags/framework.md>), [geometric-transformations](<https://devfeed.tech/tags/geometric-transformations.md>), [image](<https://devfeed.tech/tags/image.md>), [search](<https://devfeed.tech/tags/search.md>), [selection](<https://devfeed.tech/tags/selection.md>), [spoofing](<https://devfeed.tech/tags/spoofing.md>), [transformation](<https://devfeed.tech/tags/transformation.md>)

### AI overview

VisualJevBench evaluates adversarial robustness in Visual Jev using 800 images and 3,200 questions across eight scenarios and four attack surfaces: typographic manipulation, visual evidence spoofing, target selection interference, and geometric transformation. Its Adaptive Black-Box Parameter Search (ABPS) uses typed-output feedback to find image-specific and universal manipulations; universal attacks can transfer to unseen images and tasks in new scenarios without further optimization. Experiments on jev-1.14.0 report up to 69.6% attack success for image-specific attacks, 33.15% average success for universal attacks on unseen images, and 40.09% average cross-scenario transfer success for geometric transformations.

### Source excerpt

arXiv:2611.25435v1 Announce Type: new Abstract: Typed decision models (TDMs) return structured decisions instead of generating free-text responses, enabling fast and cheap decisions in real-world applications. Recent support for image inputs extends Jev, a representative TDM, to visual decision-making. We refer to this setting as Visual Jev. However, red teaming of Visual Jev leaves two gaps: a systematic evaluation of direct image manipulations is lacking, and universal visual attacks remain underexplored. To address these gaps, we introduce VisualJevBench, a benchmark for evaluating the adversarial robustness of Visual Jev. It comprises 800 images and 3,200 questions across eight scenarios and four visual attack surfaces: typographic manipulation, visual evidence spoofing, target selection interference, and geometric transformation. We also introduce Adaptive Black-Box Parameter Search (ABPS), a lightweight framework that uses typed-output feedback to find effective image manipulations. ABPS supports both image-specific attacks and universal attacks that transfer to unseen images and tasks in new scenarios without further optimization. Extensive experiments on jev-1.14.0 show that image-specific attacks achieve an attack success rate (ASR) of up to 69.6%, while universal attacks achieve an average ASR of 33.15% on unseen images. Geometric transformations achieve an average ASR of 40.09% in cross-scenario transfer.

## ChronoPatch: Black-Box Energy-Latency Attacks on Vision-Language Models

DevFeed: [ChronoPatch: Black-Box Energy-Latency Attacks on Vision-Language Models](<https://devfeed.tech/articles/chronopatch-black-box-energy-latency-attacks-on-vision-language-models-145634.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.26117>)

Author: Jiawei Xing, Li Wang, Xiangtao Meng, Chuanchao Zang, Rui Wen, Zheng Li

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Cryptography and Security](<https://devfeed.tech/sources/arxiv-cryptography-and-security.md>)

Topics: [vlm](<https://devfeed.tech/topics/vlm.md>), [Adversarial attacks](<https://devfeed.tech/topics/adversarial-attacks.md>), [LLM security](<https://devfeed.tech/topics/llm-security.md>)

Tags: [adversarial](<https://devfeed.tech/tags/adversarial.md>), [attacks](<https://devfeed.tech/tags/attacks.md>), [availability](<https://devfeed.tech/tags/availability.md>), [black-box](<https://devfeed.tech/tags/black-box.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cr](<https://devfeed.tech/tags/cs-cr.md>), [energy](<https://devfeed.tech/tags/energy.md>), [language-models](<https://devfeed.tech/tags/language-models.md>), [latency](<https://devfeed.tech/tags/latency.md>), [vision-language-models](<https://devfeed.tech/tags/vision-language-models.md>)

### AI overview

ChronoPatch shows that visual inputs can prolong vision-language model generation even when an attacker sees only the final response. It uses response-length-guided evolutionary search to find visual delay anchors, then subspace-calibrated decision-based optimization to reduce perturbation magnitude. Under a 5% spatial budget, tests on four open-source VLMs report latency increases of 47.86%-293.34% and GPU energy increases of 41.44%-249.70%, alongside more repetitive outputs. On three commercial VLM APIs, with 500 queries per API, response length rose 37.6%-81.5% and client-observed latency reached 1.82-2.29 times baseline. The abstract identifies this as a generation-based availability risk and motivates safeguards that account for generation.

### Source excerpt

arXiv:2611.26117v1 Announce Type: cross Abstract: Vision-language models (VLMs) are increasingly deployed as generative inference services, where autoregressive decoding makes response length a key determinant of latency and computational cost. This creates an availability risk: adversarial visual inputs can prolong generation and increase serving overhead. Existing energy-latency attacks typically rely on gradients, logits, or token-level feedback, leaving open whether such risks persist when an attacker observes only the final response. We present ChronoPatch, a black-box attack that induces prolonged VLM generation requiring only the final response. It discovers visual delay anchors through response-length-guided evolutionary search, then minimizes perturbation magnitude through subspace-calibrated decision-based optimization. Across four open-source VLMs, ChronoPatch increases inference latency by 47.86%--293.34% and GPU energy consumption by 41.44%--249.70% under a 5% spatial budget. It also induces more repetitive outputs, as reflected by increased n-gram repetition ratios. On three commercial VLM APIs, ChronoPatch increases response length by 37.6%--81.5% and client-observed latency by 1.82x--2.29x under query budgets of 500 per API. These results reveal a generation-based availability risk that persists under final-response-only access and motivate generation-aware safeguards for deployed generative AI services.

## I2I Backdoor: Backdoor Attacks against Image-to-Image Tasks

DevFeed: [I2I Backdoor: Backdoor Attacks against Image-to-Image Tasks](<https://devfeed.tech/articles/i2i-backdoor-backdoor-attacks-against-image-to-image-tasks-142489.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2407.10445>)

Author: Wenbo Jiang, Hongwei Li, Jiaming He, Rui Zhang, Guowen Xu, Tianwei Zhang, Rongxing Lu

Published: 2026-11-19T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Cryptography and Security](<https://devfeed.tech/sources/arxiv-cryptography-and-security.md>)

Topics: [Adversarial attacks](<https://devfeed.tech/topics/adversarial-attacks.md>), [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [Image processing](<https://devfeed.tech/topics/image-processing.md>)

Tags: [abstract](<https://devfeed.tech/tags/abstract.md>), [adversarial](<https://devfeed.tech/tags/adversarial.md>), [algorithm](<https://devfeed.tech/tags/algorithm.md>), [architectures](<https://devfeed.tech/tags/architectures.md>), [arxiv](<https://devfeed.tech/tags/arxiv.md>), [attack](<https://devfeed.tech/tags/attack.md>), [attacks](<https://devfeed.tech/tags/attacks.md>), [backdoor](<https://devfeed.tech/tags/backdoor.md>), [based](<https://devfeed.tech/tags/based.md>), [better](<https://devfeed.tech/tags/better.md>), [can](<https://devfeed.tech/tags/can.md>), [choice](<https://devfeed.tech/tags/choice.md>), [classification](<https://devfeed.tech/tags/classification.md>), [classifiers](<https://devfeed.tech/tags/classifiers.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cr](<https://devfeed.tech/tags/cs-cr.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [deep](<https://devfeed.tech/tags/deep.md>), [deep-learning](<https://devfeed.tech/tags/deep-learning.md>), [denoising](<https://devfeed.tech/tags/denoising.md>), [image](<https://devfeed.tech/tags/image.md>), [image-to-image](<https://devfeed.tech/tags/image-to-image.md>), [tasks](<https://devfeed.tech/tags/tasks.md>)

### AI overview

The paper proposes a backdoor for image-to-image networks that behaves normally on clean inputs but produces a specific inappropriate output when a universal adversarial perturbation trigger is present. Its training uses multi-task learning with dynamic weighting. The authors also show that denoised trigger-bearing images can fool downstream image classifiers and object detectors, and report effectiveness across state-of-the-art I2I architectures and robustness against different backdoor defenses. The supplied text is an abstract, so it does not provide implementation details or quantify these results.

### Source excerpt

arXiv:2407.10445v2 Announce Type: replace-cross Abstract: With the rapid development of deep learning technology, deep learning-based Image-to-Image (I2I) networks have become the predominant choice for I2I tasks like image super-resolution and denoising. Despite their remarkable performance, the security of I2I networks has not been thoroughly investigated. While some studies have probed their susceptibility to adversarial attacks, none have explored the backdoor attack against I2I networks, which is a more stealthy and severe threat. In this work, for the first time, we comprehensively investigate the vulnerability of I2I networks to backdoor attacks. We propose a backdoor attack against I2I tasks, where the backdoored I2I network behaves normally on clean input images, yet outputs a specific inappropriate image when the backdoor trigger appears on the input image. To achieve such an I2I backdoor attack, we design a universal adversarial perturbation (UAP) generation algorithm for I2I networks, where the generated UAP is used as the trigger for the I2I backdoor. Besides, multi-task learning (MTL) with dynamic weighting methods is employed in the backdoor training process to gain better results. Expanding our focus beyond I2I tasks, we extend our I2I backdoor to attack downstream tasks, including image classification and object detection. Specifically, the backdoor-triggered image processed by the backdoored image denoising network can fool the downstream image classifiers and object detectors. Extensive experiments demonstrate the effectiveness of the I2I backdoor on state-of-the-art I2I network architectures as well as the robustness against different backdoor defenses.

## Data Poisoning Attacks to Locally Differentially Private Triangle Counting Protocols

DevFeed: [Data Poisoning Attacks to Locally Differentially Private Triangle Counting Protocols](<https://devfeed.tech/articles/data-poisoning-attacks-to-locally-differentially-private-triangle-counting-protocols-143131.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.23701>)

Author: Tingxuan Han, Wei Tong, Zijian Wang, Sheng Zhong

Published: 2026-11-19T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Cryptography and Security](<https://devfeed.tech/sources/arxiv-cryptography-and-security.md>)

Topics: [Differential Privacy](<https://devfeed.tech/topics/differential-privacy.md>), [Adversarial attacks](<https://devfeed.tech/topics/adversarial-attacks.md>)

Tags: [adversarial](<https://devfeed.tech/tags/adversarial.md>), [analytics](<https://devfeed.tech/tags/analytics.md>), [analyze](<https://devfeed.tech/tags/analyze.md>), [attack](<https://devfeed.tech/tags/attack.md>), [attacks](<https://devfeed.tech/tags/attacks.md>), [bias](<https://devfeed.tech/tags/bias.md>), [cs-cr](<https://devfeed.tech/tags/cs-cr.md>), [data](<https://devfeed.tech/tags/data.md>), [datasets](<https://devfeed.tech/tags/datasets.md>), [experiments](<https://devfeed.tech/tags/experiments.md>), [framework](<https://devfeed.tech/tags/framework.md>), [global](<https://devfeed.tech/tags/global.md>), [graph](<https://devfeed.tech/tags/graph.md>), [guarantees](<https://devfeed.tech/tags/guarantees.md>), [integrity](<https://devfeed.tech/tags/integrity.md>), [interactive](<https://devfeed.tech/tags/interactive.md>), [local](<https://devfeed.tech/tags/local.md>), [malicious](<https://devfeed.tech/tags/malicious.md>), [manipulate](<https://devfeed.tech/tags/manipulate.md>), [private](<https://devfeed.tech/tags/private.md>), [protocols](<https://devfeed.tech/tags/protocols.md>)

### AI overview

The paper finds that malicious participants can significantly distort locally differentially private triangle count estimates. Because locally randomized reports make malicious users difficult to distinguish, small protocol-compliant manipulations can be amplified during reconstruction and calibration into large estimation bias. The authors develop a gain-driven attack framework for both non-interactive and interactive protocols and report experiments across four real-world graph datasets. They also find that existing and straightforward defenses are insufficient, pointing to the need for defenses tailored to LDP subgraph counting.

### Source excerpt

arXiv:2611.23701v1 Announce Type: new Abstract: Local differential privacy (LDP) has become a crucial paradigm for privacy-preserving graph analytics, enabling graph analytics tasks such as subgraph counting without requiring a trusted curator. However, while LDP provides strong privacy guarantees, the integrity and robustness of LDP subgraph counting protocols in the presence of malicious participants remain poorly understood. In particular, since data are randomized locally before collection, it is inherently difficult to distinguish malicious users from honest ones based solely on their submitted reports, and small, protocol-compliant local manipulations can be systematically amplified through reconstruction and calibration into large global estimation bias. In this paper, we use LDP triangle counting as a representative task to characterize the vulnerability of LDP subgraph counting to data poisoning attacks. For both non-interactive and interactive LDP triangle counting protocols, we analyze their attack surfaces and develop a unified gain-driven framework that captures how adversarial users can manipulate calibrated subgraph count estimates, which we instantiate on representative protocols of both types. Extensive experiments on four real-world graph datasets demonstrate that these attacks can significantly distort triangle count estimates across different protocols and adversarial settings. Moreover, we show that existing and straightforward defenses are insufficient to mitigate these attacks, underscoring the need for robustness mechanisms tailored to LDP subgraph counting.

## PIPETTE: Prompt Injection Purple-Teaming for Robust LLM Agents

DevFeed: [PIPETTE: Prompt Injection Purple-Teaming for Robust LLM Agents](<https://devfeed.tech/articles/pipette-prompt-injection-purple-teaming-for-robust-llm-agents-142786.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.24362>)

Author: Lipeng He, Yihan Wang, Jiawen Zhang, N. Asokan

Published: 2026-11-19T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Cryptography and Security](<https://devfeed.tech/sources/arxiv-cryptography-and-security.md>)

Topics: [LLM security](<https://devfeed.tech/topics/llm-security.md>), [prompt injection](<https://devfeed.tech/topics/prompt-injection.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>), [Reinforcement learning](<https://devfeed.tech/topics/reinforcement-learning.md>), [Adversarial attacks](<https://devfeed.tech/topics/adversarial-attacks.md>)

Tags: [adaptive](<https://devfeed.tech/tags/adaptive.md>), [adversarial](<https://devfeed.tech/tags/adversarial.md>), [agentic](<https://devfeed.tech/tags/agentic.md>), [agents](<https://devfeed.tech/tags/agents.md>), [asr](<https://devfeed.tech/tags/asr.md>), [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cr](<https://devfeed.tech/tags/cs-cr.md>), [cybersecurity](<https://devfeed.tech/tags/cybersecurity.md>), [defense](<https://devfeed.tech/tags/defense.md>), [indirect-prompt-injection](<https://devfeed.tech/tags/indirect-prompt-injection.md>), [large-language-model](<https://devfeed.tech/tags/large-language-model.md>), [llm](<https://devfeed.tech/tags/llm.md>), [llm-agents](<https://devfeed.tech/tags/llm-agents.md>), [prompt-injection](<https://devfeed.tech/tags/prompt-injection.md>), [red-teaming](<https://devfeed.tech/tags/red-teaming.md>), [reinforcement-learning](<https://devfeed.tech/tags/reinforcement-learning.md>), [reward](<https://devfeed.tech/tags/reward.md>), [rl](<https://devfeed.tech/tags/rl.md>), [robustness](<https://devfeed.tech/tags/robustness.md>), [training](<https://devfeed.tech/tags/training.md>), [vulnerabilities](<https://devfeed.tech/tags/vulnerabilities.md>)

### AI overview

PIPETTE aims to turn prompt-injection red-team discoveries into stronger defenses: it uses a dictionary learning-based reward to broaden attack-strategy coverage and an on-policy adversarial-training pipeline designed to preserve legitimate task completion. The abstract reports that RL-Hammer and PISmith repeatedly find similar strategies, so higher attack success during red-teaming does not necessarily yield added training benefits. On AgentDojo and ASB, PIPETTE-trained agents reportedly achieve near-zero success for static attacks, withstand strong adaptive attacks, and retain benign-task utility close to an undefended baseline.

### Source excerpt

arXiv:2611.24362v1 Announce Type: cross Abstract: Large language model (LLM) agents routinely interact with untrusted external data, exposing them to indirect prompt injection (IPI) attacks that hijack agents into executing attacker-chosen tasks. To uncover these vulnerabilities at scale, automated red-teaming methods often leverage reinforcement learning (RL) to generate attacks that achieve high attack success rates (ASR). Model developers use the attacks discovered during internal red-teaming to harden agents through adversarial training before deployment, a collaborative offensive-defensive process known in cybersecurity as purple-teaming. However, we show that achieving high ASR during red-teaming does not guarantee robustness gains from subsequent adversarial training. Specifically, we find that two state-of-the-art RL-based red-teaming methods, RL-Hammer and PISmith, suffer from mode collapse, repeatedly discovering similar attack strategies whose additional successes add no additional robustness improvements to adversarial training. To address this gap, we introduce PIPETTE, a unified prompt injection purple-teaming framework that effectively converts red-teaming discoveries into defender robustness improvements. PIPETTE employs a dictionary learning-based reward that encourages broader coverage of attack strategies during red-teaming. It also integrates an on-policy adversarial training pipeline tailored to IPI defense, enabling agents to resist attacks while completing legitimate user tasks. Extensive evaluations across agentic benchmarks (AgentDojo and ASB) demonstrate that PIPETTE produces defenders that achieve near-zero attack success rates on static attacks and can withstand strong adaptive attacks, while maintaining benign task utility close to that of the undefended baseline.

## Survival of the Fittest Perturbations: Uncertainty-Aware Evolution of Adversarial Distributions for Robust Generalization

DevFeed: [Survival of the Fittest Perturbations: Uncertainty-Aware Evolution of Adversarial Distributions for Robust Generalization](<https://devfeed.tech/articles/survival-of-the-fittest-perturbations-uncertainty-aware-evolution-of-adversarial-distributions-for-robust-generalization-143217.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2411.02871>)

Author: Junhao Dong, Zicheng Yu, Xinghua Qu, Z. Jane Wang, Yew-Soon Ong

Published: 2026-11-19T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [Adversarial attacks](<https://devfeed.tech/topics/adversarial-attacks.md>), [Training AI Models](<https://devfeed.tech/topics/training-ai-models.md>)

Tags: [adversarial](<https://devfeed.tech/tags/adversarial.md>), [alignment](<https://devfeed.tech/tags/alignment.md>), [analysis](<https://devfeed.tech/tags/analysis.md>), [anchor](<https://devfeed.tech/tags/anchor.md>), [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [certificate](<https://devfeed.tech/tags/certificate.md>), [component](<https://devfeed.tech/tags/component.md>), [conditional](<https://devfeed.tech/tags/conditional.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [deep-learning](<https://devfeed.tech/tags/deep-learning.md>), [deployment](<https://devfeed.tech/tags/deployment.md>), [design](<https://devfeed.tech/tags/design.md>), [distribution](<https://devfeed.tech/tags/distribution.md>), [distributions](<https://devfeed.tech/tags/distributions.md>), [effectiveness](<https://devfeed.tech/tags/effectiveness.md>), [elite](<https://devfeed.tech/tags/elite.md>), [evaluations](<https://devfeed.tech/tags/evaluations.md>), [evolution](<https://devfeed.tech/tags/evolution.md>), [examples](<https://devfeed.tech/tags/examples.md>), [robust](<https://devfeed.tech/tags/robust.md>), [uncertainty](<https://devfeed.tech/tags/uncertainty.md>)

### AI overview

EvoUAD-AT addresses a limitation of pointwise adversarial training, which assigns one adversary to each clean sample and misses variation in adversarial feature statistics. It maintains an instance-specific search distribution around an adversarial anchor, evaluates candidates under the current model, selects those most threatening to a benign reference, and adapts the distribution while retaining strategy parameters across epochs. It also refines the alignment reference to avoid misclassified clean samples, matches benign and adversarial distributions, and uses introspective gradient matching without external supervision. The authors report that evaluations across multiple benchmarks show improved clean accuracy and robustness over existing state-of-the-art methods; the supplied abstract gives no benchmark names or quantitative results.

### Source excerpt

arXiv:2411.02871v2 Announce Type: replace Abstract: Despite deep learning's broad successes across diverse scenarios, its vulnerability to adversarial examples impedes deployment, making adversarial training a leading approach to improve robustness. However, pointwise adversarial training maps each clean sample to a single adversary, overlooking surrounding adversarial variability and the associated shifts in feature statistics. To address these issues, we propose Evolutionary Uncertainty-Aware Distributional Adversarial Training (EvoUAD-AT). Specifically, we design an evolutionary adversarial uncertainty modeling module that maintains an instance-wise search distribution over adversarial feature statistics around an adversarially generated anchor. Candidates are evaluated under the current model, selected by their threat to a benign reference, and used to adapt the distribution; strategy parameters are retained across epochs. To avoid anchoring on misclassified clean samples, we refine the alignment reference and match benign and adversarial distributions. Introspective gradient matching further regularizes input gradients without external supervision. Our analysis characterizes the elite fitness surrogate and gives a conditional certificate for latent adversarial features. Extensive evaluations across multiple benchmarks demonstrate that our approach consistently outperforms existing state-of-the-art methods in both clean accuracy and robustness, with thorough analyses confirming the effectiveness of each proposed component.

## Stealing Exact Training Data from Other Clients during Federated Learning

DevFeed: [Stealing Exact Training Data from Other Clients during Federated Learning](<https://devfeed.tech/articles/stealing-exact-training-data-from-other-clients-during-federated-learning-142985.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.24689>)

Author: Omkar Shrikanth, Soham Dambalkar, Parag Jyoti Pal, Advik Raj Basani, Arnab K. Paul

Published: 2026-11-19T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Cryptography and Security](<https://devfeed.tech/sources/arxiv-cryptography-and-security.md>), [arXiv: Distributed Computing](<https://devfeed.tech/sources/arxiv-distributed-computing.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Federated Learning](<https://devfeed.tech/topics/federated-learning.md>), [Adversarial attacks](<https://devfeed.tech/topics/adversarial-attacks.md>), [Machine learning](<https://devfeed.tech/topics/machine-learning.md>)

Tags: [attack](<https://devfeed.tech/tags/attack.md>), [cs-cr](<https://devfeed.tech/tags/cs-cr.md>), [cs-dc](<https://devfeed.tech/tags/cs-dc.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [data-privacy](<https://devfeed.tech/tags/data-privacy.md>), [federated-learning](<https://devfeed.tech/tags/federated-learning.md>), [text](<https://devfeed.tech/tags/text.md>), [training-data](<https://devfeed.tech/tags/training-data.md>)

### AI overview

The proposed attack progressively removes the gradient contributions of sentences it has recovered exactly from a federated-learning aggregate, exposing remaining sentences for recovery in later iterations. The authors report 100% precision for recovered sentences and up to 80.2% exact recovery when aggregates contain up to four times more tokens than the model's hidden dimension; they say existing attacks recover none exactly in that regime. These are reported results from the supplied abstract.

### Source excerpt

arXiv:2611.24689v1 Announce Type: cross Abstract: Malicious adversaries can employ gradient inversion attacks to reverse-engineer private data using the model updates of a round in Federated Learning. While there have been developments in gradient inversion for text, exact text recovery by a malicious client has so far remained infeasible, as FedAvg entangles the gradient contributions of many clients and obscures the signal of any individual sentence. We introduce a novel attack pipeline that unwinds this entanglement progressively through an iterative loop, in which the gradient contributions of sentences recovered exactly in one round are subtracted from the aggregate, exposing the remaining sentences for recovery in the next iteration. We show, for the first time, exact sentence recovery, with 100% precision, and on aggregates containing up to $4\times$ more tokens than the model's hidden dimension, we recover up to 80.2% of client sentences exactly, a regime where existing attacks recover none exactly.

## RECLAIM: Robot-Vision-Informed Adversarial Patch Defense via Embodied Attention Feedback

DevFeed: [RECLAIM: Robot-Vision-Informed Adversarial Patch Defense via Embodied Attention Feedback](<https://devfeed.tech/articles/reclaim-robot-vision-informed-adversarial-patch-defense-via-embodied-attention-feedback-143156.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.24494>)

Author: Jiayou Zhu, Peng Guo, Litian Zhang, Xinze Shi, Yuting Zhang, Zelong Yu, Yize Tian, Jirong Liu, Zifeng Kang

Published: 2026-11-19T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Cryptography and Security](<https://devfeed.tech/sources/arxiv-cryptography-and-security.md>)

Topics: [Adversarial attacks](<https://devfeed.tech/topics/adversarial-attacks.md>), [Robot Manipulation](<https://devfeed.tech/topics/robot-manipulation.md>), [Machine Perception](<https://devfeed.tech/topics/machine-perception.md>)

Tags: [actions](<https://devfeed.tech/tags/actions.md>), [adversarial](<https://devfeed.tech/tags/adversarial.md>), [agent](<https://devfeed.tech/tags/agent.md>), [attack](<https://devfeed.tech/tags/attack.md>), [attention](<https://devfeed.tech/tags/attention.md>), [changes](<https://devfeed.tech/tags/changes.md>), [control](<https://devfeed.tech/tags/control.md>), [cs-cr](<https://devfeed.tech/tags/cs-cr.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [cs-ro](<https://devfeed.tech/tags/cs-ro.md>), [decision](<https://devfeed.tech/tags/decision.md>), [defense](<https://devfeed.tech/tags/defense.md>), [edits](<https://devfeed.tech/tags/edits.md>), [effective](<https://devfeed.tech/tags/effective.md>), [feedback](<https://devfeed.tech/tags/feedback.md>), [localization](<https://devfeed.tech/tags/localization.md>), [manipulation](<https://devfeed.tech/tags/manipulation.md>), [models](<https://devfeed.tech/tags/models.md>), [multiple](<https://devfeed.tech/tags/multiple.md>), [patch](<https://devfeed.tech/tags/patch.md>), [patches](<https://devfeed.tech/tags/patches.md>), [robot](<https://devfeed.tech/tags/robot.md>), [vision](<https://devfeed.tech/tags/vision.md>)

### AI overview

RECLAIM is a training-free, inference-time defense that uses a frozen vision-language-action policy's attention response to local image edits to guide patch localization and purification during robot manipulation. It combines that feedback with robot geometry, then revalidates treatments as actions change the observed scene. In evaluation against patches from four attack methods across multiple task suites, it achieved at least one successful rollout on 95% of evaluated tasks and improved LIBERO-Object rollout success from 0% without defense to 70.0%. Mean decision time was 548.9 ms per control step. The abstract reports these results under frozen VLA control; it does not detail the specific local treatments.

### Source excerpt

arXiv:2611.24494v1 Announce Type: cross Abstract: Vision-language-action policies control embodied agents through a perception-action loop grounded in physical interaction, where adversarial patches can corrupt action predictions and thereby alter subsequent observations. To mitigate these threats, many existing defenses rely on policy adaptation or auxiliary models, but require additional training or inference. We introduce RECLAIM, a training-free, inference-time defense that uses a frozen VLA's own attention response to local visual interventions to guide purification during closed-loop embodied manipulation. Our key insight is that attention changes induced by local edits provide feedback for choosing where and how to purify as the robot interacts with its environment. RECLAIM combines this feedback with robot geometry to refine patch localization and select local treatments, then revalidates them as the agent's embodied actions change the observed scene. We evaluate RECLAIM against patches generated by four adversarial attack methods across multiple manipulation task suites under frozen VLA control. Under adversarial patch attacks, RECLAIM succeeds in at least one rollout on 95% of the evaluated tasks and raises rollout success on LIBERO-Object from 0% without defense to 70.0%. Its mean decision time is only 548.9 ms per control step. These results highlight policy-guided visual purification as an effective approach to defending embodied agents against adversarial patches and recovering closed-loop task execution.

## Gradient-Based Defensive Poisoning Against System Identification Attacks on Cyber-Physical Systems

DevFeed: [Gradient-Based Defensive Poisoning Against System Identification Attacks on Cyber-Physical Systems](<https://devfeed.tech/articles/gradient-based-defensive-poisoning-against-system-identification-attacks-on-cyber-physical-systems-143161.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.24662>)

Author: Gon\\c{c}alo Fernandes, Jos\\'e Cec\\'ilio, Victor Mattos, Daniel S. Menasch\\'e, Amit Bhaya, Alan Oliveira de S\\'a

Published: 2026-11-19T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Cryptography and Security](<https://devfeed.tech/sources/arxiv-cryptography-and-security.md>)

Topics: [Adversarial attacks](<https://devfeed.tech/topics/adversarial-attacks.md>), [ai security](<https://devfeed.tech/topics/ai-security.md>), [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [Neural Network](<https://devfeed.tech/topics/neural-network.md>), [Cybersecurity](<https://devfeed.tech/topics/cybersecurity.md>)

Tags: [adversarial-machine-learning](<https://devfeed.tech/tags/adversarial-machine-learning.md>), [ai-driven-attacks](<https://devfeed.tech/tags/ai-driven-attacks.md>), [ann](<https://devfeed.tech/tags/ann.md>), [attacks](<https://devfeed.tech/tags/attacks.md>), [control-system](<https://devfeed.tech/tags/control-system.md>), [cs-cr](<https://devfeed.tech/tags/cs-cr.md>), [cs-sy](<https://devfeed.tech/tags/cs-sy.md>), [cyber-physical-systems](<https://devfeed.tech/tags/cyber-physical-systems.md>), [dc-motor](<https://devfeed.tech/tags/dc-motor.md>), [defense](<https://devfeed.tech/tags/defense.md>), [defensive](<https://devfeed.tech/tags/defensive.md>), [eess-sy](<https://devfeed.tech/tags/eess-sy.md>), [errors](<https://devfeed.tech/tags/errors.md>), [gradient](<https://devfeed.tech/tags/gradient.md>), [information](<https://devfeed.tech/tags/information.md>), [input](<https://devfeed.tech/tags/input.md>), [machine-learning](<https://devfeed.tech/tags/machine-learning.md>), [magnitude](<https://devfeed.tech/tags/magnitude.md>), [measurements](<https://devfeed.tech/tags/measurements.md>), [model](<https://devfeed.tech/tags/model.md>), [models](<https://devfeed.tech/tags/models.md>), [motor](<https://devfeed.tech/tags/motor.md>), [network](<https://devfeed.tech/tags/network.md>)

### AI overview

The work proposes poisoning measurements observed by an attacker to reduce the accuracy of system identification in cyber-physical systems. It aggregates input-gradient perturbations from independently trained ANN identification models, confines them to information-rich transient regimes, and uses sparsity to target influential samples with little reported loss in effectiveness. In a simulated DC motor control system, this approach caused larger prediction errors than stochastic poisoning at equal RMS magnitude; the advantage grew with magnitude, while the operating range is limited, so the defense is intended for decoy settings such as CPS honeynets. The signals also degraded N4SID-based identification, suggesting transfer beyond the targeted ANN models.

### Source excerpt

arXiv:2611.24662v1 Announce Type: new Abstract: AI-driven attacks learn the models of cyber-physical systems (CPS) by passively observing controller and plant signals, then use this knowledge to act covertly. Because covertness depends on the accuracy of the identified model, degrading that accuracy is a viable countermeasure. This work applies Adversarial Machine Learning as a defense, injecting small perturbations into the measurements the attacker observes to degrade an artificial neural network (ANN)-based system identification (SI) attack. The perturbations are derived from input gradients of several independently trained identification models, aggregated into a generalized perturbation, and confined to the transient regimes that carry most of the dynamic information. A sparsity parameter further restricts them to the most influential samples, with negligible cost in effectiveness. On a simulated DC motor networked control system, the proposed perturbations induce larger prediction errors than stochastic poisoning of equal root-mean-square magnitude, and the advantage widens as the magnitude grows. This limits the defense's operating range, which is intended for decoy environments such as CPS honeynets. Additional results indicate that the designed poisoning signals can also degrade N4SID-based system identification, providing evidence of their transferability beyond the ANN-based models originally targeted.

## Wrong Input, Wrong Threshold: Re-evaluating Adversarial Facial Privacy

DevFeed: [Wrong Input, Wrong Threshold: Re-evaluating Adversarial Facial Privacy](<https://devfeed.tech/articles/wrong-input-wrong-threshold-re-evaluating-adversarial-facial-privacy-141473.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.21923>)

Author: Tunahan Parlay{\\i}c{\\i}, Erchan Aptoula, Y\\"ucel Sayg{\\i}n

Published: 2026-11-18T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Cryptography and Security](<https://devfeed.tech/sources/arxiv-cryptography-and-security.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Adversarial attacks](<https://devfeed.tech/topics/adversarial-attacks.md>), [Facial recognition](<https://devfeed.tech/topics/facial-recognition.md>), [Security & Privacy](<https://devfeed.tech/topics/security-privacy.md>)

Tags: [adversarial](<https://devfeed.tech/tags/adversarial.md>), [alignment](<https://devfeed.tech/tags/alignment.md>), [attack](<https://devfeed.tech/tags/attack.md>), [computer-vision](<https://devfeed.tech/tags/computer-vision.md>), [cs-cr](<https://devfeed.tech/tags/cs-cr.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [evaluation](<https://devfeed.tech/tags/evaluation.md>), [privacy](<https://devfeed.tech/tags/privacy.md>), [recognition](<https://devfeed.tech/tags/recognition.md>)

### AI overview

The paper finds that face-privacy evaluations can overstate protection when they skip deployed systems' face alignment and use fixed recognition thresholds on mismatched inputs. Re-scoring existing images through a realistic detect-align-crop pipeline reduced dodging protection by 56% for a standard transfer attack and 7% for a budget-allocating attack. The authors argue that success-rate differences saturate and propose paired cosine shift to measure input effects without depending on threshold calibration. Alignment increased this shift on three unseen recognizers and reversed the reported L-infinity transfer ranking at matched budget. They attribute the gap to resampling mismatch; optimizing through differentiable alignment largely reduced it, though a residual remained on one system. Fixed thresholds maintained about 1% false accepts on aligned crops but yielded 2.3-5.4% on whole images, with opposite-signed bias for dodging and impersonation.

### Source excerpt

arXiv:2611.21923v1 Announce Type: cross Abstract: Published face-privacy methods report protection above 90% against black-box recognition models. That number comes from an evaluation protocol with two independent defects, reported in a measure that cannot size either one. The input is wrong: methods published since 2022 feed the network the whole photograph, or a detector's box, where deployed systems detect, align to a landmark template and crop. Re-scoring the same files through a realistic front end costs a standard transfer attack 56% of its dodging protection and a budget-allocating attack 7%, with no image regenerated. The measure is wrong: that loss is a difference of success rates, so it saturates, moving by 8.2x over a budget sweep while the effect beneath it moves by 1.34x. We therefore report the paired cosine shift, in which both thresholds cancel algebraically, so the input defect holds without committing to a calibration. Measured that way it is 2.2x and 10.7x larger on three unseen recognizers, and at matched budget it reverses the L-infinity transfer ranking: the one surveyed method that aligns at evaluation gains 0.18 where MI-FGSM loses 0.05. The cause is a resampling mismatch, not perturbation lost outside the crop; optimising through a differentiable alignment operator largely removes it, adding +24 to +58 points on unseen recognizers and a commercial system, with a residual on one of them. The thresholds are wrong: the four constants inherited since 2021 hold the false-accept rate near its nominal 1% on aligned crops but admit 2.3-5.4% on whole images, with an opposite-signed bias across dodging and impersonation, the two tasks reported side by side. Differentiable alignment and gallery scoring are not new: the first generation of this literature used both, and we show what dropping them cost.

## Who Pays for Byzantine Robustness? Validity, Data-Share Dictatorships, and Fairness When Only a Few Hospitals Collaborate

DevFeed: [Who Pays for Byzantine Robustness? Validity, Data-Share Dictatorships, and Fairness When Only a Few Hospitals Collaborate](<https://devfeed.tech/articles/who-pays-for-byzantine-robustness-validity-data-share-dictatorships-and-fairness-when-only-a-few-hospitals-collaborate-141501.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.22904>)

Author: Adnan Malkani

Published: 2026-11-18T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Cryptography and Security](<https://devfeed.tech/sources/arxiv-cryptography-and-security.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Federated Learning](<https://devfeed.tech/topics/federated-learning.md>), [Adversarial attacks](<https://devfeed.tech/topics/adversarial-attacks.md>), [Machine learning](<https://devfeed.tech/topics/machine-learning.md>)

Tags: [adversarial](<https://devfeed.tech/tags/adversarial.md>), [aggregation](<https://devfeed.tech/tags/aggregation.md>), [cs-cr](<https://devfeed.tech/tags/cs-cr.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [data-loss](<https://devfeed.tech/tags/data-loss.md>), [data-share](<https://devfeed.tech/tags/data-share.md>), [datasets](<https://devfeed.tech/tags/datasets.md>), [fairness](<https://devfeed.tech/tags/fairness.md>), [federated-learning](<https://devfeed.tech/tags/federated-learning.md>), [hospitals](<https://devfeed.tech/tags/hospitals.md>), [robust](<https://devfeed.tech/tags/robust.md>), [robustness](<https://devfeed.tech/tags/robustness.md>)

### AI overview

The audit finds that Byzantine-robust aggregation rules can behave differently in small, unevenly sized hospital consortia than client-count validity checks suggest. Across three FLamby medical datasets, 54,214 runs, and attacker coalitions of one or two clients, sample-size weighting made median and GeoMed effectively follow a single hospital when it held over half the data; weighted median, GeoMed, and Centered Clipping collapsed when that dominant centre attacked. Exact-twin controls separated honest-data loss from attack damage: on ISIC, losing the largest hospital cost 7.5-10.4 points for count-based rules, while adversarial perturbation accounted for 2.3-3.8 points; weight-dependent rules instead incurred 16.1-25.1 points of attack damage. Unweighted robust rules discounted the institution with the most atypical updates, which was the smallest hospital on Heart but larger institutions on TCGA and ISIC.

### Source excerpt

arXiv:2611.22904v1 Announce Type: cross Abstract: Clinical federated learning (FL) collaborations typically connect only three to six hospitals, yet Byzantine-robust aggregation rules are designed and benchmarked on synthetic partitions across dozens of equal-sized clients. We conduct an empirical audit of eight standard aggregation rules: FedAvg, coordinate-wise median, trimmed mean, Krum, Multi-Krum, Bulyan, geometric median (GeoMed), and Centered Clipping (CC). We evaluate both unweighted and sample-size-weighted variants across three cross-silo medical datasets from the FLamby benchmark: Fed-Heart-Disease ($n = 4$ hospitals), Fed-TCGA-BRCA ($n = 6$ regions), and Fed-ISIC2019 ($n = 6$ centres). We execute 54,214 training runs across enumerated attacker coalitions ($f \in \{1, 2\}$) paired with exact-twin leave-out controls that decompose performance drops into honest data loss versus adversarial attack damage. We report three findings. First, client-count validity checks fail to capture breakdown in skewed medical consortia: while $n = 6, f = 1$ satisfies every count condition except Bulyan's, assigning sample-size weights turns median and GeoMed into a clean-model dictatorship whenever a single hospital holds more than half the training data, and causes weighted median, GeoMed, and CC to collapse when that dominant centre attacks. Second, exact-twin leave-out controls reveal that when the largest hospital attacks, the apparent failure of count-based robust rules is predominantly caused by the loss of honest training data (7.5 to 10.4 points on ISIC) rather than adversarial perturbation (2.3 to 3.8 points on ISIC). Conversely, weight-dependent rules suffer large true attack damage (16.1 to 25.1 points on ISIC). Third, unweighted robust rules consistently discount whichever institution generates the most atypical parameter updates, penalizing the smallest hospital on Heart but penalizing larger institutions on TCGA and ISIC.

## DACE: Diversity-Driven Adversarial Co-Evolution for Robust LLM Safety Alignment

DevFeed: [DACE: Diversity-Driven Adversarial Co-Evolution for Robust LLM Safety Alignment](<https://devfeed.tech/articles/dace-diversity-driven-adversarial-co-evolution-for-robust-llm-safety-alignment-139998.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.22278>)

Author: Peng Yu, Xiaoyu Wen, Zhida He, Ziyuan Zhou, Han Qi, Shao Zhang, Qiaosheng Zhang, Ying Wen, Chaochao Lu

Published: 2026-11-18T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [LLM security](<https://devfeed.tech/topics/llm-security.md>), [ai alignment](<https://devfeed.tech/topics/ai-alignment.md>), [Adversarial attacks](<https://devfeed.tech/topics/adversarial-attacks.md>), [post-training](<https://devfeed.tech/topics/post-training.md>), [thompson sampling](<https://devfeed.tech/topics/thompson-sampling.md>)

Tags: [adversarial](<https://devfeed.tech/tags/adversarial.md>), [alignment](<https://devfeed.tech/tags/alignment.md>), [attack](<https://devfeed.tech/tags/attack.md>), [attacks](<https://devfeed.tech/tags/attacks.md>), [automated](<https://devfeed.tech/tags/automated.md>), [bayesian](<https://devfeed.tech/tags/bayesian.md>), [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [bernoulli](<https://devfeed.tech/tags/bernoulli.md>), [beta](<https://devfeed.tech/tags/beta.md>), [bounded](<https://devfeed.tech/tags/bounded.md>), [capabilities](<https://devfeed.tech/tags/capabilities.md>), [coverage](<https://devfeed.tech/tags/coverage.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [defender](<https://devfeed.tech/tags/defender.md>), [defense](<https://devfeed.tech/tags/defense.md>), [diversity](<https://devfeed.tech/tags/diversity.md>), [evolution](<https://devfeed.tech/tags/evolution.md>), [llm](<https://devfeed.tech/tags/llm.md>), [llm-safety](<https://devfeed.tech/tags/llm-safety.md>), [robust](<https://devfeed.tech/tags/robust.md>), [safety](<https://devfeed.tech/tags/safety.md>)

### AI overview

DACE targets attacker strategy collapse and defender forgetting during adversarial co-evolution for LLM safety alignment. It represents attacks across risk category and attack style, rewarding normalized marginal coverage gain to encourage strategy-level exploration. For defense, a Bayesian replay pool tracks threats with Beta-Bernoulli posteriors, refreshes them through time decay, and samples them using Thompson sampling. The abstract reports improved robustness to out-of-distribution attacks, broader attacker strategy coverage, and preserved general capabilities across safety, automated-attacker, and general-capability benchmarks; it provides no benchmark-specific results.

### Source excerpt

arXiv:2611.22278v1 Announce Type: new Abstract: Ensuring robust safety alignment of large language models (LLMs) is increasingly difficult as adversarial attacks evolve and outpace alignment pipelines built on pre-collected data. Existing co-evolutionary frameworks let the defender chase a moving attacker, yet their training dynamics expose two compounding pathologies: \emph{attack strategy collapse}, where the attacker overfits to a narrow set of high-reward rewrites, and \emph{defense adversarial forgetting}, where the defender loses competence against earlier attacks as the attack distribution drifts. Existing remedies are partial: attacker-side diversity rewards score textual novelty rather than strategy novelty, and defender-side replay relies on discrete judge scores that ignore estimation uncertainty, response stochasticity, and defender non-stationarity. We introduce \textbf{\method}, a diversity-driven adversarial co-evolution framework that targets both pathologies jointly. On the attacker side, \method couples an explicit two-dimensional strategy space (risk category $\times$ attack style) with a \emph{normalized marginal coverage gain} reward, providing a bounded, non-vanishing exploration signal at the strategy layer. On the defender side, \method maintains a unified \emph{Bayesian adversarial replay pool} whose Beta--Bernoulli threat posteriors are refreshed by time decay and sampled via Thompson sampling, anchoring defender training to an evolving threat landscape. Across standard safety, automated-attacker, and general-capability benchmarks, \method improves robustness to out-of-distribution attacks while preserving general capabilities, and yields broader attacker strategy coverage.

## From Strategy Search to Policy Learning: MCTS-Guided Adversarial Co-Evolution for LLM Safety Alignment

DevFeed: [From Strategy Search to Policy Learning: MCTS-Guided Adversarial Co-Evolution for LLM Safety Alignment](<https://devfeed.tech/articles/from-strategy-search-to-policy-learning-mcts-guided-adversarial-co-evolution-for-llm-safety-alignment-139751.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2511.19218>)

Author: Xurui Li, Kaisong Song

Published: 2026-11-18T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Cryptography and Security](<https://devfeed.tech/sources/arxiv-cryptography-and-security.md>)

Topics: [LLM security](<https://devfeed.tech/topics/llm-security.md>), [ai alignment](<https://devfeed.tech/topics/ai-alignment.md>), [grpo](<https://devfeed.tech/topics/grpo.md>), [Adversarial attacks](<https://devfeed.tech/topics/adversarial-attacks.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>)

Tags: [adaptive](<https://devfeed.tech/tags/adaptive.md>), [adversarial](<https://devfeed.tech/tags/adversarial.md>), [alignment](<https://devfeed.tech/tags/alignment.md>), [attacks](<https://devfeed.tech/tags/attacks.md>), [bypass](<https://devfeed.tech/tags/bypass.md>), [credit-assignment](<https://devfeed.tech/tags/credit-assignment.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cr](<https://devfeed.tech/tags/cs-cr.md>), [curriculum](<https://devfeed.tech/tags/curriculum.md>), [defender](<https://devfeed.tech/tags/defender.md>), [defense](<https://devfeed.tech/tags/defense.md>), [evolution](<https://devfeed.tech/tags/evolution.md>), [framework](<https://devfeed.tech/tags/framework.md>), [jailbreak](<https://devfeed.tech/tags/jailbreak.md>), [language-models](<https://devfeed.tech/tags/language-models.md>), [large-language-models-llms](<https://devfeed.tech/tags/large-language-models-llms.md>), [learning](<https://devfeed.tech/tags/learning.md>), [llm](<https://devfeed.tech/tags/llm.md>), [llm-safety](<https://devfeed.tech/tags/llm-safety.md>), [policy](<https://devfeed.tech/tags/policy.md>), [safety](<https://devfeed.tech/tags/safety.md>), [search](<https://devfeed.tech/tags/search.md>)

### AI overview

MACE uses Monte Carlo Tree Search to turn jailbreak discovery into learning signals for both attackers and defenders. GS-MCTS searches multi-step rewriting strategies and records attack success frequency and path depth. AC-TGPO uses those statistics for difficulty-aware credit assignment and combines current interactions with asymmetric attacker-defender pairings and persistent attacks that survive defense updates. In ablations, removing difficulty weighting, asymmetric samples, or persistent hard samples increased defense ASR-LR by 3.7, 6.0, and 5.0 percentage points, respectively. The attacker achieved the highest ASR-LR in all six evaluated model-dataset settings; the defender reduced average ASR-LR from the best baseline's 11.9% to 8.8%.

### Source excerpt

arXiv:2511.19218v3 Announce Type: replace-cross Abstract: Large Language Models (LLMs) increasingly power web-facing applications, including conversational assistants and content services, where open user interaction exposes them to diverse safety threats and adaptive attacks. In particular, jailbreak attackers repeatedly reformulate harmful requests to bypass safety alignment. Although existing studies have concentrated on defending dynamic jailbreaks, they often retain discovered attacks as flat samples and discard information about how those attacks were found. In this paper, we introduce MACE, an MCTS-guided adversarial co-evolution framework that turns search into policy-learning signals for both an attacker and a defender. Its search component, GS-MCTS, explores multi-step compositions of rewriting operators and associates discovered attacks with empirical success frequency and path depth. Its optimization component, AC-TGPO, uses these statistics for difficulty-aware group-relative credit assignment and constructs a temporally asymmetric curriculum from current interactions, pairings between the latest attacker and preceding defender, and attacks that survive a defense update. Thus, MACE connects search structure to both within-round policy optimization and cross-round memory, rather than using MCTS only to generate attacks or treating each round as a flat adversarial dataset. Removing difficulty-aware weighting, asymmetric samples, or persistent hard samples increases defense ASR-LR by 3.7 %, 6.0 %, and 5.0 %, respectively. Across evaluated models, the resulting attacker achieves the highest ASR-LR in all six model--dataset settings, while the defender reduces average ASR-LR from 11.9 % for the best baseline to 8.8 %.

## GuardAgent: A Memory-Augmented Agentic Framework for Adaptive Detection of Adversarial Promotions in LLM-Based Recommendation

DevFeed: [GuardAgent: A Memory-Augmented Agentic Framework for Adaptive Detection of Adversarial Promotions in LLM-Based Recommendation](<https://devfeed.tech/articles/guardagent-a-memory-augmented-agentic-framework-for-adaptive-detection-of-adversarial-promotions-in-llm-based-recommendation-141482.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.22274>)

Author: Shiyi Yang, Xinshu Li, Chen Wang, Xiwei Xu, Liming Zhu, Lina Yao

Published: 2026-11-18T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Cryptography and Security](<https://devfeed.tech/sources/arxiv-cryptography-and-security.md>)

Topics: [threat detection](<https://devfeed.tech/topics/threat-detection.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>), [Adversarial attacks](<https://devfeed.tech/topics/adversarial-attacks.md>), [LLM security](<https://devfeed.tech/topics/llm-security.md>), [Machine Intelligence](<https://devfeed.tech/topics/machine-intelligence.md>)

Tags: [adaptive](<https://devfeed.tech/tags/adaptive.md>), [adversarial](<https://devfeed.tech/tags/adversarial.md>), [agentic](<https://devfeed.tech/tags/agentic.md>), [attack](<https://devfeed.tech/tags/attack.md>), [cs-cr](<https://devfeed.tech/tags/cs-cr.md>), [cs-ir](<https://devfeed.tech/tags/cs-ir.md>), [datasets](<https://devfeed.tech/tags/datasets.md>), [detection](<https://devfeed.tech/tags/detection.md>), [framework](<https://devfeed.tech/tags/framework.md>), [llm](<https://devfeed.tech/tags/llm.md>), [memory](<https://devfeed.tech/tags/memory.md>), [promotion](<https://devfeed.tech/tags/promotion.md>), [reasoning](<https://devfeed.tech/tags/reasoning.md>), [recommendation](<https://devfeed.tech/tags/recommendation.md>), [recommender-systems](<https://devfeed.tech/tags/recommender-systems.md>), [reflection](<https://devfeed.tech/tags/reflection.md>), [robust](<https://devfeed.tech/tags/robust.md>)

### AI overview

GuardAgent targets promotion-driven attacks on LLM-based recommender systems, where text drives recommendation through semantic understanding and reasoning, creating a harder-to-detect attack surface. The proposed LLM-powered detector loops through Perception, Action, Reflection, and Memory, using continual learning to adapt to unseen threat patterns and producing explainable reasoning traces and outcomes. The abstract reports state-of-the-art generalization in experiments on real-world datasets; it provides no specific metrics or implementation details.

### Source excerpt

arXiv:2611.22274v1 Announce Type: cross Abstract: In traditional recommender systems (RSs), textual information typically serves as auxiliary signals for representation learning. In contrast, in LLM-based RSs, textual information becomes the primary medium for recommendation generation through semantic understanding and reasoning. This shift fundamentally changes the attack surface, making promotion-driven manipulations more feasible, effective, and harder to detect. However, effectively detecting such attacks remains largely unexplored. To fill this gap, we present the first systematic study on detecting promotion-driven attacks against LLM-based RSs, aiming to safeguard next-generation recommendation platforms. Specifically, we propose GuardAgent, a novel LLM-powered agentic detection framework that operates through a loop of Perception, Action, Reflection and Memory. In contrast to conventional LLM-based detectors that function in a static, one-off, and opaque fashion, GuardAgent continually learns and self-evolves to adapt to unseen threat patterns while generating explainable reasoning traces and outcomes. Extensive experiments on real-world datasets demonstrate that GuardAgent achieves state-of-the-art generalization performance, laying the foundation for robust and trustworthy recommendation in practical settings.

## GPTZero: Robust Detection of LLM-Generated Texts

DevFeed: [GPTZero: Robust Detection of LLM-Generated Texts](<https://devfeed.tech/articles/gptzero-robust-detection-of-llm-generated-texts-141542.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2602.13042>)

Author: George Alexandru Adam, Alexander Cui, Edwin Thomas, Emily Napier, Nazar Shmatko, Yue Gao, Zikun Chen, Jacob Schnell, Jacob Junqi Tian, Alekhya Dronavalli, Edward Tian, Dongwon Lee

Published: 2026-11-18T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Large Language Model](<https://devfeed.tech/topics/llm.md>), [LLM evaluation / benchmarking](<https://devfeed.tech/topics/llm-evaluation-benchmarking.md>), [Adversarial attacks](<https://devfeed.tech/topics/adversarial-attacks.md>), [ai-governance](<https://devfeed.tech/topics/ai-governance.md>)

Tags: [adversarial](<https://devfeed.tech/tags/adversarial.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-detection](<https://devfeed.tech/tags/ai-detection.md>), [architecture](<https://devfeed.tech/tags/architecture.md>), [arxiv](<https://devfeed.tech/tags/arxiv.md>), [assessment](<https://devfeed.tech/tags/assessment.md>), [attacks](<https://devfeed.tech/tags/attacks.md>), [automated](<https://devfeed.tech/tags/automated.md>), [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [challenge](<https://devfeed.tech/tags/challenge.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [detection](<https://devfeed.tech/tags/detection.md>), [domains](<https://devfeed.tech/tags/domains.md>), [evaluations](<https://devfeed.tech/tags/evaluations.md>), [flexible](<https://devfeed.tech/tags/flexible.md>), [independent](<https://devfeed.tech/tags/independent.md>), [industrial](<https://devfeed.tech/tags/industrial.md>), [industrial-ai](<https://devfeed.tech/tags/industrial-ai.md>), [key](<https://devfeed.tech/tags/key.md>), [llm](<https://devfeed.tech/tags/llm.md>), [robust](<https://devfeed.tech/tags/robust.md>)

### AI overview

GPTZero 5.0 is presented as an AI text detector built to distinguish human-written from LLM-generated text. Its hierarchical, multi-task architecture supports a flexible taxonomy of human and AI texts. The abstract reports top accuracy on 11 independent third-party benchmarks across 18 writing domains and improved robustness to adversarial attacks and paraphrasing, evaluated through multi-tiered automated red teaming. It also describes the system as explainable and says it educates users on responsible use. The supplied text provides no benchmark scores or details about the evaluations.

### Source excerpt

arXiv:2602.13042v2 Announce Type: replace Abstract: While historical considerations surrounding text authenticity revolved primarily around plagiarism, the advent of large language models (LLMs) has introduced a new challenge: distinguishing human-authored from AI-generated text. This shift raises significant concerns, including the undermining of skill evaluations, the mass-production of low-quality content, and the proliferation of misinformation. Addressing these issues, we introduce GPTZero 5.0, a state-of-the-art industrial AI detection solution, offering reliable discernment between human and LLM-generated text. Our key contributions include: introducing a hierarchical, multi-task architecture enabling a flexible taxonomy of human and AI texts, setting a new state-of-the-art for AI detection accuracy on 11 independent third-party benchmarks and 18 writing domains, and achieving superior robustness to adversarial attacks and paraphrasing via multi-tiered automated red teaming. GPTZero offers accurate and explainable detection, and educates users on its responsible use, ensuring fair and transparent assessment of text.

## BadGCL: Towards Persistent Backdoor Attack on Graph Continual Learning Systems

DevFeed: [BadGCL: Towards Persistent Backdoor Attack on Graph Continual Learning Systems](<https://devfeed.tech/articles/badgcl-towards-persistent-backdoor-attack-on-graph-continual-learning-systems-137480.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.19656>)

Author: Dong Li, Yufei Han, Xintao Wu, Xujiang Zhao, Wei Cheng, Chen Zhao

Published: 2026-11-17T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Adversarial attacks](<https://devfeed.tech/topics/adversarial-attacks.md>), [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [ai security](<https://devfeed.tech/topics/ai-security.md>)

Tags: [analysis](<https://devfeed.tech/tags/analysis.md>), [attack](<https://devfeed.tech/tags/attack.md>), [backdoor](<https://devfeed.tech/tags/backdoor.md>), [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [compatibility](<https://devfeed.tech/tags/compatibility.md>), [consistency](<https://devfeed.tech/tags/consistency.md>), [continual-learning](<https://devfeed.tech/tags/continual-learning.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [early-stage](<https://devfeed.tech/tags/early-stage.md>), [effectiveness](<https://devfeed.tech/tags/effectiveness.md>), [effects](<https://devfeed.tech/tags/effects.md>), [experiments](<https://devfeed.tech/tags/experiments.md>), [framework](<https://devfeed.tech/tags/framework.md>), [gradient](<https://devfeed.tech/tags/gradient.md>), [graph](<https://devfeed.tech/tags/graph.md>), [graph-neural-networks](<https://devfeed.tech/tags/graph-neural-networks.md>), [knowledge](<https://devfeed.tech/tags/knowledge.md>), [learning](<https://devfeed.tech/tags/learning.md>), [machine-learning](<https://devfeed.tech/tags/machine-learning.md>), [malicious](<https://devfeed.tech/tags/malicious.md>), [systems](<https://devfeed.tech/tags/systems.md>)

### AI overview

BadGCL targets persistent backdoors in graph continual learning (GCL), where knowledge-preservation mechanisms can retain an early malicious association if its poisoned update aligns with signals preserved from clean tasks. The model-agnostic framework learns topology-aware structural triggers with a gradient mutual-information objective linking poisoned gradients to clean-task optimization, and a propagation-consistency objective intended to stabilize their effects through GNN message passing. The authors' theoretical analysis connects gradient compatibility and propagation stability to persistence under benign updates. They report experiments across task-IL and class-IL, four GCL benchmarks, and diverse victim systems, finding persistent attack effectiveness with limited clean-utility degradation, better results than graph and continual-learning backdoor baselines, and robustness to representative defenses. Evidence is limited to the supplied abstract.

### Source excerpt

arXiv:2611.19656v1 Announce Type: cross Abstract: Backdoor attacks pose a persistent security threat to machine learning systems, yet their behavior under graph continual learning (GCL) remains underexplored. Conventional graph backdoor attacks often lose effectiveness as models are updated across successive tasks, creating the impression that continual updates naturally weaken malicious associations. This work reveals a contrasting vulnerability: knowledge-preservation mechanisms in GCL can also preserve early injected backdoors when the poisoned update is compatible with the clean-task signal being retained. For graph data, this risk is amplified by structural triggers, which are embedded into the message-passing paths of graph neural networks (GNNs) and can induce stable representation shifts across tasks. To expose this risk, we propose BadGCL, a model-agnostic framework for persistent backdoor attacks under a strict early-stage intervention setting. BadGCL learns topology-aware structural triggers using a gradient mutual-information objective that couples poisoned gradients with clean-task optimization signals, together with a propagation consistency objective that stabilizes trigger-induced effects through graph message passing. We provide theoretical analysis showing that gradient compatibility and propagation stability jointly support long-term backdoor persistence under benign continual updates. Extensive experiments across task-IL and class-IL protocols, four GCL benchmarks, and diverse victim systems show that BadGCL achieves persistent attack effectiveness with limited clean-utility degradation, substantially outperforming graph and continual-learning backdoor baselines and remaining robust against representative defenses.

## NL-SME: Enhancing Multi-Step Gradient Inversion through Adaptive Two-State Matching

DevFeed: [NL-SME: Enhancing Multi-Step Gradient Inversion through Adaptive Two-State Matching](<https://devfeed.tech/articles/nl-sme-enhancing-multi-step-gradient-inversion-through-adaptive-two-state-matching-138722.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2509.22082>)

Author: Li Xia, Jing Yu, Zheng Liu, Sili Huang, Wei Tang, Xuan Liu

Published: 2026-11-17T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Cryptography and Security](<https://devfeed.tech/sources/arxiv-cryptography-and-security.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Federated Learning](<https://devfeed.tech/topics/federated-learning.md>), [Adversarial attacks](<https://devfeed.tech/topics/adversarial-attacks.md>)

Tags: [adaptive](<https://devfeed.tech/tags/adaptive.md>), [attacks](<https://devfeed.tech/tags/attacks.md>), [cs-cr](<https://devfeed.tech/tags/cs-cr.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [data](<https://devfeed.tech/tags/data.md>), [federated](<https://devfeed.tech/tags/federated.md>), [federated-learning](<https://devfeed.tech/tags/federated-learning.md>), [gradient](<https://devfeed.tech/tags/gradient.md>), [hidden](<https://devfeed.tech/tags/hidden.md>), [local](<https://devfeed.tech/tags/local.md>), [matching](<https://devfeed.tech/tags/matching.md>), [mini](<https://devfeed.tech/tags/mini.md>), [model](<https://devfeed.tech/tags/model.md>), [model-training](<https://devfeed.tech/tags/model-training.md>), [models](<https://devfeed.tech/tags/models.md>), [privacy](<https://devfeed.tech/tags/privacy.md>), [private](<https://devfeed.tech/tags/private.md>), [quality](<https://devfeed.tech/tags/quality.md>), [raw](<https://devfeed.tech/tags/raw.md>), [sharing](<https://devfeed.tech/tags/sharing.md>), [sme](<https://devfeed.tech/tags/sme.md>), [state](<https://devfeed.tech/tags/state.md>)

### AI overview

NL-SME reconstructs private data from an individual federated-learning client's endpoint models, assuming the training labels are known. To match updates accumulated over multiple local steps without replaying training, it uses a regularized two-state gradient surrogate: an adaptive evaluation center, regularized displacement, and raw parameter gradients averaged at two shared symmetric states. The abstract reports improved reconstruction quality over existing gradient inversion methods on FEMNIST and CIFAR-100 across varying numbers of local steps, underscoring the privacy risk of sharing individual client updates. It does not provide quantitative results or details of the compared methods.

### Source excerpt

arXiv:2509.22082v4 Announce Type: replace Abstract: Federated learning (FL) enables collaborative model training among multiple clients without directly sharing their raw data.However, gradient inversion attacks can exploit the model updates exchanged during training to reconstruct clients' private data.With multiple local training steps, an observed update combines gradients evaluated at changing model states and potentially different mini-batches, while intermediate states remain hidden. Restricting gradient evaluation to a single state on the segment between the initial and final models can exclude states that better fit the accumulated update.We consider reconstruction from an individual client's endpoint models, assuming that the training labels are known.We propose NL-SME, a regularized two-state gradient surrogate for matching multi-step client updates without replaying local training.The surrogate adapts its evaluation center, regularizes its displacement, and averages raw parameter gradients at two shared symmetric states.We evaluate NL-SME against existing gradient inversion methods on FEMNIST and CIFAR-100 with varying numbers of local training steps.The results show improved reconstruction quality, highlighting the privacy risks of sharing individual client updates after multiple local steps.

## Ambiguous Examples Help, But Full Data Wins: Revisiting Dataset Cartography for Extractive Adversarial QA

DevFeed: [Ambiguous Examples Help, But Full Data Wins: Revisiting Dataset Cartography for Extractive Adversarial QA](<https://devfeed.tech/articles/ambiguous-examples-help-but-full-data-wins-revisiting-dataset-cartography-for-extractive-adversarial-qa-138926.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.19763>)

Author: Paul K. Mandal

Published: 2026-11-17T05:00:00Z

Content type: news

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [Dataset development](<https://devfeed.tech/topics/dataset-development.md>), [Adversarial attacks](<https://devfeed.tech/topics/adversarial-attacks.md>)

Tags: [adversarial](<https://devfeed.tech/tags/adversarial.md>), [cs-cl](<https://devfeed.tech/tags/cs-cl.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [dataset](<https://devfeed.tech/tags/dataset.md>), [evaluation](<https://devfeed.tech/tags/evaluation.md>), [qa](<https://devfeed.tech/tags/qa.md>), [robustness](<https://devfeed.tech/tags/robustness.md>), [train](<https://devfeed.tech/tags/train.md>), [training](<https://devfeed.tech/tags/training.md>)

### AI overview

In extractive adversarial question answering, selecting the 33% most ambiguous SQuAD examples using clean-source training dynamics generally outperforms a random 33% subset, but does not replace training on the full dataset. Ambiguous-ranked subsets beat random subsets on exact match in all 14 model-evaluation cells and on F1 in 12 of 14; full-data training beat ambiguous-ranked training on both metrics in 12 of 14 cells, with mean gains of 3.40 exact-match points and 4.10 F1 points. The study evaluates adversarial-only AddSent and AddOneSent rows. ELECTRA-small is the main exception to the ambiguous-over-random F1 gains. These findings support cartography for reduced-data selection while limiting claims that its out-of-distribution advantage generalizes as a full-data replacement.

### Source excerpt

arXiv:2611.19763v1 Announce Type: cross Abstract: Dataset cartography argues that training dynamics can reveal regions of a dataset that matter differently for learning: easy-to-learn examples aid optimization, hard-to-learn examples often expose annotation problems, and ambiguous examples contribute strongly to out-of-distribution generalization. A striking implication of the original work is that a model trained on only 33% ambiguous examples can outperform full-data training on several OOD evaluations, including a QNLI reformulation of SQuAD evaluated on Adversarial SQuAD. We revisit this claim in a stricter setting: open-ended extractive question answering, where models must predict answer spans rather than classify question-sentence pairs. For each encoder family, we compute clean-source SQuAD training dynamics, train on full, random, easy-ranked, ambiguous-ranked, and hard-ranked subsets, and evaluate robustness on adversarial-only AddSent and AddOneSent rows. Ambiguous-ranked 33% subsets beat random 33% subsets on exact-match accuracy in all 14 model-evaluation cells and on F1 in 12/14 cells. However, full-data training beats ambiguous-ranked training in 12/14 exact-match cells and 12/14 F1 cells, with mean full-minus-ambiguous gains of 3.40 EM and 4.10 F1. The most thoroughly replicated ELECTRA-small panel is also the main exception to the ambiguous-over-random F1 gains. These results show that clean-source cartography remains useful for reduced-data selection, but its OOD advantage does not carry over as a general replacement for full-data training in extractive adversarial QA.

[Next page](<https://devfeed.tech/topics/adversarial-attacks.md?cursor=WyIyMDI2LTExLTE3VDA1OjAwOjAwKzAwOjAwIiwgImI1YmI2YzhmLTRlYTgtNDI2Ni05ZTMwLWJlZWQ3Zjg3Mjk4MyJd>)