# SecPO: Principled Adversarial Training for Prompt Injection Security

DevFeed: [SecPO: Principled Adversarial Training for Prompt Injection Security](<https://devfeed.tech/articles/secpo-principled-adversarial-training-for-prompt-injection-security-111235.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2610.41998>)

Author: Sizhe Chen, Yibo Peng, Jaewon Chang, Chawin Sitawarin, David Wagner

Published: 2026-11-02T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Cryptography and Security](<https://devfeed.tech/sources/arxiv-cryptography-and-security.md>)

Topics: [prompt injection](<https://devfeed.tech/topics/prompt-injection.md>), [LLM security](<https://devfeed.tech/topics/llm-security.md>), [Adversarial attacks](<https://devfeed.tech/topics/adversarial-attacks.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>), [Fine-tuning](<https://devfeed.tech/topics/fine-tuning.md>), [dpo](<https://devfeed.tech/topics/dpo.md>)

Tags: [adaptive](<https://devfeed.tech/tags/adaptive.md>), [adversarial](<https://devfeed.tech/tags/adversarial.md>), [agent](<https://devfeed.tech/tags/agent.md>), [agentic](<https://devfeed.tech/tags/agentic.md>), [agentic-workflows](<https://devfeed.tech/tags/agentic-workflows.md>), [agents](<https://devfeed.tech/tags/agents.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-agents](<https://devfeed.tech/tags/ai-agents.md>), [asr](<https://devfeed.tech/tags/asr.md>), [attack](<https://devfeed.tech/tags/attack.md>), [attacks](<https://devfeed.tech/tags/attacks.md>), [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [code](<https://devfeed.tech/tags/code.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cr](<https://devfeed.tech/tags/cs-cr.md>), [data](<https://devfeed.tech/tags/data.md>), [dpo](<https://devfeed.tech/tags/dpo.md>), [email](<https://devfeed.tech/tags/email.md>), [prompt](<https://devfeed.tech/tags/prompt.md>), [prompt-injection](<https://devfeed.tech/tags/prompt-injection.md>), [security](<https://devfeed.tech/tags/security.md>), [training](<https://devfeed.tech/tags/training.md>)

## AI overview

SecPO combines a theoretically motivated training loss with adversarial training to improve prompt injection resistance. The authors argue DPO can be limited by anchoring to an insecure reference model; SecPO instead approximates an ideal injection-invariant, utility-preserving reference. Their training pipeline synthesizes stronger injected inputs (best@240). Against Meta-SecAlign, adversarial training alone lowers attack success rates by 1.5-14.5x across three adaptive attacks, and SecPO lowers them by a further 2.4-4.6x versus DPO. On AgentDojo under AttackerMovesSecond, SecPO plus adversarial training with Qwen3.6-27B has 8.8% ASR@800 versus 78.8% for Meta-SecAlign. The authors report no noticeable benign utility reduction across nine benchmarks spanning tool calling, instruction following, science, and math. This overview is based on the supplied abstract.

## Source excerpt

arXiv:2610.41998v1 Announce Type: new Abstract: Recognized as a top threat to AI agents, prompt injection allows instructions from untrusted data (email, documents, websites) to manipulate the agent's operation. The existing best-defended LLM, Meta-SecAlign, is trained with the DPO loss but still suffers from attack success rates (ASRs) approaching 100% under adaptive prompt injections. One reason is that DPO anchors the model around the undefended reference model, which is insecure. We show an idealized reference model---an injection-invariant, utility-preserving oracle---can nevertheless be approximated, as its output distribution in training is interestingly accessible. This new observation motivates a theoretically principled SecPO loss. Another Meta-SecAlign shortcoming is that it trains on weak simulated attacks. We implement an efficient adversarial training (AT) pipeline that synthesizes strong injected inputs (best@240), and train the model on them. Our AT reduces ASRs by 1.5-14.5$\times$ across three SoTA adaptive attacks, compared with Meta-SecAlign; the SecPO loss reduces them by a further 2.4-4.6$\times$ compared with using DPO loss. In agentic workflows, our SecPO + AT Qwen3.6-27B is much more secure than Meta-SecAlign: 8.8% vs. 78.8% AgentDojo ASR@800 under the AttackerMovesSecond attack. Moreover, our defended model shows no noticeable reduction in benign utility in nine benchmarks covering agentic tool calling, instruction following, science, and math. We release our $\href{//github.com/Sizhe-Chen/SecPO}{\text{code}}$, $\href{//huggingface.co/datasets/Sizhe-Chen/Qwen3.6-27B-Instruct-SecPO-trainset}{\text{training data}}$ (with optimized injections from AT), and $\href{//huggingface.co/Sizhe-Chen/Qwen3.6-27B-Instruct-SecPO}{\text{model}}$.