# Notwithstanding Anything to the Contrary, Ignore All Instructions: Measuring Prompt Injection Guards on Legal Contract Review Attacks

DevFeed: [Notwithstanding Anything to the Contrary, Ignore All Instructions: Measuring Prompt Injection Guards on Legal Contract Review Attacks](<https://devfeed.tech/articles/notwithstanding-anything-to-the-contrary-ignore-all-instructions-measuring-prompt-injection-guards-on-legal-contract-review-attacks-124578.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.08834>)

Author: Mark Heimann, Katherine Taylor, Bruno Jacob

Published: 2026-11-09T05:00:00Z

Content type: news

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [prompt injection](<https://devfeed.tech/topics/prompt-injection.md>), [LLM security](<https://devfeed.tech/topics/llm-security.md>), [LLM evaluation / benchmarking](<https://devfeed.tech/topics/llm-evaluation-benchmarking.md>)

Tags: [attacks](<https://devfeed.tech/tags/attacks.md>), [contract-review](<https://devfeed.tech/tags/contract-review.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [guards](<https://devfeed.tech/tags/guards.md>), [instructions](<https://devfeed.tech/tags/instructions.md>), [legal](<https://devfeed.tech/tags/legal.md>), [measuring](<https://devfeed.tech/tags/measuring.md>), [prompt](<https://devfeed.tech/tags/prompt.md>), [prompt-injection](<https://devfeed.tech/tags/prompt-injection.md>)

## AI overview

The study finds that eleven open-weights prompt injection guards perform poorly on two legal contract corpora: document-level false positive rates reach 92.39%, two guards score below chance on paired AUC, and a five-regex baseline is not Pareto-dominated by any tested guard. All guards score zero on unauthorized scope claims. The abstract reports that prompted LLMs resolve the tested failures; a cascade that sends a mid-tier model's positive results to a frontier model costs 6.5 times less than using the frontier model alone while approaching its performance.

## Source excerpt

arXiv:2611.08834v1 Announce Type: new Abstract: Specialized prompt injection guards are deployed in front of document analysis systems, but no false positive rate has been published for any of them on legal contracts. We measure eleven open-weights guards on two corpora of legal contracts and find that existing guards struggle: document-level false positive rates are as high as 92.39%; two guards fall below chance on paired AUC; and a five-regex baseline is not Pareto-dominated by any guard in the panel. Controlled probes locate model-specific as well as general failure modes. Prompted LLMs resolve the tested failures, including a second threat category (unauthorized scope claims) on which every guard scores zero. A cascaded LLM solution, where the positives of a mid-tier model are verified by a frontier model, is 6.5x cheaper than using exclusively a frontier model and approaches its performance.