# Can LLMs Write Backdoors? An Empirical Security Study of Weight-Space Programming

DevFeed: [Can LLMs Write Backdoors? An Empirical Security Study of Weight-Space Programming](<https://devfeed.tech/articles/can-llms-write-backdoors-an-empirical-security-study-of-weight-space-programming-135063.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.17395>)

Author: Zhaojia Wang, Jiucui Lu, Zheng Li

Published: 2026-11-16T05:00:00Z

Content type: news

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Cryptography and Security](<https://devfeed.tech/sources/arxiv-cryptography-and-security.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [LLM security](<https://devfeed.tech/topics/llm-security.md>), [ai security](<https://devfeed.tech/topics/ai-security.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>), [Neural Network](<https://devfeed.tech/topics/neural-network.md>)

Tags: [backdoors](<https://devfeed.tech/tags/backdoors.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cr](<https://devfeed.tech/tags/cs-cr.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [llms](<https://devfeed.tech/tags/llms.md>), [security](<https://devfeed.tech/tags/security.md>), [weight-space-programming](<https://devfeed.tech/tags/weight-space-programming.md>)

## AI overview

The study finds that LLMs can inject backdoors by editing model weights from natural-language instructions, without task-specific data, forward passes, or gradient calculations, while largely preserving clean performance. It evaluates five LLMs on an MNIST MLP, CIFAR-10 ResNet-18, and Qwen2.5-0.5B-Instruct on AG News. Some attacks modified very few weights: six parameters in the MLP and 0.02% of weights in a fine-tuned Qwen model. Results varied by editor and target: all five LLMs backdoored the MLP, but only GPT-6 Astra succeeded across all three targets. The findings indicate a security risk in LLM-driven weight editing.

## Source excerpt

arXiv:2611.17395v1 Announce Type: cross Abstract: Recent advances in weight-space programming (WSP) have demonstrated that large language models (LLMs) can directly inspect and modify neural network weights without conventional gradient-based training. While this capability opens new possibilities for model improvement, it also raises an important security question: Can LLMs exploit weight-space programming to inject backdoors into neural networks? To answer this question, we present an empirical security study of LLM-driven weight-space programming for backdoor injection. Specifically, we investigate whether general-purpose LLMs can directly modify the weights of target models according to natural-language instructions, without accessing task-specific data, performing forward passes, or computing gradients. We evaluate five LLMs as weight editors across three target architectures: an MLP on MNIST, ResNet-18 on CIFAR-10, and Qwen2.5-0.5B-Instruct on AG News. Our experiments reveal three key findings. First, LLM-driven weight editing can successfully inject backdoors across different architectures while largely preserving clean performance. Second, successful attacks can require remarkably few parameter modifications, including only six parameters for the MLP and 0.02% of the weights for a fine-tuned Qwen model. Third, editing capabilities vary substantially across LLMs and target architectures: although all five LLMs successfully backdoor the MLP, only GPT-6 Astra succeeds across all three targets. These findings highlight an emerging security risk of weight-space programming and underscore the importance of safeguarding LLM-driven weight editing.