# Breaking the Privacy Promise: Gradient Inversion Attacks on Federated Low-Rank Adapted Vision Transformers

DevFeed: [Breaking the Privacy Promise: Gradient Inversion Attacks on Federated Low-Rank Adapted Vision Transformers](<https://devfeed.tech/articles/breaking-the-privacy-promise-gradient-inversion-attacks-on-federated-low-rank-adapted-vision-transformers-122807.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.09652>)

Author: Ritik K. Badiya, Siladittya Manna, Ashmit Sinha, M Yashwanth, Anirban Chakraborty

Published: 2026-11-09T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Cryptography and Security](<https://devfeed.tech/sources/arxiv-cryptography-and-security.md>), [arXiv: Distributed Computing](<https://devfeed.tech/sources/arxiv-distributed-computing.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [lora](<https://devfeed.tech/topics/lora.md>), [Federated Learning](<https://devfeed.tech/topics/federated-learning.md>), [peft](<https://devfeed.tech/topics/peft.md>), [Computer vision](<https://devfeed.tech/topics/computer-vision.md>), [Image processing](<https://devfeed.tech/topics/image-processing.md>), [Cybersecurity](<https://devfeed.tech/topics/cybersecurity.md>)

Tags: [abstract](<https://devfeed.tech/tags/abstract.md>), [act](<https://devfeed.tech/tags/act.md>), [also](<https://devfeed.tech/tags/also.md>), [artifacts](<https://devfeed.tech/tags/artifacts.md>), [arxiv](<https://devfeed.tech/tags/arxiv.md>), [attacks](<https://devfeed.tech/tags/attacks.md>), [based](<https://devfeed.tech/tags/based.md>), [batch](<https://devfeed.tech/tags/batch.md>), [blocks](<https://devfeed.tech/tags/blocks.md>), [can](<https://devfeed.tech/tags/can.md>), [cars](<https://devfeed.tech/tags/cars.md>), [cifar-10](<https://devfeed.tech/tags/cifar-10.md>), [client](<https://devfeed.tech/tags/client.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cr](<https://devfeed.tech/tags/cs-cr.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [cs-dc](<https://devfeed.tech/tags/cs-dc.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [federated](<https://devfeed.tech/tags/federated.md>), [gradient](<https://devfeed.tech/tags/gradient.md>), [privacy](<https://devfeed.tech/tags/privacy.md>), [rank](<https://devfeed.tech/tags/rank.md>), [transformers](<https://devfeed.tech/tags/transformers.md>), [vision](<https://devfeed.tech/tags/vision.md>), [vision-transformers](<https://devfeed.tech/tags/vision-transformers.md>)

## AI overview

The study finds that LoRA fine-tuning alone does not prevent private image leakage from federated Vision Transformers when a server can access individual client updates. It attributes reconstruction ambiguity to weak low-rank gradient signals and missing spatial cues from frozen positional embeddings. The proposed framework adds server-prescribed trainable positional embeddings during client fine-tuning; their gradients provide layer-wise spatial anchors, while a generative prior reduces duplicate-data and patch-permutation artifacts. When added to GIAS, this approach improves reconstruction metrics in FedSGD: on CIFAR-10, PSNR rises 10.14 dB, SSIM 0.45, and LPIPS improves by 0.098; PSNR gains are 7.83 dB on CelebA and 11.74 dB on CARS. The abstract reports that gains persist at higher batch sizes and with FedAvg.

## Source excerpt

arXiv:2611.09652v1 Announce Type: cross Abstract: Parameter-efficient fine-tuning (PEFT) has been increasingly adopted to fine-tune pretrained models with low computational cost and is therefore also preferred in the federated learning paradigm to reduce computational and communication overhead. In this work, we investigate the sparsely explored problem of the risk of private data leakage in LoRA-adapted Vision Transformers (ViTs), where low-rank updates weaken the gradient signal available to a server with access to individual client updates. We show that quality of reconstruction is limited by two coupled ambiguities: reduced information in LoRA gradients and missing spatial cues from frozen positional embeddings. To address this, we propose a spatially anchored generative inversion framework that incorporates server-prescribed trainable positional embeddings into ViT blocks during fine-tuning at clients. The resulting positional gradients act as layer-wise spatial anchors, providing additional positional cues, while the generative prior keeps reconstructions on a realistic image manifold, reducing duplicate-data and patch-permutation artifacts, {which produce similar gradients as those produced when the target client images are used}. Adding our proposed spatial anchoring to GIAS improves PSNR by 10.14 dB, SSIM by 0.45, and LPIPS by 0.098 on CIFAR-10, as well as PSNR gains of 7.83 and 11.74 dB on high-resolution CelebA and CARS datasets, respectively, in the FedSGD setting. Such gains in reconstructed image quality metrics are also maintained at higher batch sizes and in FedAvg settings. These results indicate that LoRA-based PEFT does not, by itself, prevent data leakage in FL, and carefully incorporated positional embeddings can help in exposing private information.