# ReplayAudit: Auditing Outsourced LLM Fine-Tuning via Sealed Records and Selective Replay

DevFeed: [ReplayAudit: Auditing Outsourced LLM Fine-Tuning via Sealed Records and Selective Replay](<https://devfeed.tech/articles/replayaudit-auditing-outsourced-llm-fine-tuning-via-sealed-records-and-selective-replay-122806.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.09607>)

Author: Yihan Xia, Wenxin Xu, Taotao Wang

Published: 2026-11-09T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Cryptography and Security](<https://devfeed.tech/sources/arxiv-cryptography-and-security.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [LLM security](<https://devfeed.tech/topics/llm-security.md>), [Fine-tuning](<https://devfeed.tech/topics/fine-tuning.md>), [lora](<https://devfeed.tech/topics/lora.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>)

Tags: [auditing](<https://devfeed.tech/tags/auditing.md>), [cs-cr](<https://devfeed.tech/tags/cs-cr.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [fine-tuning](<https://devfeed.tech/tags/fine-tuning.md>), [llm-fine-tuning](<https://devfeed.tech/tags/llm-fine-tuning.md>), [lora](<https://devfeed.tech/tags/lora.md>), [records](<https://devfeed.tech/tags/records.md>), [sampling](<https://devfeed.tech/tags/sampling.md>), [sealed](<https://devfeed.tech/tags/sealed.md>), [telemetry](<https://devfeed.tech/tags/telemetry.md>), [training](<https://devfeed.tech/tags/training.md>), [verification](<https://devfeed.tech/tags/verification.md>), [work](<https://devfeed.tech/tags/work.md>)

## AI overview

ReplayAudit audits outsourced LLM fine-tuning by sealing training records before challenges and independently replaying sampled intervals from sealed checkpoints. Loss changes and parameter-update norms guide stratified random sampling; replay checks whether selected computations match the records. In real LoRA training, a 7B BF16 prototype caught a false update-norm report in a random audit, while targeted checks caught false loss reports and skipped updates. Offline-calibrated budgets aimed for at least a 90% chance of directly selecting an injected step. On an honest 1.5B trajectory, two learning-rate deviation settings reduced median complete-audit time by 49% and 67% versus uniform sampling; when skipped updates avoided the high-anomaly stratum, audit cost was 38% higher. The results indicate that audit cost depends on both deviation capture and replay work.

## Source excerpt

arXiv:2611.09607v1 Announce Type: new Abstract: Auditing outsourced LLM fine-tuning requires checking an untrusted provider's training while controlling verification cost. Uniform sampling ignores training-time signals, while checking a suspicious record can require replaying several preceding steps. We present ReplayAudit, a telemetry-guided auditing system that seals training records before challenges and independently replays sampled intervals from sealed checkpoints. Loss changes and parameter-update norms guide stratified random sampling; replay determines whether the selected computation agrees with its records. We relate sampling probabilities to replay work and evaluate the system on real LoRA training. A 7B BF16 prototype rejects a false update-norm report during a random audit; targeted checks also reject false loss reports and skipped updates. We calibrate sampling budgets offline to at least a 90% probability of directly selecting an injected step. For two learning-rate deviation settings, executing these budgets on the same honest 1.5B trajectory reduces median complete-audit wall time by 49% and 67% relative to uniform sampling. When skipped updates evade the high-anomaly stratum, the corresponding audit instead costs 38% more. These results show how deviation capture and actual replay work jointly determine the cost of telemetry-guided auditing.