# Auditable Unlearning for LLMs

DevFeed: [Auditable Unlearning for LLMs](<https://devfeed.tech/articles/auditable-unlearning-for-llms-118393.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.05866>)

Author: Piyush Tiwary, Anushka Srivastava, Prathosh A. P

Published: 2026-11-05T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [Large Language Model](<https://devfeed.tech/topics/llm.md>)

Tags: [artificial-intelligence](<https://devfeed.tech/tags/artificial-intelligence.md>), [attacks](<https://devfeed.tech/tags/attacks.md>), [audit](<https://devfeed.tech/tags/audit.md>), [auditability](<https://devfeed.tech/tags/auditability.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [inference](<https://devfeed.tech/tags/inference.md>), [llms](<https://devfeed.tech/tags/llms.md>), [machine-unlearning](<https://devfeed.tech/tags/machine-unlearning.md>), [model](<https://devfeed.tech/tags/model.md>), [muse](<https://devfeed.tech/tags/muse.md>), [oracle](<https://devfeed.tech/tags/oracle.md>), [performance](<https://devfeed.tech/tags/performance.md>), [statistics](<https://devfeed.tech/tags/statistics.md>)

## AI overview

The authors add auditability as a fourth criterion for machine unlearning: whether a provider-computed statistic separates data requested for deletion from retained data. Across eleven methods on TOFU, MUSE, and RWKU, none met all four criteria of forgetting, capability retention, resistance to membership inference, and auditability. Their two-stage add-on first uses GapWiden, a hinge loss that widens the audit-statistic gap, then GapVeil, an inference-time rejection sampler that drafts with the base model and resamples using those statistics. The authors report oracle parity for auditability and leakage across methods, forgetting performance converging to the oracle, and improved retention for nine methods, though retention remained below parity.

## Source excerpt

arXiv:2611.05866v1 Announce Type: new Abstract: Machine unlearning aims to remove a user's data from a trained model after a deletion request. Existing methods are majorly evaluated on three axes - how well they forget, how much capability they retain, and how robust they are to membership inference attacks, but not on whether the provider can offer the user any evidence that the deletion occurred. To fill this gap, we introduce auditability as a fourth axis, defined as the separation between requested (forget set) and retained (retain set) data under a statistic the provider computes. We show that across eleven unlearning methods on TOFU, MUSE and RWKU no method satisfies all four axes, with different methods falling short on different axes. To address this, we propose a two stage method that can be used on top of any unlearning method, in the the first stage we propose ${GapWiden}$ which uses a hinge loss to increase the difference between audit statistics of forget set and retain set. In the second stage we propose ${GapVeil}$, an inference time rejection-sampler which uses the base model as the drafter and re-samples based on the audit statistics of the drafted response. We observe that using our method, auditability and leakage reach oracle parity for every unlearning method, and forget performance converges to the oracle; retain performance improves for nine of eleven methods but remains short of parity. Our results show that evidence of deletion or auditability can be constructed by design, and that treating it as an explicit requirement changes the viability of existing unlearning methods.