# Anchor Behavior, Not Parameters: Rehearsal-Free Continual Learning with a Single Shared Prompt

DevFeed: [Anchor Behavior, Not Parameters: Rehearsal-Free Continual Learning with a Single Shared Prompt](<https://devfeed.tech/articles/anchor-behavior-not-parameters-rehearsal-free-continual-learning-with-a-single-shared-prompt-125590.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.12777>)

Author: Duy-Cuong Bui, Trung-Anh Dang, Vincent Nguyen, Duc V. Le, Ngoc-Son Vu

Published: 2026-11-10T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Fine-tuning](<https://devfeed.tech/topics/fine-tuning.md>), [peft](<https://devfeed.tech/topics/peft.md>), [prompt](<https://devfeed.tech/topics/prompt.md>)

Tags: [anchor](<https://devfeed.tech/tags/anchor.md>), [attention](<https://devfeed.tech/tags/attention.md>), [behavior](<https://devfeed.tech/tags/behavior.md>), [continual-learning](<https://devfeed.tech/tags/continual-learning.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [function](<https://devfeed.tech/tags/function.md>), [plasticity](<https://devfeed.tech/tags/plasticity.md>), [prompt](<https://devfeed.tech/tags/prompt.md>), [query](<https://devfeed.tech/tags/query.md>), [relevance](<https://devfeed.tech/tags/relevance.md>), [statistics](<https://devfeed.tech/tags/statistics.md>), [token](<https://devfeed.tech/tags/token.md>), [transformer](<https://devfeed.tech/tags/transformer.md>)

## AI overview

TEA-Prompt aims to retain old-class knowledge while one shared prompt learns new tasks, without storing old images. It uses input-conditional token relevance scores in attention logits to softly specialize prompt tokens, then anchors each token's attention output around class-specific query regions using compact Gaussian query statistics weighted by past relevance. This preserves attention behavior while allowing prompt parameters to keep updating. The abstract reports state-of-the-art accuracy and markedly lower forgetting on CIFAR-100, ImageNet-R, and CUB-200 with 0.38M parameters; it does not provide numerical comparisons.

## Source excerpt

arXiv:2611.12777v1 Announce Type: new Abstract: Prompt-based continual learning adapts a frozen pre-trained vision transformer by tuning a small set of prompt tokens. Recently, one-prompt methods which share one single prompt across all tasks make this paradigm maximally parameter efficient. The open question is how a shared prompt can keep the knowledge of old classes while being updated for new ones. Existing methods protect old knowledge in parameter space, where they freeze task-specific prompts, expand new components, or sparsely gate which tokens receive gradients. However, this method sacrifices plasticity, yet the drift it still permits in shared tokens can corrupt the prompt's function on past classes. What must be preserved is the prompt's behavior. We propose TEA-Prompt, which couples (i) a token evaluation module that assigns each prompt token a continuous, input-conditional relevance score injected into the attention logits, softly specializing tokens across tasks without discarding any of them, and (ii) an attention-behavior anchoring loss that constrains each token's attention output around class-specific query regions using only compact Gaussian query statistics without stored old images, weighted by the token's past relevance. Anchoring the attention value allows the parameters to continue updating freely, as long as their interaction remains unchanged. TEA-Prompt achieves state-of-the-art accuracy with markedly lower forgetting on CIFAR-100, ImageNet-R, and CUB-200 with only 0.38M parameters. Code is publicy available at https://github.com/cuonga1cvp/TEA-Prompt.git