# When Should an LLM Steering Edit Depend on the Input?

DevFeed: [When Should an LLM Steering Edit Depend on the Input?](<https://devfeed.tech/articles/when-should-an-llm-steering-edit-depend-on-the-input-117521.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.04757>)

Author: Rachael Harkness, Konrad Rawlik

Published: 2026-11-04T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [AI Chat](<https://devfeed.tech/topics/ai-chat.md>)

Tags: [abstract](<https://devfeed.tech/tags/abstract.md>), [activation](<https://devfeed.tech/tags/activation.md>), [activation-steering](<https://devfeed.tech/tags/activation-steering.md>), [addition](<https://devfeed.tech/tags/addition.md>), [apply](<https://devfeed.tech/tags/apply.md>), [arxiv](<https://devfeed.tech/tags/arxiv.md>), [benchmark](<https://devfeed.tech/tags/benchmark.md>), [best](<https://devfeed.tech/tags/best.md>), [can](<https://devfeed.tech/tags/can.md>), [changes](<https://devfeed.tech/tags/changes.md>), [class](<https://devfeed.tech/tags/class.md>), [conditional](<https://devfeed.tech/tags/conditional.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [difference](<https://devfeed.tech/tags/difference.md>), [edits](<https://devfeed.tech/tags/edits.md>), [express](<https://devfeed.tech/tags/express.md>), [external](<https://devfeed.tech/tags/external.md>), [first](<https://devfeed.tech/tags/first.md>), [input](<https://devfeed.tech/tags/input.md>), [llm](<https://devfeed.tech/tags/llm.md>), [steering](<https://devfeed.tech/tags/steering.md>)

## AI overview

Steering should depend on the input when the desired destination varies within the requested class. A fixed Contrastive Activation Addition (CAA) direction cannot capture that within-class variation; conditioning on the class captures between-class differences, while retrieval averages the k nearest stored destination states within that class. Across six substitution corpora and an external benchmark, input-dependent steering outperformed fixed-direction steering when within-class destinations varied. In two models, retrieval led at every tested layer on the substitution corpora: word-level edits worked best near the network's first tenth using a few neighbours, while clause-level edits worked at any depth using the class mean. When the class determines the answer, the class mean sufficed and retrieval only matched it. The abstract says four state-derived quantities identify which regime applies.

## Source excerpt

arXiv:2611.04757v1 Announce Type: new Abstract: An activation steering edit should depend on the input exactly when the destination varies \emph{inside} the class a query requests, and that is measurable before any rule is fitted or applied. Contrastive Activation Addition (CAA) adds one fixed direction to every input, the difference of the class means. Conditioning it on the requested class accounts for the between-class variation in the required displacement and leaves the within-class variation untouched, which no rule that reads only the source state can express. We study the impact of increasing input-dependence in the steering edit, moving from a difference of means to a destination retrieved per query --- the mean of the $k$ stored states nearest the query, inside the requested class --- in which the two baselines are endpoints: at $k$ equal to the class size the retrieved rule \emph{is} the class-conditional difference of means. We measure input-dependent steering on six substitution corpora that vary in whether the edit changes a word or a clause, and on an external benchmark, and find that input-dependent steering outperforms fixed-direction steering when the destination varies within the class. Retrieval wins at every layer of every substitution corpus in two models, and the level of input-dependence needed governs how to apply it: word-level edits are best made in the first tenth of the network from a few neighbours, clause-level edits at any depth from the class mean. Where the class already determines the answer the class mean suffices and retrieval only matches it, though a fixed direction still fails there. Four quantities computed from the states alone identify the regime in which the edit should be applied.