# Using group theory to explore the space of positional encodings for attention

DevFeed: [Using group theory to explore the space of positional encodings for attention](<https://devfeed.tech/articles/using-group-theory-to-explore-the-space-of-positional-encodings-for-attention-20221.md>)

Original publisher: [Read original article](<https://blog.janestreet.com/using-group-theory-to-explore-positional-encodings-attention/>)

Author: Alok Puranik

Published: 2026-04-22T00:00:00Z

Content type: article

Language: en

Sources: [Jane Street](<https://devfeed.tech/sources/jane-street.md>)

Topics: [Language models](<https://devfeed.tech/topics/language-models.md>), [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [Temporal data](<https://devfeed.tech/topics/temporal-data.md>)

Tags: [analysis](<https://devfeed.tech/tags/analysis.md>), [blog-post](<https://devfeed.tech/tags/blog-post.md>), [language-models](<https://devfeed.tech/tags/language-models.md>), [machine-learning](<https://devfeed.tech/tags/machine-learning.md>), [ml](<https://devfeed.tech/tags/ml.md>)

## AI overview

The article examines positional encodings for attention using group theory. It argues that formalizing desirable properties leaves only a few valid families, most of which are already used in real systems, while also identifying a technically valid but apparently unexplored class.

## Source excerpt

Attention is a computational primitive at the core of modern language models, allowing internal representations to reference and influence each other. It's how these models handle sequential data in the first place.