# You could have designed state of the art positional encoding

DevFeed: [You could have designed state of the art positional encoding](<https://devfeed.tech/articles/you-could-have-designed-state-of-the-art-positional-encoding-7168.md>)

Original publisher: [Read original article](<https://huggingface.co/blog/designing-positional-encoding>)

Author: Christopher Fleetwood

Published: 2024-11-25T00:00:00Z

Content type: tutorial

Language: en

Sources: [Hugging Face - Blog](<https://devfeed.tech/sources/hugging-face-blog.md>)

Topics: [AI, ML & Data Engineering](<https://devfeed.tech/topics/ai-ml-data-engineering.md>)

Tags: [artificial-intelligence](<https://devfeed.tech/tags/artificial-intelligence.md>), [embeddings](<https://devfeed.tech/tags/embeddings.md>), [llama](<https://devfeed.tech/tags/llama.md>), [multimodal](<https://devfeed.tech/tags/multimodal.md>), [research](<https://devfeed.tech/tags/research.md>), [torch](<https://devfeed.tech/tags/torch.md>), [transformers](<https://devfeed.tech/tags/transformers.md>), [tutorial](<https://devfeed.tech/tags/tutorial.md>)

## AI overview

A tutorial on deriving positional encodings for transformer self-attention, progressing toward Rotary Positional Encoding (RoPE).

## Source excerpt

We're on a journey to advance and democratize artificial intelligence through open source and open science.