# SmolLM3: smol, multilingual, long-context reasoner

DevFeed: [SmolLM3: smol, multilingual, long-context reasoner](<https://devfeed.tech/articles/smollm3-smol-multilingual-long-context-reasoner-7482.md>)

Original publisher: [Read original article](<https://huggingface.co/blog/smollm3>)

Author: Elie Bakouch; Carlos Miguel Patiño; Anton Lozhkov; Edward Beeching; Aymeric Roucher; Nouamane Tazi; Aksel Joonas Reedi; Guilherme Penedo; Hynek Kydlicek; Clémentine Fourrier

Published: 2025-07-08T00:00:00Z

Content type: article

Language: en

Sources: [Hugging Face - Blog](<https://devfeed.tech/sources/hugging-face-blog.md>)

Topics: [AI Chat](<https://devfeed.tech/topics/ai-chat.md>)

Tags: [architecture](<https://devfeed.tech/tags/architecture.md>), [artificial-intelligence](<https://devfeed.tech/tags/artificial-intelligence.md>), [community](<https://devfeed.tech/tags/community.md>), [datasets](<https://devfeed.tech/tags/datasets.md>), [grouped-query-attention](<https://devfeed.tech/tags/grouped-query-attention.md>), [inference](<https://devfeed.tech/tags/inference.md>), [llama](<https://devfeed.tech/tags/llama.md>), [llm](<https://devfeed.tech/tags/llm.md>), [long-context](<https://devfeed.tech/tags/long-context.md>), [models](<https://devfeed.tech/tags/models.md>), [nlp](<https://devfeed.tech/tags/nlp.md>), [open-source](<https://devfeed.tech/tags/open-source.md>), [performance](<https://devfeed.tech/tags/performance.md>), [reasoning](<https://devfeed.tech/tags/reasoning.md>), [research](<https://devfeed.tech/tags/research.md>), [training](<https://devfeed.tech/tags/training.md>)

## AI overview

SmolLM3 is a 3B multilingual, long-context reasoning model. The article shares its training recipe and describes architecture changes including grouped-query attention, NoPE, and intra-document masking.

## Source excerpt

We're on a journey to advance and democratize artificial intelligence through open source and open science.