# Think Twice: The Metis-1.6 Technical Report

DevFeed: [Think Twice: The Metis-1.6 Technical Report](<https://devfeed.tech/articles/think-twice-the-metis-1-6-technical-report-107984.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2610.37903>)

Author: Giulianno Vollmer

Published: 2026-10-30T04:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [llm-reasoning](<https://devfeed.tech/topics/llm-reasoning.md>), [post-training](<https://devfeed.tech/topics/post-training.md>), [Fine-tuning](<https://devfeed.tech/topics/fine-tuning.md>), [long-context](<https://devfeed.tech/topics/long-context.md>), [qwen](<https://devfeed.tech/topics/qwen.md>)

Tags: [2](<https://devfeed.tech/tags/2.md>), [architecture](<https://devfeed.tech/tags/architecture.md>), [attention](<https://devfeed.tech/tags/attention.md>), [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [code](<https://devfeed.tech/tags/code.md>), [context](<https://devfeed.tech/tags/context.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cl](<https://devfeed.tech/tags/cs-cl.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [data](<https://devfeed.tech/tags/data.md>), [fine-tuning](<https://devfeed.tech/tags/fine-tuning.md>), [frontier](<https://devfeed.tech/tags/frontier.md>), [frontier-models](<https://devfeed.tech/tags/frontier-models.md>), [mathematics](<https://devfeed.tech/tags/mathematics.md>), [median](<https://devfeed.tech/tags/median.md>), [memory](<https://devfeed.tech/tags/memory.md>), [mixture-of-experts](<https://devfeed.tech/tags/mixture-of-experts.md>), [model](<https://devfeed.tech/tags/model.md>), [models](<https://devfeed.tech/tags/models.md>), [pretraining](<https://devfeed.tech/tags/pretraining.md>), [qwen3](<https://devfeed.tech/tags/qwen3.md>), [reward](<https://devfeed.tech/tags/reward.md>), [technical](<https://devfeed.tech/tags/technical.md>)

## AI overview

Metis-1.6 presents two language models trained from scratch with a hybrid Mamba-2 and attention mixture-of-experts stack that processes each token twice. Logos scores 52.93 across 40 post-trained benchmarks, ranking seventh of 17 and near Qwen3-4B; it uses roughly one-twelfth the median training data of comparators that disclose it and fewer active parameters per token than models that outperform it. It ranks second of 30 in mathematics, including 67.50% on AIME 2025, but is weak at code, instruction following, and long context. The report also describes failures: the configured optimizer never ran, a 1.77B-parameter n-gram memory switched itself off, five fine-tuning choices reduced capability, and a length reward began penalizing long correct solutions above a group pass rate of two-thirds. The abstract says this reward sign inversion was fixed with a one-line clamp.

## Source excerpt

arXiv:2610.37903v1 Announce Type: cross Abstract: Metis is a bet that better data, better training methods and new architectures can buy what a bigger parameter count cannot, at a fraction of the cost. Metis-1.6 is its latest step: two language models built from scratch by a single researcher, Praxis (2.94B parameters, 0.46B active per pass) and Logos (10.23B parameters, 1.18B active per pass). Both run a hybrid Mamba-2 and attention mixture-of-experts stack twice for every token, so each token gets a second look. Each was trained on 1.02T tokens in total: 804.76B of pretraining, 18B of context extension to 131,072 tokens, and 200B of supervised fine-tuning. Metis-1.6 was never meant to challenge frontier models, and across 40 post-trained benchmarks it mostly does not: Logos averages 52.93, seventh of 17 models in its comparison set and within a point of Qwen3-4B. But it gets there on roughly a twelfth of the median training data of the comparators that disclose it, and with fewer active parameters per token than any model that beats it, which puts it on the Pareto frontier for both. In mathematics it ranks second of 30 models, with 67.50% on AIME 2025 and 63.64% on HMMT February 2026. It is weak at code, instruction following and long context. The rest of the report is the part I found most valuable: what went wrong. The optimizer I configured never ran, and attention trained with AdamW for all of pretraining. A 1.77B-parameter n-gram memory learned to switch itself off. Five fine-tuning choices cost capability. And a length reward under group-mean normalization punished correct solutions for being long once a group's pass rate passed 2/3, a sign inversion I prove and fix with a one-line clamp. I close with the 10B-token ablations behind MoRE (Mixture of Recursive Experts), the looped architecture of Metis-1.7.