# Which Routing Does Each MoE Model Use? DeepSeek, Qwen, Mixtral, Llama 4, gpt-oss Compared

DevFeed: [Which Routing Does Each MoE Model Use? DeepSeek, Qwen, Mixtral, Llama 4, gpt-oss Compared](<https://devfeed.tech/articles/which-routing-does-each-moe-model-use-deepseek-qwen-mixtral-llama-4-gpt-oss-compared-122492.md>)

Original publisher: [Read original article](<https://www.marktechpost.com/2026/11/08/moe-model-routing-comparison/>)

Author: Michal Sutter

Published: 2026-11-08T10:27:27Z

Content type: comparison

Language: en

Sources: [MarkTechPost](<https://devfeed.tech/sources/marktechpost.md>)

Topics: [Mixture of Experts (MoE)](<https://devfeed.tech/topics/mixture-of-experts-moe.md>), [model architecture](<https://devfeed.tech/topics/model-architecture.md>), [Open Source Models & Datasets](<https://devfeed.tech/topics/open-source-models-datasets.md>), [llama](<https://devfeed.tech/topics/llama.md>), [qwen](<https://devfeed.tech/topics/qwen.md>), [mixtral](<https://devfeed.tech/topics/mixtral.md>)

Tags: [ai-infrastructure](<https://devfeed.tech/tags/ai-infrastructure.md>), [applications](<https://devfeed.tech/tags/applications.md>), [artificial-intelligence](<https://devfeed.tech/tags/artificial-intelligence.md>), [balancing](<https://devfeed.tech/tags/balancing.md>), [comparison](<https://devfeed.tech/tags/comparison.md>), [config](<https://devfeed.tech/tags/config.md>), [counts](<https://devfeed.tech/tags/counts.md>), [deepseek](<https://devfeed.tech/tags/deepseek.md>), [deployment](<https://devfeed.tech/tags/deployment.md>), [editors-pick](<https://devfeed.tech/tags/editors-pick.md>), [embedding-model](<https://devfeed.tech/tags/embedding-model.md>), [experts](<https://devfeed.tech/tags/experts.md>), [gpt-oss](<https://devfeed.tech/tags/gpt-oss.md>), [language-model](<https://devfeed.tech/tags/language-model.md>), [large-language-model](<https://devfeed.tech/tags/large-language-model.md>), [llama](<https://devfeed.tech/tags/llama.md>), [load-balancing](<https://devfeed.tech/tags/load-balancing.md>), [machine-learning](<https://devfeed.tech/tags/machine-learning.md>), [mixtral](<https://devfeed.tech/tags/mixtral.md>), [model](<https://devfeed.tech/tags/model.md>), [models](<https://devfeed.tech/tags/models.md>), [moe](<https://devfeed.tech/tags/moe.md>), [open](<https://devfeed.tech/tags/open.md>), [qwen](<https://devfeed.tech/tags/qwen.md>), [route](<https://devfeed.tech/tags/route.md>), [routing](<https://devfeed.tech/tags/routing.md>), [staff](<https://devfeed.tech/tags/staff.md>), [technology](<https://devfeed.tech/tags/technology.md>), [tokens](<https://devfeed.tech/tags/tokens.md>), [top-k](<https://devfeed.tech/tags/top-k.md>)

## AI overview

The excerpt compares routing configurations across 10 open MoE models, showing that all use token-choice routing but vary widely in expert count, top-k, shared experts, scoring and balancing. DeepSeek-V3 selects 8 of 256 routed experts using sigmoid scores, grouped selection and bias-based balancing, with one shared expert; DeepSeek-V4-Pro uses 6 of 384, switches to sqrtsoftplus and uses hash routing in its first three MoE layers. Qwen3 routes to 8 of 128 experts without shared experts, while Qwen3.6 uses 256 experts with 8 routed and one shared. Llama 4 Maverick combines dense and MoE layers, selecting one of 128 routed experts plus a shared expert; gpt-oss-120b selects 4 of 128 and its MXFP4 weights are said to fit on one 80 GB GPU. Kimi K2 and GLM-4.5 use 384 and 160 experts respectively. The excerpt says more experts increase resident weight memory and all-to-all traffic, while lower top-k reduces per-token compute and communication. It also notes that some configurations omit gating or load-balancing details, limiting direct auditability.

## Source excerpt

A config-verified comparison of how 10 open MoE models route tokens: total and active parameters, expert counts, top-k, shared experts, gating functions and load-balancing methods. The post Which Routing Does Each MoE Model Use? DeepSeek, Qwen, Mixtral, Llama 4, gpt-oss Compared appeared first on MarkTechPost.