# Squeezing ~2x MFU out of our training runs

DevFeed: [Squeezing ~2x MFU out of our training runs](<https://devfeed.tech/articles/squeezing-2-mfu-out-of-our-training-runs-88031.md>)

Original publisher: [Read original article](<https://building.nu.com/squeezing-2x-mfu-out-of-our-training-runs/>)

Author: Nubank Editorial

Published: 2026-10-15T17:53:22Z

Content type: article

Language: en

Sources: [Building Nubank](<https://devfeed.tech/sources/building-nubank.md>), [Nubank](<https://devfeed.tech/sources/nubank.md>)

Topics: [AI Infrastructure](<https://devfeed.tech/topics/ai-infrastructure.md>), [LLM optimization](<https://devfeed.tech/topics/llm-optimization.md>), [Transformers](<https://devfeed.tech/topics/transformers.md>)

Tags: [ai-research](<https://devfeed.tech/tags/ai-research.md>), [batching](<https://devfeed.tech/tags/batching.md>), [benchmarking](<https://devfeed.tech/tags/benchmarking.md>), [compilation](<https://devfeed.tech/tags/compilation.md>), [cudnn](<https://devfeed.tech/tags/cudnn.md>), [efficiency](<https://devfeed.tech/tags/efficiency.md>), [faster](<https://devfeed.tech/tags/faster.md>), [model-training](<https://devfeed.tech/tags/model-training.md>), [speed](<https://devfeed.tech/tags/speed.md>), [transformer](<https://devfeed.tech/tags/transformer.md>)

## AI overview

Nubank describes improving training efficiency for a hybrid Transformer model on B200 GPUs through attention-kernel selection, reusable torch.compile output, compiler-friendly Gated DeltaNet integration, and batched Muon updates. In fixed-workload comparisons, model FLOPs utilization rose from 21.1% to 39.3% with AdamW and from 20.2% to 39.2% with Muon, without changing the model architecture or training batch size.

## Source excerpt

How attention-kernel selection, compiler integration, and optimizer implementation shape Transformer model training speed The post Squeezing ~2x MFU out of our training runs appeared first on Building Nubank.