# Capability Gain from ISA to Shared Execution on AMD MI300A and MI250X

DevFeed: [Capability Gain from ISA to Shared Execution on AMD MI300A and MI250X](<https://devfeed.tech/articles/capability-gain-from-isa-to-shared-execution-on-amd-mi300a-and-mi250x-121265.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.07198>)

Author: Aaron Jarmusch, Sunita Chandrasekaran

Published: 2026-11-06T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Distributed Computing](<https://devfeed.tech/sources/arxiv-distributed-computing.md>)

Topics: [GPU optimization](<https://devfeed.tech/topics/gpu-optimization.md>), [rocm](<https://devfeed.tech/topics/rocm.md>), [LLM optimization](<https://devfeed.tech/topics/llm-optimization.md>), [quantization](<https://devfeed.tech/topics/quantization.md>)

Tags: [amd](<https://devfeed.tech/tags/amd.md>), [cs-dc](<https://devfeed.tech/tags/cs-dc.md>), [execution](<https://devfeed.tech/tags/execution.md>), [fp8](<https://devfeed.tech/tags/fp8.md>), [inference](<https://devfeed.tech/tags/inference.md>), [isa](<https://devfeed.tech/tags/isa.md>), [llm](<https://devfeed.tech/tags/llm.md>), [llm-serving](<https://devfeed.tech/tags/llm-serving.md>), [metric](<https://devfeed.tech/tags/metric.md>), [performance](<https://devfeed.tech/tags/performance.md>), [precision](<https://devfeed.tech/tags/precision.md>), [queue](<https://devfeed.tech/tags/queue.md>), [queues](<https://devfeed.tech/tags/queues.md>), [sparsity](<https://devfeed.tech/tags/sparsity.md>), [streams](<https://devfeed.tech/tags/streams.md>), [theoretical](<https://devfeed.tech/tags/theoretical.md>)

## AI overview

The paper defines capability gain as the fraction of an instruction-level speedup retained at higher execution layers, and compares AMD MI300A with MI250X from ISA microbenchmarks through libraries, applications and shared-device workloads. MI300A's FP8 retains most of its speedup in hipBLASLt: an 8192³ GEMM is reported faster than MI250X FP16, though the abstract's speedup value is missing. Unfused quantization makes FP8 slower per layer in LLM inference, and shared LLM serving gains nothing from FP8. 2:4 sparsity retains at most about half its speedup in hipSPARSELt. With queue sharing, FP8 GEMM capability gain exceeds 1; stream fairness depends on hardware queue count rather than precision.

## Source excerpt

arXiv:2611.07198v1 Announce Type: new Abstract: Each new GPU generation is showcased by its peak instruction rates, yet users hardly ever run instructions directly: they reach the hardware's performance through vendor libraries, applications and devices shared with other jobs. How much of a theoretical speedup survives these layers is rarely measured, so a team deciding whether to move to a new generation, or to adopt FP8 or 2:4 sparsity, lacks adequate guidance. AMD's MI300A (CDNA3) adds new FP8 matrix instructions and 2:4 structured-sparse instructions that the predecessor, the MI250X (CDNA2), does not have. We analyze this theoretical speedup gap on AMD's MI300A (CDNA3) against the MI250X (CDNA2) by defining a new metric, capability gain. Capability gain is the fraction of a feature's instruction-level speedup that it keeps at a higher level, such as a library, an application or a shared device. We trace it from ISA microbenchmarks through hipBLASLt and hipSPARSELt to LLM inference and mixed-precision HPL, and onto a device shared by up to eight concurrent streams. FP8 keeps most of its speedup through the library, completing an $8192^3$ GEMM \genCapPkgEndToEndGainEightK{} faster than the MI250X with FP16, but unfused quantization turns it into a per-layer slowdown in an LLM, and 2:4 sparsity keeps at most about half of its speedup in hipSPARSELt. Under queue sharing, FP8's GEMM capability gain rises above 1. How fairly streams share the device depends on the number of hardware queues, not on the numeric precision, yet shared LLM serving gains nothing from FP8.