# Creating custom kernels for the AMD MI300

DevFeed: [Creating custom kernels for the AMD MI300](<https://devfeed.tech/articles/creating-custom-kernels-for-the-amd-mi300-7347.md>)

Original publisher: [Read original article](<https://huggingface.co/blog/mi300kernels>)

Author: Rémi Ouazan Reboul; seungrok jung

Published: 2025-07-09T00:00:00Z

Content type: article

Language: en

Sources: [Hugging Face - Blog](<https://devfeed.tech/sources/hugging-face-blog.md>)

Topics: [kernels](<https://devfeed.tech/topics/kernels.md>), [GPU](<https://devfeed.tech/topics/gpu.md>), [vllm](<https://devfeed.tech/topics/vllm.md>), [hugging face](<https://devfeed.tech/topics/hugging-face.md>), [model-deployment](<https://devfeed.tech/topics/model-deployment.md>), [Optimization](<https://devfeed.tech/topics/optimization.md>), [llama](<https://devfeed.tech/topics/llama.md>), [Open Source](<https://devfeed.tech/topics/open-source.md>)

Tags: [amd](<https://devfeed.tech/tags/amd.md>), [cuda](<https://devfeed.tech/tags/cuda.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [hugging-face](<https://devfeed.tech/tags/hugging-face.md>), [inference](<https://devfeed.tech/tags/inference.md>), [kernels](<https://devfeed.tech/tags/kernels.md>), [latency](<https://devfeed.tech/tags/latency.md>), [llama](<https://devfeed.tech/tags/llama.md>), [open-source](<https://devfeed.tech/tags/open-source.md>), [optimization](<https://devfeed.tech/tags/optimization.md>), [vllm](<https://devfeed.tech/tags/vllm.md>)

## AI overview

Hugging Face and AMD describe open-source, device-specific kernel optimizations for serving Llama 3.1 405B in FP8 with vLLM on eight AMD MI300X GPUs. The article explains kernel-level optimization and reports latency improvements from fused residual, normalization, FP8 conversion, SwiGLU, and Skinny GEMM kernels.

## Source excerpt

We're on a journey to advance and democratize artificial intelligence through open source and open science.