# Accelerate ND-Parallel: A guide to Efficient Multi-GPU Training

DevFeed: [Accelerate ND-Parallel: A guide to Efficient Multi-GPU Training](<https://devfeed.tech/articles/accelerate-nd-parallel-a-guide-to-efficient-multi-gpu-training-7062.md>)

Original publisher: [Read original article](<https://huggingface.co/blog/accelerate-nd-parallel>)

Author: Salman Mohammadi; Matej Sirovatka; wing lian; Marc Sun; Dan Saunders

Published: 2025-08-08T00:00:00Z

Content type: tutorial

Language: en

Sources: [Hugging Face - Blog](<https://devfeed.tech/sources/hugging-face-blog.md>)

Topics: [GPU](<https://devfeed.tech/topics/gpu.md>), [AI, ML & Data Engineering](<https://devfeed.tech/topics/ai-ml-data-engineering.md>)

Tags: [accelerate](<https://devfeed.tech/tags/accelerate.md>), [fine-tuning](<https://devfeed.tech/tags/fine-tuning.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [guide](<https://devfeed.tech/tags/guide.md>), [how-to](<https://devfeed.tech/tags/how-to.md>), [llm](<https://devfeed.tech/tags/llm.md>), [multi-gpu](<https://devfeed.tech/tags/multi-gpu.md>), [training](<https://devfeed.tech/tags/training.md>), [transformers](<https://devfeed.tech/tags/transformers.md>)

## AI overview

A guide to composing data-parallel and other parallelism strategies for efficient multi-GPU model training with Accelerate and Axolotl.

## Source excerpt

We're on a journey to advance and democratize artificial intelligence through open source and open science.