# gradient\_accumulation

A machine-learning training technique that accumulates gradients over several batches before updating model parameters.

This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles.

## PipelineRL

DevFeed: [PipelineRL](<https://devfeed.tech/articles/pipelinerl-7056.md>)

Original publisher: [Read original article](<https://huggingface.co/blog/ServiceNow/pipelinerl>)

Author: Alex Piche; Rafael Pardinas; Ehsan Kamalloo; Bahdanau

Published: 2025-04-25T22:37:16Z

Content type: article

Language: en

Sources: [Hugging Face - Blog](<https://devfeed.tech/sources/hugging-face-blog.md>)

Topics: [grpo](<https://devfeed.tech/topics/grpo.md>), [model-serving](<https://devfeed.tech/topics/model-serving.md>), [gradient\_accumulation](<https://devfeed.tech/topics/gradient-accumulation.md>), [GPU](<https://devfeed.tech/topics/gpu.md>), [datasets](<https://devfeed.tech/topics/datasets.md>), [math](<https://devfeed.tech/topics/math.md>), [Optimization](<https://devfeed.tech/topics/optimization.md>)

Tags: [batch](<https://devfeed.tech/tags/batch.md>), [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [blog-post](<https://devfeed.tech/tags/blog-post.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [grpo](<https://devfeed.tech/tags/grpo.md>), [optimization](<https://devfeed.tech/tags/optimization.md>), [performance](<https://devfeed.tech/tags/performance.md>), [rl](<https://devfeed.tech/tags/rl.md>), [training](<https://devfeed.tech/tags/training.md>)

### AI overview

This blog post presents PipelineRL, a modular reinforcement-learning architecture for training language models. It uses inflight weight updates to preserve efficient inference batching while keeping collected data on-policy or near on-policy. The article reports competitive results against Open-Reasoner-Zero on AIME 2024 and MATH 500 using a simpler GRPO-based implementation.

### Source excerpt

In this blog post, we show that 1) inflight weight updates do not harm the training process and 2) PipelineRL achieves competitive results compared to Open-Reasoner-Zero, while using a simpler RL algorithm. We also present the modular PipelineRL architecture that facilitates trying new inference / trainer combinations. In conventional RL approaches (Figure 1a), there is a trade-off between high throughput inference and on-policy data collection.

## Fixing Gradient Accumulation

DevFeed: [Fixing Gradient Accumulation](<https://devfeed.tech/articles/fixing-gradient-accumulation-7222.md>)

Original publisher: [Read original article](<https://huggingface.co/blog/gradient_accumulation>)

Author: Lysandre; Arthur Zucker; Zachary Mueller; Yih-Dar SHIEH; Benjamin Bossan; Pedro Cuenca

Published: 2024-10-16T00:00:00Z

Content type: article

Language: en

Sources: [Hugging Face - Blog](<https://devfeed.tech/sources/hugging-face-blog.md>)

Topics: [gradient\_accumulation](<https://devfeed.tech/topics/gradient-accumulation.md>), [Transformers](<https://devfeed.tech/topics/transformers.md>)

Tags: [bug](<https://devfeed.tech/tags/bug.md>), [gradient-accumulation](<https://devfeed.tech/tags/gradient-accumulation.md>), [training](<https://devfeed.tech/tags/training.md>), [transformers](<https://devfeed.tech/tags/transformers.md>)

### AI overview

The article explains a bug in gradient accumulation for token-level tasks, where averaging per-batch losses produced results that did not match full-batch training. It describes a fix that accounts for the total number of non-padding tokens and introduces an API for supplying custom loss functions.

### Source excerpt

We're on a journey to advance and democratize artificial intelligence through open source and open science.

## Does batch size matter?

DevFeed: [Does batch size matter?](<https://devfeed.tech/articles/does-batch-size-matter-20165.md>)

Original publisher: [Read original article](<https://blog.janestreet.com/does-batch-size-matter/>)

Author: Chris Hardin

Published: 2017-10-31T00:00:00Z

Content type: opinion

Language: en

Sources: [Jane Street](<https://devfeed.tech/sources/jane-street.md>)

Topics: [gradient\_accumulation](<https://devfeed.tech/topics/gradient-accumulation.md>), [Deep learning](<https://devfeed.tech/topics/deep-learning.md>)

Tags: [batch](<https://devfeed.tech/tags/batch.md>), [deep-learning](<https://devfeed.tech/tags/deep-learning.md>), [learning](<https://devfeed.tech/tags/learning.md>), [machine-learning](<https://devfeed.tech/tags/machine-learning.md>), [models](<https://devfeed.tech/tags/models.md>), [training](<https://devfeed.tech/tags/training.md>)

### AI overview

The post argues that batch size mostly does not matter in stochastic gradient descent when it is not too large. Apparent batch-size effects can arise from expressing updates with averaged gradients or from fixing the learning rate while changing batch size, which also changes the effective temperature parameter.

### Source excerpt

This post is aimed at readers who are already familiar with stochastic gradient descent (SGD) and terms like "batch size". For an introduction to these ideas, I recommend Goodfellow et al.'s Deep Learning, in particular the introduction and, for more about SGD, Chapter 8. The relevance of SGD is that it has made it feasible to work with much more complex models than was formerly possible.