# Does batch size matter?

DevFeed: [Does batch size matter?](<https://devfeed.tech/articles/does-batch-size-matter-20165.md>)

Original publisher: [Read original article](<https://blog.janestreet.com/does-batch-size-matter/>)

Author: Chris Hardin

Published: 2017-10-31T00:00:00Z

Content type: opinion

Language: en

Sources: [Jane Street](<https://devfeed.tech/sources/jane-street.md>)

Topics: [gradient\_accumulation](<https://devfeed.tech/topics/gradient-accumulation.md>), [Deep learning](<https://devfeed.tech/topics/deep-learning.md>)

Tags: [batch](<https://devfeed.tech/tags/batch.md>), [deep-learning](<https://devfeed.tech/tags/deep-learning.md>), [learning](<https://devfeed.tech/tags/learning.md>), [machine-learning](<https://devfeed.tech/tags/machine-learning.md>), [models](<https://devfeed.tech/tags/models.md>), [training](<https://devfeed.tech/tags/training.md>)

## AI overview

The post argues that batch size mostly does not matter in stochastic gradient descent when it is not too large. Apparent batch-size effects can arise from expressing updates with averaged gradients or from fixing the learning rate while changing batch size, which also changes the effective temperature parameter.

## Source excerpt

This post is aimed at readers who are already familiar with stochastic gradient descent (SGD) and terms like "batch size". For an introduction to these ideas, I recommend Goodfellow et al.'s Deep Learning, in particular the introduction and, for more about SGD, Chapter 8. The relevance of SGD is that it has made it feasible to work with much more complex models than was formerly possible.