# L2 Regularization and Batch Norm

DevFeed: [L2 Regularization and Batch Norm](<https://devfeed.tech/articles/l2-regularization-and-batch-norm-20188.md>)

Original publisher: [Read original article](<https://blog.janestreet.com/l2-regularization-and-batch-norm/>)

Author: David Wu

Published: 2019-01-29T00:00:00Z

Content type: article

Language: en

Sources: [Jane Street](<https://devfeed.tech/sources/jane-street.md>)

Topics: [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [machine learning overfitting](<https://devfeed.tech/topics/machine-learning-overfitting.md>)

Tags: [adaptive](<https://devfeed.tech/tags/adaptive.md>), [analysis](<https://devfeed.tech/tags/analysis.md>), [blog-post](<https://devfeed.tech/tags/blog-post.md>), [machine-learning](<https://devfeed.tech/tags/machine-learning.md>), [model](<https://devfeed.tech/tags/model.md>), [training](<https://devfeed.tech/tags/training.md>)

## AI overview

This blog post explains how L2 regularization, or weight decay, interacts with batch normalization in convolutional neural networks. Under typical architectures and stochastic gradient descent, the L2 penalty can lose its usual regularizing effect and instead behave essentially like an adaptive adjustment to the learning rate. The analysis may not apply to ADAM or other alternative optimizers.

## Source excerpt

This blog post is about an interesting detail about machine learning that I came across as a researcher at Jane Street - that of the interaction between L2 regularization, also known as weight decay, and batch normalization.