# gradient\_accumulation

Published articles for gradient\_accumulation.

This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles.

## Finetuning olmOCR to be a faithful OCR-Engine

DevFeed: [Finetuning olmOCR to be a faithful OCR-Engine](<https://devfeed.tech/articles/finetuning-olmocr-to-be-a-faithful-ocr-engine-7515.md>)

Original publisher: [Read original article](<https://huggingface.co/blog/tngtech/finetuning-olmocr-to-be-a-faithful-ocr-engine>)

Author: Johannes EsslingerTNG; Innovation Hacking

Published: 2025-04-22T18:33:09Z

Content type: tutorial

Language: en

Sources: [Hugging Face - Blog](<https://devfeed.tech/sources/hugging-face-blog.md>)

Topics: [Training AI Models](<https://devfeed.tech/topics/training-ai-models.md>), [Language models](<https://devfeed.tech/topics/language-models.md>)

Tags: [datasets](<https://devfeed.tech/tags/datasets.md>), [evaluation](<https://devfeed.tech/tags/evaluation.md>), [gradient-accumulation](<https://devfeed.tech/tags/gradient-accumulation.md>), [hugging-face](<https://devfeed.tech/tags/hugging-face.md>), [inference](<https://devfeed.tech/tags/inference.md>), [nvidia](<https://devfeed.tech/tags/nvidia.md>), [ocr](<https://devfeed.tech/tags/ocr.md>), [training](<https://devfeed.tech/tags/training.md>), [vision](<https://devfeed.tech/tags/vision.md>)

### AI overview

The article describes fine-tuning olmOCR to retain header and footer information that its original training data intentionally omitted. The authors generate an 8,000-document dataset with Qwen2.5-VL-72B-Instruct, train using the open-source olmOCR pipeline, and evaluate on a customized dataset containing header and footer content.

### Source excerpt

A Blog post by TNG Technology Consulting GmbH on Hugging Face

## Fixing Gradient Accumulation

DevFeed: [Fixing Gradient Accumulation](<https://devfeed.tech/articles/fixing-gradient-accumulation-7222.md>)

Original publisher: [Read original article](<https://huggingface.co/blog/gradient_accumulation>)

Author: Lysandre; Arthur Zucker; Zachary Mueller; Yih-Dar SHIEH; Benjamin Bossan; Pedro Cuenca

Published: 2024-10-16T00:00:00Z

Content type: article

Language: en

Sources: [Hugging Face - Blog](<https://devfeed.tech/sources/hugging-face-blog.md>)

Topics: [gradient\_accumulation](<https://devfeed.tech/topics/gradient-accumulation.md>), [Transformers](<https://devfeed.tech/topics/transformers.md>)

Tags: [bug](<https://devfeed.tech/tags/bug.md>), [gradient-accumulation](<https://devfeed.tech/tags/gradient-accumulation.md>), [training](<https://devfeed.tech/tags/training.md>), [transformers](<https://devfeed.tech/tags/transformers.md>)

### AI overview

The article explains a bug in gradient accumulation for token-level tasks, where averaging per-batch losses produced results that did not match full-batch training. It describes a fix that accounts for the total number of non-padding tokens and introduces an API for supplying custom loss functions.

### Source excerpt

We're on a journey to advance and democratize artificial intelligence through open source and open science.