# Direct Preference Optimization Beyond Chatbots

DevFeed: [Direct Preference Optimization Beyond Chatbots](<https://devfeed.tech/articles/direct-preference-optimization-beyond-chatbots-6992.md>)

Original publisher: [Read original article](<https://huggingface.co/blog/Dharma-AI/direct-preference-optimization-beyond-chatbots>)

Author: Erick Lachmann; Gabriel Pimenta de Freitas Cardoso; Francisco de Almeida Rocha Alves

Published: 2026-06-03T12:55:11Z

Content type: article

Language: en

Sources: [Hugging Face - Blog](<https://devfeed.tech/sources/hugging-face-blog.md>)

Topics: [dpo](<https://devfeed.tech/topics/dpo.md>), [vlm](<https://devfeed.tech/topics/vlm.md>), [Benchmark](<https://devfeed.tech/topics/benchmark.md>), [Fine-tuning](<https://devfeed.tech/topics/fine-tuning.md>), [hugging face](<https://devfeed.tech/topics/hugging-face.md>), [Open Source](<https://devfeed.tech/topics/open-source.md>)

Tags: [benchmark](<https://devfeed.tech/tags/benchmark.md>), [cost](<https://devfeed.tech/tags/cost.md>), [dpo](<https://devfeed.tech/tags/dpo.md>), [efficiency](<https://devfeed.tech/tags/efficiency.md>), [fine-tuning](<https://devfeed.tech/tags/fine-tuning.md>), [hugging-face](<https://devfeed.tech/tags/hugging-face.md>), [model](<https://devfeed.tech/tags/model.md>), [ocr](<https://devfeed.tech/tags/ocr.md>), [open-source](<https://devfeed.tech/tags/open-source.md>), [vision](<https://devfeed.tech/tags/vision.md>)

## AI overview

The article presents Direct Preference Optimization (DPO) as a second training stage for reducing text degeneration in DharmaOCR, a structured OCR model. Applied after supervised fine-tuning, DPO reduced degeneration across every tested model family, with an average reduction of 59.4% and a best-case reduction of 87.6%.

## Source excerpt

In April, we released DharmaOCR, our specialized structured OCR model (available on Hugging Face) along with a paper detailing the methodology behind it and a benchmark demonstrating its superior quality and cost efficiency. The paper benchmarked leading vision-language model families - both open-source and commercial - on a structured document extraction task: OCR on Brazilian Portuguese text.