# Preference Optimization for Vision Language Models

DevFeed: [Preference Optimization for Vision Language Models](<https://devfeed.tech/articles/preference-optimization-for-vision-language-models-7175.md>)

Original publisher: [Read original article](<https://huggingface.co/blog/dpo_vlm>)

Author: Quentin Gallouédec; Shengyi Costa Huang; merve; Kashif Rasul

Published: 2024-07-10T00:00:00Z

Content type: tutorial

Language: en

Sources: [Hugging Face - Blog](<https://devfeed.tech/sources/hugging-face-blog.md>)

Topics: [vlm](<https://devfeed.tech/topics/vlm.md>), [Language models](<https://devfeed.tech/topics/language-models.md>)

Tags: [data](<https://devfeed.tech/tags/data.md>), [dpo](<https://devfeed.tech/tags/dpo.md>), [fine-tuning](<https://devfeed.tech/tags/fine-tuning.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [images](<https://devfeed.tech/tags/images.md>), [language-models](<https://devfeed.tech/tags/language-models.md>), [multimodal](<https://devfeed.tech/tags/multimodal.md>), [optimization](<https://devfeed.tech/tags/optimization.md>), [rlhf](<https://devfeed.tech/tags/rlhf.md>), [training](<https://devfeed.tech/tags/training.md>), [trl](<https://devfeed.tech/tags/trl.md>), [vlm](<https://devfeed.tech/tags/vlm.md>), [vlms](<https://devfeed.tech/tags/vlms.md>)

## AI overview

A tutorial on training vision-language models with TRL's direct preference optimization support, covering preference data formatting and memory considerations.

## Source excerpt

We're on a journey to advance and democratize artificial intelligence through open source and open science.