# trl

TRL is a full-stack library for training transformer language models with reinforcement-learning and related methods.

This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles.

## TRL v1.0: Post-Training Library Built to Move with the Field

DevFeed: [TRL v1.0: Post-Training Library Built to Move with the Field](<https://devfeed.tech/articles/trl-v1-0-post-training-library-built-to-move-with-the-field-7541.md>)

Original publisher: [Read original article](<https://huggingface.co/blog/trl-v1>)

Author: Quentin Gallouédec; Steven Liu; Pedro Cuenca; Sergio Paniego

Published: 2026-03-31T00:00:00Z

Content type: article

Language: en

Sources: [Hugging Face - Blog](<https://devfeed.tech/sources/hugging-face-blog.md>)

Topics: [trl](<https://devfeed.tech/topics/trl.md>), [post-training](<https://devfeed.tech/topics/post-training.md>), [dpo](<https://devfeed.tech/topics/dpo.md>), [grpo](<https://devfeed.tech/topics/grpo.md>), [rlvr](<https://devfeed.tech/topics/rlvr.md>), [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [Code](<https://devfeed.tech/topics/code.md>)

Tags: [announcement](<https://devfeed.tech/tags/announcement.md>), [artificial-intelligence](<https://devfeed.tech/tags/artificial-intelligence.md>), [code](<https://devfeed.tech/tags/code.md>), [dpo](<https://devfeed.tech/tags/dpo.md>), [grpo](<https://devfeed.tech/tags/grpo.md>), [open-source](<https://devfeed.tech/tags/open-source.md>), [post-training](<https://devfeed.tech/tags/post-training.md>), [reinforcement-learning](<https://devfeed.tech/tags/reinforcement-learning.md>), [rlvr](<https://devfeed.tech/tags/rlvr.md>), [software](<https://devfeed.tech/tags/software.md>), [trl](<https://devfeed.tech/tags/trl.md>), [v1](<https://devfeed.tech/tags/v1.md>)

### AI overview

TRL v1.0 presents a post-training library designed to remain useful as algorithms, models, and paradigms change. The article explains the evolution from PPO to DPO-style preference optimization and RLVR methods such as GRPO, emphasizing flexible abstractions and practical comparison over a supposedly permanent design.

### Source excerpt

We're on a journey to advance and democratize artificial intelligence through open source and open science.

## DeepMath: A lightweight math reasoning Agent with smolagents

DevFeed: [DeepMath: A lightweight math reasoning Agent with smolagents](<https://devfeed.tech/articles/deepmath-a-lightweight-math-reasoning-agent-with-smolagents-7288.md>)

Original publisher: [Read original article](<https://huggingface.co/blog/intel-deepmath>)

Author: Daniel Fleischer; Moshe Berchansky; Moshe Wasserblat

Published: 2025-12-04T00:00:00Z

Content type: article

Language: en

Sources: [Hugging Face - Blog](<https://devfeed.tech/sources/hugging-face-blog.md>)

Topics: [smolagents](<https://devfeed.tech/topics/smolagents.md>), [math](<https://devfeed.tech/topics/math.md>), [grpo](<https://devfeed.tech/topics/grpo.md>), [Fine-tuning](<https://devfeed.tech/topics/fine-tuning.md>), [Python](<https://devfeed.tech/topics/python.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>), [vllm](<https://devfeed.tech/topics/vllm.md>), [trl](<https://devfeed.tech/topics/trl.md>), [datasets](<https://devfeed.tech/topics/datasets.md>), [Inference](<https://devfeed.tech/topics/inference.md>), [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [Open Source](<https://devfeed.tech/topics/open-source.md>)

Tags: [agents](<https://devfeed.tech/tags/agents.md>), [artificial-intelligence](<https://devfeed.tech/tags/artificial-intelligence.md>), [datasets](<https://devfeed.tech/tags/datasets.md>), [fine-tuning](<https://devfeed.tech/tags/fine-tuning.md>), [grpo](<https://devfeed.tech/tags/grpo.md>), [inference](<https://devfeed.tech/tags/inference.md>), [large-language-models-llms](<https://devfeed.tech/tags/large-language-models-llms.md>), [llm](<https://devfeed.tech/tags/llm.md>), [math](<https://devfeed.tech/tags/math.md>), [math-reasoning](<https://devfeed.tech/tags/math-reasoning.md>), [open-source](<https://devfeed.tech/tags/open-source.md>), [python](<https://devfeed.tech/tags/python.md>), [reasoning](<https://devfeed.tech/tags/reasoning.md>), [sandbox](<https://devfeed.tech/tags/sandbox.md>), [smolagents](<https://devfeed.tech/tags/smolagents.md>), [trl](<https://devfeed.tech/tags/trl.md>), [vllm](<https://devfeed.tech/tags/vllm.md>)

### AI overview

DeepMath is a lightweight math reasoning agent built on Qwen3-4B Thinking and implemented with smolagents. It uses GRPO fine-tuning to encourage concise Python snippets for deterministic computation, executes those snippets in a constrained sandbox, and reintegrates the results into the reasoning process. Evaluation on MATH500, AIME, HMMT, and HLE reports shorter outputs and often improved accuracy.

### Source excerpt

We're on a journey to advance and democratize artificial intelligence through open source and open science.

## 20x Faster TRL Fine-tuning with RapidFire AI

DevFeed: [20x Faster TRL Fine-tuning with RapidFire AI](<https://devfeed.tech/articles/20x-faster-trl-fine-tuning-with-rapidfire-ai-7452.md>)

Original publisher: [Read original article](<https://huggingface.co/blog/rapidfireai>)

Author: Kamran Bigdely; Arun Kumar; Quentin Gallouédec

Published: 2025-11-21T00:00:00Z

Content type: article

Language: en

Sources: [Hugging Face - Blog](<https://devfeed.tech/sources/hugging-face-blog.md>)

Topics: [Fine-tuning](<https://devfeed.tech/topics/fine-tuning.md>), [trl](<https://devfeed.tech/topics/trl.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>), [post-training](<https://devfeed.tech/topics/post-training.md>), [Multi-GPU](<https://devfeed.tech/topics/multi-gpu.md>), [Orchestration](<https://devfeed.tech/topics/orchestration.md>), [dashboards](<https://devfeed.tech/topics/dashboards.md>), [GPU](<https://devfeed.tech/topics/gpu.md>), [data](<https://devfeed.tech/topics/data.md>)

Tags: [dashboards](<https://devfeed.tech/tags/dashboards.md>), [data](<https://devfeed.tech/tags/data.md>), [dpo](<https://devfeed.tech/tags/dpo.md>), [experimentation](<https://devfeed.tech/tags/experimentation.md>), [fine-tuning](<https://devfeed.tech/tags/fine-tuning.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [grpo](<https://devfeed.tech/tags/grpo.md>), [llm](<https://devfeed.tech/tags/llm.md>), [llms](<https://devfeed.tech/tags/llms.md>), [logs](<https://devfeed.tech/tags/logs.md>), [metrics](<https://devfeed.tech/tags/metrics.md>), [multi-gpu](<https://devfeed.tech/tags/multi-gpu.md>), [orchestration](<https://devfeed.tech/tags/orchestration.md>), [post-training](<https://devfeed.tech/tags/post-training.md>), [rapidfireai](<https://devfeed.tech/tags/rapidfireai.md>), [trl](<https://devfeed.tech/tags/trl.md>)

### AI overview

RapidFire AI accelerates LLM fine-tuning and post-training experimentation by running multiple TRL configurations concurrently, including on a single GPU. Its adaptive chunk-based scheduling, live metrics dashboard, multi-GPU orchestration, and interactive controls help teams compare configurations sooner, stop weak runs, and clone promising ones. The article cites internal benchmarks reporting approximately 16-24x higher experimentation throughput than sequential comparison.

### Source excerpt

We're on a journey to advance and democratize artificial intelligence through open source and open science.

## Vision Language Model Alignment in TRL ⚡

DevFeed: [Vision Language Model Alignment in TRL ⚡](<https://devfeed.tech/articles/vision-language-model-alignment-in-trl-7542.md>)

Original publisher: [Read original article](<https://huggingface.co/blog/trl-vlm-alignment>)

Author: Sergio Paniego; merve; Quentin Gallouédec; Kashif Rasul; Aritra Roy Gosthipaty

Published: 2025-08-07T00:00:00Z

Content type: article

Language: en

Sources: [Hugging Face - Blog](<https://devfeed.tech/sources/hugging-face-blog.md>)

Topics: [trl](<https://devfeed.tech/topics/trl.md>), [vlm](<https://devfeed.tech/topics/vlm.md>), [multimodal](<https://devfeed.tech/topics/multimodal.md>), [Fine-tuning](<https://devfeed.tech/topics/fine-tuning.md>)

Tags: [artificial-intelligence](<https://devfeed.tech/tags/artificial-intelligence.md>), [dpo](<https://devfeed.tech/tags/dpo.md>), [fine-tuning](<https://devfeed.tech/tags/fine-tuning.md>), [grpo](<https://devfeed.tech/tags/grpo.md>), [model](<https://devfeed.tech/tags/model.md>), [multimodal](<https://devfeed.tech/tags/multimodal.md>), [trl](<https://devfeed.tech/tags/trl.md>), [vision](<https://devfeed.tech/tags/vision.md>), [vlm](<https://devfeed.tech/tags/vlm.md>)

### AI overview

This article presents new and extended methods in TRL for aligning Vision Language Models with human preferences. It covers Mixed Preference Optimization, Group Relative Policy Optimization, Group Sequence Policy Optimization, Reinforce Leave One Out, and Online Direct Preference Optimization, along with native supervised fine-tuning support, training scripts, and demo notebooks.

### Source excerpt

We're on a journey to advance and democratize artificial intelligence through open source and open science.

## No GPU left behind: Unlocking Efficiency with Co-located vLLM in TRL

DevFeed: [No GPU left behind: Unlocking Efficiency with Co-located vLLM in TRL](<https://devfeed.tech/articles/no-gpu-left-behind-unlocking-efficiency-with-co-located-vllm-in-trl-7558.md>)

Original publisher: [Read original article](<https://huggingface.co/blog/vllm-colocate>)

Author: Mert Toslali; Yu Chin Fabian Lim; Quentin Gallouédec; Ed Snible; Raghu Ganti; Mudhakar Srivatsa

Published: 2025-06-03T00:00:00Z

Content type: article

Language: en

Sources: [Hugging Face - Blog](<https://devfeed.tech/sources/hugging-face-blog.md>)

Topics: [vllm](<https://devfeed.tech/topics/vllm.md>), [trl](<https://devfeed.tech/topics/trl.md>), [GPU](<https://devfeed.tech/topics/gpu.md>), [Inference](<https://devfeed.tech/topics/inference.md>), [Machine learning](<https://devfeed.tech/topics/machine-learning.md>)

Tags: [efficiency](<https://devfeed.tech/tags/efficiency.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [grpo](<https://devfeed.tech/tags/grpo.md>), [hardware](<https://devfeed.tech/tags/hardware.md>), [inference](<https://devfeed.tech/tags/inference.md>), [llm](<https://devfeed.tech/tags/llm.md>), [performance](<https://devfeed.tech/tags/performance.md>), [rlhf](<https://devfeed.tech/tags/rlhf.md>), [train](<https://devfeed.tech/tags/train.md>), [training](<https://devfeed.tech/tags/training.md>), [trl](<https://devfeed.tech/tags/trl.md>), [vllm](<https://devfeed.tech/tags/vllm.md>)

### AI overview

This article explains how TRL integrates vLLM to accelerate GRPO training of LLMs. It describes the GPU inefficiencies caused by running training and generation on separate devices and introduces colocated vLLM, which allows both tasks to share GPUs within the same distributed process group.

### Source excerpt

We're on a journey to advance and democratize artificial intelligence through open source and open science.

## 🐯 Liger GRPO meets TRL

DevFeed: [🐯 Liger GRPO meets TRL](<https://devfeed.tech/articles/liger-grpo-meets-trl-7332.md>)

Original publisher: [Read original article](<https://huggingface.co/blog/liger-grpo>)

Author: Shivam Sahni; Kashif Rasul; Salman Mohammadi; Shirin Yamani; Yanning Chen; Liberty

Published: 2025-05-25T00:00:00Z

Content type: tutorial

Language: en

Sources: [Hugging Face - Blog](<https://devfeed.tech/sources/hugging-face-blog.md>)

Topics: [grpo](<https://devfeed.tech/topics/grpo.md>), [trl](<https://devfeed.tech/topics/trl.md>), [Reinforcement learning](<https://devfeed.tech/topics/reinforcement-learning.md>), [Fine-tuning](<https://devfeed.tech/topics/fine-tuning.md>), [Optimization](<https://devfeed.tech/topics/optimization.md>), [GPU](<https://devfeed.tech/topics/gpu.md>), [LLMs](<https://devfeed.tech/topics/llms.md>), [rlhf](<https://devfeed.tech/topics/rlhf.md>), [coding](<https://devfeed.tech/topics/coding.md>), [math](<https://devfeed.tech/topics/math.md>)

Tags: [coding](<https://devfeed.tech/tags/coding.md>), [fine-tuning](<https://devfeed.tech/tags/fine-tuning.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [grpo](<https://devfeed.tech/tags/grpo.md>), [large-language-models-llms](<https://devfeed.tech/tags/large-language-models-llms.md>), [liger](<https://devfeed.tech/tags/liger.md>), [llm](<https://devfeed.tech/tags/llm.md>), [optimization](<https://devfeed.tech/tags/optimization.md>), [partnership](<https://devfeed.tech/tags/partnership.md>), [performance](<https://devfeed.tech/tags/performance.md>), [reinforcement-learning](<https://devfeed.tech/tags/reinforcement-learning.md>), [rlhf](<https://devfeed.tech/tags/rlhf.md>), [trl](<https://devfeed.tech/tags/trl.md>)

### AI overview

This article explains how Group Relative Policy Optimization (GRPO) can reduce the resource requirements of reinforcement learning fine-tuning for language models. It presents a TRL optimization based on chunked GRPO loss that reduces peak memory usage by 40% and discusses scaling GRPO across multiple GPUs and nodes while preserving performance and correctness.

### Source excerpt

We're on a journey to advance and democratize artificial intelligence through open source and open science.

## Open-R1: Update #1

DevFeed: [Open-R1: Update #1](<https://devfeed.tech/articles/open-r1-update-1-7420.md>)

Original publisher: [Read original article](<https://huggingface.co/blog/open-r1/update-1>)

Author: Leandro von Werra; Lewis Tunstall; Quentin Gallouédec; Guilherme Penedo; Edward Beeching; Anton Lozhkov; Brigitte Tousignant; Daniel van Strien

Published: 2025-02-02T00:04:28Z

Content type: article

Language: en

Sources: [Hugging Face - Blog](<https://devfeed.tech/sources/hugging-face-blog.md>)

Topics: [deepseek](<https://devfeed.tech/topics/deepseek.md>), [synthetic-data](<https://devfeed.tech/topics/synthetic-data.md>), [datasets](<https://devfeed.tech/topics/datasets.md>), [grpo](<https://devfeed.tech/topics/grpo.md>), [trl](<https://devfeed.tech/topics/trl.md>), [Benchmark](<https://devfeed.tech/topics/benchmark.md>), [vllm](<https://devfeed.tech/topics/vllm.md>), [GPU](<https://devfeed.tech/topics/gpu.md>)

Tags: [benchmark](<https://devfeed.tech/tags/benchmark.md>), [deepseek](<https://devfeed.tech/tags/deepseek.md>), [follow](<https://devfeed.tech/tags/follow.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [grpo](<https://devfeed.tech/tags/grpo.md>), [leaderboard](<https://devfeed.tech/tags/leaderboard.md>), [synthetic-data](<https://devfeed.tech/tags/synthetic-data.md>), [trl](<https://devfeed.tech/tags/trl.md>), [vllm](<https://devfeed.tech/tags/vllm.md>)

### AI overview

This update reports early progress on the open-r1 project to reproduce the DeepSeek-R1 training pipeline and dataset. It covers matching MATH-500 evaluation results, the unusually long model responses and their GPU-memory implications, a public evaluation leaderboard, and the integration of GRPO into TRL with DeepSpeed and vLLM support.

### Source excerpt

It's been two weeks since the release of DeepSeek R1 and just a week since we started the open-r1 project to replicate the missing pieces, namely the training pipeline and the synthetic data. This post summarizes: - the progress of Open-R1 to replicate the DeepSeek-R1 pipeline and dataset - what we learned about DeepSeek-R1 and discussions around it - cool projects the community has built since the release of DeepSeek-R1 It should serve both as an update on the project and as a collection of...

## Putting RL back in RLHF

DevFeed: [Putting RL back in RLHF](<https://devfeed.tech/articles/putting-rl-back-in-rlhf-7446.md>)

Original publisher: [Read original article](<https://huggingface.co/blog/putting_rl_back_in_rlhf_with_rloo>)

Author: Shengyi Costa Huang; Arash Ahmadian

Published: 2024-06-12T00:00:00Z

Content type: article

Language: en

Sources: [Hugging Face - Blog](<https://devfeed.tech/sources/hugging-face-blog.md>)

Topics: [rlhf](<https://devfeed.tech/topics/rlhf.md>), [trl](<https://devfeed.tech/topics/trl.md>), [Reinforcement learning](<https://devfeed.tech/topics/reinforcement-learning.md>), [dpo](<https://devfeed.tech/topics/dpo.md>), [cohere](<https://devfeed.tech/topics/cohere.md>)

Tags: [cohere](<https://devfeed.tech/tags/cohere.md>), [dpo](<https://devfeed.tech/tags/dpo.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [reinforcement-learning](<https://devfeed.tech/tags/reinforcement-learning.md>), [research](<https://devfeed.tech/tags/research.md>), [rl](<https://devfeed.tech/tags/rl.md>), [rlhf](<https://devfeed.tech/tags/rlhf.md>), [trl](<https://devfeed.tech/tags/trl.md>)

### AI overview

This article introduces the RLOO Trainer in TRL, an online reinforcement learning algorithm for RLHF designed as a more accessible alternative to PPO. It explains that RLOO uses less GPU memory, converges faster, performs competitively with PPO, and outperforms offline methods such as DPO in the reported comparisons.

### Source excerpt

We're on a journey to advance and democratize artificial intelligence through open source and open science.

## Welcome Llama 3 - Meta's new open LLM

DevFeed: [Welcome Llama 3 - Meta's new open LLM](<https://devfeed.tech/articles/welcome-llama-3-meta-s-new-open-llm-7334.md>)

Original publisher: [Read original article](<https://huggingface.co/blog/llama3>)

Author: Philipp Schmid; Omar Sanseviero; Pedro Cuenca; Younes B; Leandro von Werra

Published: 2024-04-18T00:00:00Z

Content type: release

Language: en

Sources: [Hugging Face - Blog](<https://devfeed.tech/sources/hugging-face-blog.md>)

Topics: [llama](<https://devfeed.tech/topics/llama.md>), [Meta](<https://devfeed.tech/topics/meta.md>), [hugging face](<https://devfeed.tech/topics/hugging-face.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>), [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [Open Source](<https://devfeed.tech/topics/open-source.md>), [Fine-tuning](<https://devfeed.tech/topics/fine-tuning.md>), [Transformers](<https://devfeed.tech/topics/transformers.md>), [trl](<https://devfeed.tech/topics/trl.md>), [Google AI](<https://devfeed.tech/topics/google-ai.md>), [Amazon SageMaker](<https://devfeed.tech/topics/amazon-sagemaker.md>), [Google Cloud Platform (GCP)](<https://devfeed.tech/topics/google-cloud.md>)

Tags: [amazon-sagemaker](<https://devfeed.tech/tags/amazon-sagemaker.md>), [artificial-intelligence](<https://devfeed.tech/tags/artificial-intelligence.md>), [community](<https://devfeed.tech/tags/community.md>), [fine-tuning](<https://devfeed.tech/tags/fine-tuning.md>), [google-cloud](<https://devfeed.tech/tags/google-cloud.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [hardware](<https://devfeed.tech/tags/hardware.md>), [hugging-face](<https://devfeed.tech/tags/hugging-face.md>), [inference](<https://devfeed.tech/tags/inference.md>), [inference-endpoints](<https://devfeed.tech/tags/inference-endpoints.md>), [llama](<https://devfeed.tech/tags/llama.md>), [llm](<https://devfeed.tech/tags/llm.md>), [meta](<https://devfeed.tech/tags/meta.md>), [nlp](<https://devfeed.tech/tags/nlp.md>), [open-source](<https://devfeed.tech/tags/open-source.md>), [research](<https://devfeed.tech/tags/research.md>), [tokens](<https://devfeed.tech/tags/tokens.md>), [transformers](<https://devfeed.tech/tags/transformers.md>), [trl](<https://devfeed.tech/tags/trl.md>)

### AI overview

Meta has released Llama 3, an open-access LLM family available through Hugging Face. The release includes 8B and 70B base and instruction-tuned models, plus Llama Guard 2 for classifying potentially unsafe LLM inputs and responses. It also adds integrations with Transformers, Hugging Chat, Inference Endpoints, Google Cloud, Amazon SageMaker, and TRL.

### Source excerpt

We're on a journey to advance and democratize artificial intelligence through open source and open science.

## Vision Language Models Explained

DevFeed: [Vision Language Models Explained](<https://devfeed.tech/articles/vision-language-models-explained-7560.md>)

Original publisher: [Read original article](<https://huggingface.co/blog/vlms>)

Author: merve; Edward Beeching

Published: 2024-04-11T00:00:00Z

Content type: article

Language: en

Sources: [Hugging Face - Blog](<https://devfeed.tech/sources/hugging-face-blog.md>)

Topics: [vlm](<https://devfeed.tech/topics/vlm.md>), [multimodal](<https://devfeed.tech/topics/multimodal.md>), [hugging face](<https://devfeed.tech/topics/hugging-face.md>), [trl](<https://devfeed.tech/topics/trl.md>), [Inference](<https://devfeed.tech/topics/inference.md>), [Machine learning](<https://devfeed.tech/topics/machine-learning.md>)

Tags: [blog-post](<https://devfeed.tech/tags/blog-post.md>), [guide](<https://devfeed.tech/tags/guide.md>), [hallucinations](<https://devfeed.tech/tags/hallucinations.md>), [hugging-face](<https://devfeed.tech/tags/hugging-face.md>), [inference](<https://devfeed.tech/tags/inference.md>), [leaderboard](<https://devfeed.tech/tags/leaderboard.md>), [models](<https://devfeed.tech/tags/models.md>), [multimodal](<https://devfeed.tech/tags/multimodal.md>), [trl](<https://devfeed.tech/tags/trl.md>), [vision](<https://devfeed.tech/tags/vision.md>), [vlm](<https://devfeed.tech/tags/vlm.md>), [zero-shot](<https://devfeed.tech/tags/zero-shot.md>)

### AI overview

An introduction to vision language models that explains their architecture, capabilities, use cases, model selection, inference, and fine-tuning with trl.

### Source excerpt

We're on a journey to advance and democratize artificial intelligence through open source and open science.