# grpo

Group Relative Policy Optimization (GRPO) is an algorithm for training large language models with reinforcement learning that uses relative rewards from multiple sampled responses instead of a value-function approximation.

This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles.

## How to Fine-Tune LLMs in 2026

DevFeed: [How to Fine-Tune LLMs in 2026](<https://devfeed.tech/articles/how-to-fine-tune-llms-in-2026-31467.md>)

Original publisher: [Read original article](<https://blog.dailydoseofds.com/p/how-to-fine-tune-llms-in-2026-bf8>)

Author: Avi Chawla

Published: 2026-09-16T20:40:26Z

Content type: tutorial

Language: en

Sources: [Daily Dose of Data Science](<https://devfeed.tech/sources/daily-dose-of-data-science.md>)

Topics: [Fine-tuning](<https://devfeed.tech/topics/fine-tuning.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>), [grpo](<https://devfeed.tech/topics/grpo.md>), [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>)

Tags: [agents](<https://devfeed.tech/tags/agents.md>), [ai](<https://devfeed.tech/tags/ai.md>), [fine-tuning](<https://devfeed.tech/tags/fine-tuning.md>), [llms](<https://devfeed.tech/tags/llms.md>), [rl](<https://devfeed.tech/tags/rl.md>)

### AI overview

A developer newsletter explains how supervised fine-tuning differs from reinforcement fine-tuning for LLMs and describes GRPO and RULER as approaches for training agents through experience without manually written reward functions or labeled examples. It also briefly discusses Rowboat Spaces, an open-source shared workspace for personal AI assistants.

### Source excerpt

Reward-free RL is here!

## DACA-GRPO: Denoising-Aware Credit Assignment for Reinforcement Learning in Diffusion Language Models

DevFeed: [DACA-GRPO: Denoising-Aware Credit Assignment for Reinforcement Learning in Diffusion Language Models](<https://devfeed.tech/articles/daca-grpo-denoising-aware-credit-assignment-for-reinforcement-learning-in-diffusion-language-models-31489.md>)

Original publisher: [Read original article](<https://machinelearning.apple.com/research/denoising-aware-credit-assignment>)

Published: 2026-09-16T00:00:00Z

Content type: article

Language: en

Sources: [Apple Machine Learning Research](<https://devfeed.tech/sources/apple-machine-learning-research.md>)

Topics: [grpo](<https://devfeed.tech/topics/grpo.md>), [Reinforcement learning](<https://devfeed.tech/topics/reinforcement-learning.md>), [large-language-models](<https://devfeed.tech/topics/large-language-models.md>), [LLM Techniques](<https://devfeed.tech/topics/llm-techniques.md>), [Code](<https://devfeed.tech/topics/code.md>)

Tags: [code-generation](<https://devfeed.tech/tags/code-generation.md>), [denoising](<https://devfeed.tech/tags/denoising.md>), [diffusion](<https://devfeed.tech/tags/diffusion.md>), [grpo](<https://devfeed.tech/tags/grpo.md>), [json-schema](<https://devfeed.tech/tags/json-schema.md>), [optimization](<https://devfeed.tech/tags/optimization.md>), [reinforcement-learning](<https://devfeed.tech/tags/reinforcement-learning.md>)

### AI overview

The article introduces DACA-GRPO, a denoising-aware enhancement for GRPO-style reinforcement-learning trainers used with diffusion language models. It uses per-token denoising progress scores and stratified masking likelihood to improve credit assignment and reduce bias in likelihood estimates. Across seven benchmarks, the method reports gains in mathematical reasoning, code generation, constraint satisfaction, and JSON Schema adherence.

### Source excerpt

Diffusion large language models are a compelling alternative to autoregressive models, yet existing RL methods for diffusion treat all denoising steps as equally important and rely on biased, high-variance likelihood estimates. We identify two fundamental weaknesses: the absence of temporal credit assignment across the denoising trajectory, and the systematic bias of mean-field likelihood estimates used for policy optimization. To address these, we propose Denoising-Aware Credit Assignment for GRPO (DACA-GRPO), a lightweight, plug-and-play enhancement for any GRPO-style trainer. DACA-GRPO...

## Build an AI-powered product tagging system with Amazon SageMaker serverless model customization

DevFeed: [Build an AI-powered product tagging system with Amazon SageMaker serverless model customization](<https://devfeed.tech/articles/build-an-ai-powered-product-tagging-system-with-amazon-sagemaker-serverless-model-customization-26940.md>)

Original publisher: [Read original article](<https://aws.amazon.com/blogs/machine-learning/build-an-ai-powered-product-tagging-system-with-amazon-sagemaker-serverless-model-customization/>)

Author: Linpo Guo

Published: 2026-09-15T16:11:36Z

Content type: tutorial

Language: en

Sources: [Artificial Intelligence](<https://devfeed.tech/sources/artificial-intelligence.md>)

Topics: [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [Amazon SageMaker AI](<https://devfeed.tech/topics/amazon-sagemaker-ai.md>), [Fine-tuning](<https://devfeed.tech/topics/fine-tuning.md>), [rlvr](<https://devfeed.tech/topics/rlvr.md>), [grpo](<https://devfeed.tech/topics/grpo.md>), [Inference](<https://devfeed.tech/topics/inference.md>), [model-deployment](<https://devfeed.tech/topics/model-deployment.md>), [Reinforcement learning](<https://devfeed.tech/topics/reinforcement-learning.md>), [SDK](<https://devfeed.tech/topics/sdk.md>), [Serverless](<https://devfeed.tech/topics/serverless.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [amazon-sagemaker](<https://devfeed.tech/tags/amazon-sagemaker.md>), [amazon-sagemaker-ai](<https://devfeed.tech/tags/amazon-sagemaker-ai.md>), [aws](<https://devfeed.tech/tags/aws.md>), [compute](<https://devfeed.tech/tags/compute.md>), [cost](<https://devfeed.tech/tags/cost.md>), [customization](<https://devfeed.tech/tags/customization.md>), [expert-400](<https://devfeed.tech/tags/expert-400.md>), [fine-tuning](<https://devfeed.tech/tags/fine-tuning.md>), [inference](<https://devfeed.tech/tags/inference.md>), [jobs](<https://devfeed.tech/tags/jobs.md>), [reinforcement-learning](<https://devfeed.tech/tags/reinforcement-learning.md>), [rlvr](<https://devfeed.tech/tags/rlvr.md>), [sdk](<https://devfeed.tech/tags/sdk.md>), [technical-how-to](<https://devfeed.tech/tags/technical-how-to.md>)

### AI overview

This walkthrough shows how to build a product tagging system by customizing Qwen3-8B with supervised fine-tuning and reinforcement learning with verifiable rewards on Amazon SageMaker serverless model customization. It then deploys the optimized model for asynchronous inference to enrich retail catalogs.

### Source excerpt

Manually tagging thousands of catalog products is slow and inconsistent. This walkthrough shows how to customize Qwen3-8B with supervised fine-tuning (SFT) and reinforcement learning with verifiable rewards (RLVR) on Amazon SageMaker serverless model customization, then deploy it for asynchronous inference to build a cost-efficient product tagging system.

## Autonomous LLM post-training with Tunix on TPUs

DevFeed: [Autonomous LLM post-training with Tunix on TPUs](<https://devfeed.tech/articles/autonomous-llm-post-training-with-tunix-on-tpus-4205.md>)

Original publisher: [Read original article](<https://developers.googleblog.com/autonomous-llm-post-training-with-tunix-on-tpus/>)

Author: Wei Wei

Published: 2026-09-12T11:04:33.891311Z

Content type: article

Language: en

Sources: [Google Developers Blog](<https://devfeed.tech/sources/google-developers-blog.md>)

Topics: [post-training](<https://devfeed.tech/topics/post-training.md>), [AI Agent](<https://devfeed.tech/topics/ai-agent.md>), [Fine-tuning](<https://devfeed.tech/topics/fine-tuning.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>), [grpo](<https://devfeed.tech/topics/grpo.md>), [lora](<https://devfeed.tech/topics/lora.md>), [gemma](<https://devfeed.tech/topics/gemma.md>), [Google](<https://devfeed.tech/topics/google.md>), [Reinforcement learning](<https://devfeed.tech/topics/reinforcement-learning.md>), [dataset](<https://devfeed.tech/topics/dataset.md>), [Command-line interface](<https://devfeed.tech/topics/cli.md>)

Tags: [autonomous](<https://devfeed.tech/tags/autonomous.md>), [cli](<https://devfeed.tech/tags/cli.md>), [fine-tuning](<https://devfeed.tech/tags/fine-tuning.md>), [gemma](<https://devfeed.tech/tags/gemma.md>), [git](<https://devfeed.tech/tags/git.md>), [google](<https://devfeed.tech/tags/google.md>), [grpo](<https://devfeed.tech/tags/grpo.md>), [llm](<https://devfeed.tech/tags/llm.md>), [lora](<https://devfeed.tech/tags/lora.md>), [post-training](<https://devfeed.tech/tags/post-training.md>), [reinforcement-learning](<https://devfeed.tech/tags/reinforcement-learning.md>), [training](<https://devfeed.tech/tags/training.md>)

### AI overview

This article presents autofinetune, an autonomous research loop for LLM post-training. Using AI agents and Google's AI stack, including Tunix, Gemma, Cloud TPUs, Antigravity CLI, and Gemini Flash 3.7, it automates supervised fine-tuning and reinforcement learning with GRPO, exploring hyperparameters such as LoRA configurations, learning rates, batch sizes, and rollout settings.

### Source excerpt

Imagine going to sleep after writing a single Markdown specification and waking up to find that an A...

## Async GRPO with LoRA across HF Jobs: a bucket, a proxy, and no NCCL

DevFeed: [Async GRPO with LoRA across HF Jobs: a bucket, a proxy, and no NCCL](<https://devfeed.tech/articles/async-grpo-with-lora-across-hf-jobs-a-bucket-a-proxy-and-no-nccl-17376.md>)

Original publisher: [Read original article](<https://huggingface.co/blog/asyncgrpo-lora-hfjobs>)

Author: Amine Dirhoussi; Quentin Gallouédec; Kashif Rasul; Sergio Paniego

Published: 2026-09-10T00:00:00Z

Content type: article

Language: en

Sources: [Hugging Face - Blog](<https://devfeed.tech/sources/hugging-face-blog.md>)

Topics: [lora](<https://devfeed.tech/topics/lora.md>), [grpo](<https://devfeed.tech/topics/grpo.md>), [jobs](<https://devfeed.tech/topics/jobs.md>), [vllm](<https://devfeed.tech/topics/vllm.md>), [NCCL](<https://devfeed.tech/topics/nccl.md>), [async](<https://devfeed.tech/topics/async.md>), [hugging face](<https://devfeed.tech/topics/hugging-face.md>)

Tags: [async](<https://devfeed.tech/tags/async.md>), [grpo](<https://devfeed.tech/tags/grpo.md>), [hub](<https://devfeed.tech/tags/hub.md>), [hugging-face](<https://devfeed.tech/tags/hugging-face.md>), [jobs](<https://devfeed.tech/tags/jobs.md>), [llm](<https://devfeed.tech/tags/llm.md>), [lora](<https://devfeed.tech/tags/lora.md>), [nccl](<https://devfeed.tech/tags/nccl.md>), [open-source](<https://devfeed.tech/tags/open-source.md>), [rl](<https://devfeed.tech/tags/rl.md>), [storage](<https://devfeed.tech/tags/storage.md>), [trl](<https://devfeed.tech/tags/trl.md>), [vllm](<https://devfeed.tech/tags/vllm.md>)

### AI overview

This article describes asynchronous GRPO training with a LoRA adapter across separate Hugging Face Jobs. The adapter is synchronized to vLLM replicas through a shared Storage Bucket, while a proxy handles authentication, rollout routing, and adapter-load broadcasts. Five runs reduced the time for 500 steps from 3 hours 27 minutes to 53 minutes.

### Source excerpt

We're on a journey to advance and democratize artificial intelligence through open source and open science.

## 🗓 This Week In AI Research (1-8 July 26)

DevFeed: [🗓 This Week In AI Research (1-8 July 26)](<https://devfeed.tech/articles/this-week-in-ai-research-1-8-july-26-18283.md>)

Original publisher: [Read original article](<https://www.intoai.pub/p/this-week-in-ai-research-1-8-july>)

Author: Dr. Ashish Bamania

Published: 2026-07-12T11:25:32Z

Content type: article

Language: en

Sources: [Into AI](<https://devfeed.tech/sources/into-ai.md>)

Topics: [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [AI Research](<https://devfeed.tech/topics/ai-research.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>), [Transformer](<https://devfeed.tech/topics/transformer.md>), [qwen](<https://devfeed.tech/topics/qwen.md>), [grpo](<https://devfeed.tech/topics/grpo.md>), [Algorithms](<https://devfeed.tech/topics/algorithms.md>), [releases](<https://devfeed.tech/topics/releases.md>)

Tags: [agentic](<https://devfeed.tech/tags/agentic.md>), [ai](<https://devfeed.tech/tags/ai.md>), [algorithm](<https://devfeed.tech/tags/algorithm.md>), [algorithms](<https://devfeed.tech/tags/algorithms.md>), [grpo](<https://devfeed.tech/tags/grpo.md>), [llm](<https://devfeed.tech/tags/llm.md>), [models](<https://devfeed.tech/tags/models.md>), [performance](<https://devfeed.tech/tags/performance.md>), [qwen](<https://devfeed.tech/tags/qwen.md>), [ranking](<https://devfeed.tech/tags/ranking.md>), [releases](<https://devfeed.tech/tags/releases.md>), [research](<https://devfeed.tech/tags/research.md>), [rl](<https://devfeed.tech/tags/rl.md>), [training](<https://devfeed.tech/tags/training.md>), [update](<https://devfeed.tech/tags/update.md>)

### AI overview

A weekly roundup of AI research papers and releases highlights findings that reinforcement-learning gains can be concentrated in a single transformer layer and presents LLM-as-a-Verifier, a framework for continuous scoring and ranking of agentic-task solutions.

### Source excerpt

The top 10 research papers and AI releases this week (SpaceXAI's Grok 4.5, OpenAI's GPT-Live voice models, Cognition's SWE-1.7, Meta's Muse Spark 1.1, and many more)

## 10 Confusing LLM Concepts, Explained Simply

DevFeed: [10 Confusing LLM Concepts, Explained Simply](<https://devfeed.tech/articles/10-confusing-llm-concepts-explained-simply-18351.md>)

Original publisher: [Read original article](<https://levelup.gitconnected.com/10-confusing-llm-concepts-explained-simply-031246b8ea34?source=rss-f10e9a50984a------2>)

Author: Dr. Ashish Bamania

Published: 2026-06-01T15:52:18Z

Content type: tutorial

Language: en

Sources: [Dr. Ashish Bamania](<https://devfeed.tech/sources/dr-ashish-bamania.md>)

Topics: [Large Language Model](<https://devfeed.tech/topics/llm.md>), [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [AI Agent](<https://devfeed.tech/topics/ai-agent.md>), [grpo](<https://devfeed.tech/topics/grpo.md>), [dataset](<https://devfeed.tech/topics/dataset.md>), [coding](<https://devfeed.tech/topics/coding.md>), [math](<https://devfeed.tech/topics/math.md>)

Tags: [agent](<https://devfeed.tech/tags/agent.md>), [ai](<https://devfeed.tech/tags/ai.md>), [artificial-intelligence](<https://devfeed.tech/tags/artificial-intelligence.md>), [coding](<https://devfeed.tech/tags/coding.md>), [cpu](<https://devfeed.tech/tags/cpu.md>), [data-science](<https://devfeed.tech/tags/data-science.md>), [dataset](<https://devfeed.tech/tags/dataset.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [grpo](<https://devfeed.tech/tags/grpo.md>), [llm](<https://devfeed.tech/tags/llm.md>), [llms](<https://devfeed.tech/tags/llms.md>), [machine-learning](<https://devfeed.tech/tags/machine-learning.md>), [math](<https://devfeed.tech/tags/math.md>), [programming](<https://devfeed.tech/tags/programming.md>), [quantization](<https://devfeed.tech/tags/quantization.md>), [technology](<https://devfeed.tech/tags/technology.md>), [tpu](<https://devfeed.tech/tags/tpu.md>), [train](<https://devfeed.tech/tags/train.md>), [training](<https://devfeed.tech/tags/training.md>)

### AI overview

This tutorial introduces LLM concepts including on-policy and off-policy learning. It explains how models generate, score, and learn from responses, including the use of GRPO, teacher models, and datasets. The supplied excerpt also identifies CPU, GPU, TPU, pruning, and quantization as covered topics.

### Source excerpt

The role of CPU/ GPU/ TPU in LLM workflows, Pruning, Quantization, and more. Continue reading on Level Up Coding "

## Granite 4.1 LLMs: How They're Built

DevFeed: [Granite 4.1 LLMs: How They're Built](<https://devfeed.tech/articles/granite-4-1-llms-how-they-re-built-7256.md>)

Original publisher: [Read original article](<https://huggingface.co/blog/ibm-granite/granite-4-1>)

Author: Yousaf Shah

Published: 2026-04-29T15:01:48Z

Content type: article

Language: en

Sources: [Hugging Face - Blog](<https://devfeed.tech/sources/hugging-face-blog.md>)

Topics: [LLMs](<https://devfeed.tech/topics/llms.md>), [ibm](<https://devfeed.tech/topics/ibm.md>), [long-context](<https://devfeed.tech/topics/long-context.md>), [Fine-tuning](<https://devfeed.tech/topics/fine-tuning.md>), [grpo](<https://devfeed.tech/topics/grpo.md>), [Training AI Models](<https://devfeed.tech/topics/training-ai-models.md>), [Transformer architecture](<https://devfeed.tech/topics/transformer-architecture.md>), [Data Quality](<https://devfeed.tech/topics/data-quality.md>)

Tags: [apache](<https://devfeed.tech/tags/apache.md>), [fine-tuning](<https://devfeed.tech/tags/fine-tuning.md>), [grpo](<https://devfeed.tech/tags/grpo.md>), [ibm](<https://devfeed.tech/tags/ibm.md>), [learning](<https://devfeed.tech/tags/learning.md>), [llms](<https://devfeed.tech/tags/llms.md>), [long-context](<https://devfeed.tech/tags/long-context.md>), [tokens](<https://devfeed.tech/tags/tokens.md>), [training](<https://devfeed.tech/tags/training.md>), [transformer-architecture](<https://devfeed.tech/tags/transformer-architecture.md>)

### AI overview

The article introduces Granite 4.1, IBM's family of dense decoder-only LLMs in 3B, 8B, and 30B sizes. It describes their five-stage training process, which uses about 15 trillion tokens, data-quality refinement, long-context extension up to 512K tokens, supervised fine-tuning, and reinforcement learning with on-policy GRPO and DAPO loss. The models use a dense transformer architecture and are released under the Apache 2.0 license.

### Source excerpt

Authors: Granite Team, IBM TL;DR -- Granite 4.1 is a family of dense, decoder-only LLMs (3B, 8B, and 30B) trained on ~15T tokens using a multi-stage pre-training pipeline, including long-context extension of up to 512K tokens. The models are further refined with supervised fine-tuning on ~4.1M high-quality curated samples and reinforcement learning via on-policy GRPO with DAPO loss (Yu et al., 2025).

## TRL v1.0: Post-Training Library Built to Move with the Field

DevFeed: [TRL v1.0: Post-Training Library Built to Move with the Field](<https://devfeed.tech/articles/trl-v1-0-post-training-library-built-to-move-with-the-field-7541.md>)

Original publisher: [Read original article](<https://huggingface.co/blog/trl-v1>)

Author: Quentin Gallouédec; Steven Liu; Pedro Cuenca; Sergio Paniego

Published: 2026-03-31T00:00:00Z

Content type: article

Language: en

Sources: [Hugging Face - Blog](<https://devfeed.tech/sources/hugging-face-blog.md>)

Topics: [trl](<https://devfeed.tech/topics/trl.md>), [post-training](<https://devfeed.tech/topics/post-training.md>), [dpo](<https://devfeed.tech/topics/dpo.md>), [grpo](<https://devfeed.tech/topics/grpo.md>), [rlvr](<https://devfeed.tech/topics/rlvr.md>), [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [Code](<https://devfeed.tech/topics/code.md>)

Tags: [announcement](<https://devfeed.tech/tags/announcement.md>), [artificial-intelligence](<https://devfeed.tech/tags/artificial-intelligence.md>), [code](<https://devfeed.tech/tags/code.md>), [dpo](<https://devfeed.tech/tags/dpo.md>), [grpo](<https://devfeed.tech/tags/grpo.md>), [open-source](<https://devfeed.tech/tags/open-source.md>), [post-training](<https://devfeed.tech/tags/post-training.md>), [reinforcement-learning](<https://devfeed.tech/tags/reinforcement-learning.md>), [rlvr](<https://devfeed.tech/tags/rlvr.md>), [software](<https://devfeed.tech/tags/software.md>), [trl](<https://devfeed.tech/tags/trl.md>), [v1](<https://devfeed.tech/tags/v1.md>)

### AI overview

TRL v1.0 presents a post-training library designed to remain useful as algorithms, models, and paradigms change. The article explains the evolution from PPO to DPO-style preference optimization and RLVR methods such as GRPO, emphasizing flexible abstractions and practical comparison over a supposedly permanent design.

### Source excerpt

We're on a journey to advance and democratize artificial intelligence through open source and open science.

## DeepMath: A lightweight math reasoning Agent with smolagents

DevFeed: [DeepMath: A lightweight math reasoning Agent with smolagents](<https://devfeed.tech/articles/deepmath-a-lightweight-math-reasoning-agent-with-smolagents-7288.md>)

Original publisher: [Read original article](<https://huggingface.co/blog/intel-deepmath>)

Author: Daniel Fleischer; Moshe Berchansky; Moshe Wasserblat

Published: 2025-12-04T00:00:00Z

Content type: article

Language: en

Sources: [Hugging Face - Blog](<https://devfeed.tech/sources/hugging-face-blog.md>)

Topics: [smolagents](<https://devfeed.tech/topics/smolagents.md>), [math](<https://devfeed.tech/topics/math.md>), [grpo](<https://devfeed.tech/topics/grpo.md>), [Fine-tuning](<https://devfeed.tech/topics/fine-tuning.md>), [Python](<https://devfeed.tech/topics/python.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>), [vllm](<https://devfeed.tech/topics/vllm.md>), [trl](<https://devfeed.tech/topics/trl.md>), [datasets](<https://devfeed.tech/topics/datasets.md>), [Inference](<https://devfeed.tech/topics/inference.md>), [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [Open Source](<https://devfeed.tech/topics/open-source.md>)

Tags: [agents](<https://devfeed.tech/tags/agents.md>), [artificial-intelligence](<https://devfeed.tech/tags/artificial-intelligence.md>), [datasets](<https://devfeed.tech/tags/datasets.md>), [fine-tuning](<https://devfeed.tech/tags/fine-tuning.md>), [grpo](<https://devfeed.tech/tags/grpo.md>), [inference](<https://devfeed.tech/tags/inference.md>), [large-language-models-llms](<https://devfeed.tech/tags/large-language-models-llms.md>), [llm](<https://devfeed.tech/tags/llm.md>), [math](<https://devfeed.tech/tags/math.md>), [math-reasoning](<https://devfeed.tech/tags/math-reasoning.md>), [open-source](<https://devfeed.tech/tags/open-source.md>), [python](<https://devfeed.tech/tags/python.md>), [reasoning](<https://devfeed.tech/tags/reasoning.md>), [sandbox](<https://devfeed.tech/tags/sandbox.md>), [smolagents](<https://devfeed.tech/tags/smolagents.md>), [trl](<https://devfeed.tech/tags/trl.md>), [vllm](<https://devfeed.tech/tags/vllm.md>)

### AI overview

DeepMath is a lightweight math reasoning agent built on Qwen3-4B Thinking and implemented with smolagents. It uses GRPO fine-tuning to encourage concise Python snippets for deterministic computation, executes those snippets in a constrained sandbox, and reintegrates the results into the reasoning process. Evaluation on MATH500, AIME, HMMT, and HLE reports shorter outputs and often improved accuracy.

### Source excerpt

We're on a journey to advance and democratize artificial intelligence through open source and open science.

## On the Shifting Global Compute Landscape

DevFeed: [On the Shifting Global Compute Landscape](<https://devfeed.tech/articles/on-the-shifting-global-compute-landscape-7253.md>)

Original publisher: [Read original article](<https://huggingface.co/blog/huggingface/shifting-compute-landscape>)

Author: Tiezhen WANG; Irene Solaiman

Published: 2025-10-29T13:56:45Z

Content type: article

Language: en

Sources: [Hugging Face - Blog](<https://devfeed.tech/sources/hugging-face-blog.md>)

Topics: [AI Development](<https://devfeed.tech/topics/ai-development.md>), [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [Nvidia](<https://devfeed.tech/topics/nvidia.md>), [Open Source](<https://devfeed.tech/topics/open-source.md>), [Inference](<https://devfeed.tech/topics/inference.md>), [CUDA](<https://devfeed.tech/topics/cuda.md>), [Optimization](<https://devfeed.tech/topics/optimization.md>), [huawei](<https://devfeed.tech/topics/huawei.md>), [model-deployment](<https://devfeed.tech/topics/model-deployment.md>), [deepseek](<https://devfeed.tech/topics/deepseek.md>), [grpo](<https://devfeed.tech/topics/grpo.md>), [Deployment](<https://devfeed.tech/topics/deployment.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [ai-development](<https://devfeed.tech/tags/ai-development.md>), [ai-models](<https://devfeed.tech/tags/ai-models.md>), [china](<https://devfeed.tech/tags/china.md>), [cuda](<https://devfeed.tech/tags/cuda.md>), [culture](<https://devfeed.tech/tags/culture.md>), [deepseek](<https://devfeed.tech/tags/deepseek.md>), [deployment](<https://devfeed.tech/tags/deployment.md>), [developers](<https://devfeed.tech/tags/developers.md>), [efficiency](<https://devfeed.tech/tags/efficiency.md>), [grpo](<https://devfeed.tech/tags/grpo.md>), [huawei](<https://devfeed.tech/tags/huawei.md>), [inference](<https://devfeed.tech/tags/inference.md>), [nvidia](<https://devfeed.tech/tags/nvidia.md>), [open-source](<https://devfeed.tech/tags/open-source.md>), [optimization](<https://devfeed.tech/tags/optimization.md>), [production](<https://devfeed.tech/tags/production.md>), [training](<https://devfeed.tech/tags/training.md>)

### AI overview

The article examines how China's progress in open-weight AI models is coinciding with rapid domestic AI chip development. It discusses Huawei Ascend and Cambricon chips, reduced reliance on NVIDIA, compute-efficiency innovations such as DeepSeek's MLA and GRPO, and the emergence of software alternatives to CUDA.

### Source excerpt

The status quo of AI chip usage, that was once almost entirely U.S.-based, is changing. China's immense progress in open-weight AI development is now being met with rapid domestic AI chip development. In the past few months, highly performant open-weight AI models' inference in China has started to be powered by chips such as Huawei's Ascend and Cambricon, with some models starting to be trained using domestic chips.

## Building production-ready agentic systems: Lessons from Shopify Sidekick

DevFeed: [Building production-ready agentic systems: Lessons from Shopify Sidekick](<https://devfeed.tech/articles/building-production-ready-agentic-systems-lessons-from-shopify-sidekick-1329.md>)

Original publisher: [Read original article](<https://shopify.engineering/building-production-ready-agentic-systems>)

Author: Andrew McNamara

Published: 2025-08-26T10:00:00Z

Content type: article

Language: en

Sources: [Shopify Engineering](<https://devfeed.tech/sources/shopify-engineering.md>), [Shopify Engineering - Shopify Engineering](<https://devfeed.tech/sources/shopify-engineering-shopify-engineering.md>)

Topics: [AI Engineering](<https://devfeed.tech/topics/ai-engineering.md>), [Shopify](<https://devfeed.tech/topics/shopify.md>), [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [Architecture & Design](<https://devfeed.tech/topics/architecture-design.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>), [grpo](<https://devfeed.tech/topics/grpo.md>), [JIT](<https://devfeed.tech/topics/jit.md>), [data](<https://devfeed.tech/topics/data.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [ai-engineering](<https://devfeed.tech/tags/ai-engineering.md>), [architecture-design](<https://devfeed.tech/tags/architecture-design.md>), [evaluation](<https://devfeed.tech/tags/evaluation.md>), [grpo](<https://devfeed.tech/tags/grpo.md>), [jit](<https://devfeed.tech/tags/jit.md>), [llm](<https://devfeed.tech/tags/llm.md>), [platform](<https://devfeed.tech/tags/platform.md>), [production](<https://devfeed.tech/tags/production.md>), [shopify](<https://devfeed.tech/tags/shopify.md>)

### AI overview

This article describes how Shopify evolved Sidekick from a tool-calling system into a production-oriented agentic platform. It discusses the agentic loop, scaling challenges caused by a growing tool inventory, architecture design, evaluation, GRPO training, and the use of just-in-time instructions to reduce system-prompt complexity.

### Source excerpt

Learn how we evolved our AI assistant architecture and built robust evaluation frameworks for real-world deployment.

## Kimina-Prover-RL

DevFeed: [Kimina-Prover-RL](<https://devfeed.tech/articles/kimina-prover-rl-6980.md>)

Original publisher: [Read original article](<https://huggingface.co/blog/AI-MO/kimina-prover-rl>)

Author: Thibaut Barroyer; Jonas Bayer; Marina Vinyes; Mert Unsal; Haiming Wang; Xiaohan Lin; MantasBaksys; Junqi Liu; Marco Dos Santos; Flood Sung; Ying; Zhu Zekai; Lujianqiao; Hugues de Saxcé; Ebony Zhang; B

Published: 2025-08-14T12:13:01Z

Content type: article

Language: en

Sources: [Hugging Face - Blog](<https://devfeed.tech/sources/hugging-face-blog.md>)

Topics: [Lean](<https://devfeed.tech/topics/lean.md>), [Open Source](<https://devfeed.tech/topics/open-source.md>), [grpo](<https://devfeed.tech/topics/grpo.md>), [Reinforcement learning](<https://devfeed.tech/topics/reinforcement-learning.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>), [Benchmark](<https://devfeed.tech/topics/benchmark.md>)

Tags: [benchmark](<https://devfeed.tech/tags/benchmark.md>), [code](<https://devfeed.tech/tags/code.md>), [datasets](<https://devfeed.tech/tags/datasets.md>), [grpo](<https://devfeed.tech/tags/grpo.md>), [open-source](<https://devfeed.tech/tags/open-source.md>), [reasoning](<https://devfeed.tech/tags/reasoning.md>), [reinforcement-learning](<https://devfeed.tech/tags/reinforcement-learning.md>), [training](<https://devfeed.tech/tags/training.md>), [verification](<https://devfeed.tech/tags/verification.md>)

### AI overview

Kimina-Prover-RL is an open-source training pipeline for formal theorem proving in Lean 4. It uses a structured reasoning-then-generation approach, GRPO reinforcement learning, format-checking rewards, error-correction turns, and high-throughput proof verification through kimina-lean-server. The article also introduces 1.7B- and 0.6B-parameter models evaluated on the MiniF2F benchmark.

### Source excerpt

We are happy to introduce kimina-prover-rl, an open-source training pipeline for formal theorem proving in Lean 4, based on a structured reasoning-then-generation paradigm inspired by DeepSeek-R1. This training pipelinee is a simplified version of the system we used to train Kimina Prover, preserving the key components of the system and offering full compatibility with the open-source Verl framework.

## 🐯 Liger GRPO meets TRL

DevFeed: [🐯 Liger GRPO meets TRL](<https://devfeed.tech/articles/liger-grpo-meets-trl-7332.md>)

Original publisher: [Read original article](<https://huggingface.co/blog/liger-grpo>)

Author: Shivam Sahni; Kashif Rasul; Salman Mohammadi; Shirin Yamani; Yanning Chen; Liberty

Published: 2025-05-25T00:00:00Z

Content type: tutorial

Language: en

Sources: [Hugging Face - Blog](<https://devfeed.tech/sources/hugging-face-blog.md>)

Topics: [grpo](<https://devfeed.tech/topics/grpo.md>), [trl](<https://devfeed.tech/topics/trl.md>), [Reinforcement learning](<https://devfeed.tech/topics/reinforcement-learning.md>), [Fine-tuning](<https://devfeed.tech/topics/fine-tuning.md>), [Optimization](<https://devfeed.tech/topics/optimization.md>), [GPU](<https://devfeed.tech/topics/gpu.md>), [LLMs](<https://devfeed.tech/topics/llms.md>), [rlhf](<https://devfeed.tech/topics/rlhf.md>), [coding](<https://devfeed.tech/topics/coding.md>), [math](<https://devfeed.tech/topics/math.md>)

Tags: [coding](<https://devfeed.tech/tags/coding.md>), [fine-tuning](<https://devfeed.tech/tags/fine-tuning.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [grpo](<https://devfeed.tech/tags/grpo.md>), [large-language-models-llms](<https://devfeed.tech/tags/large-language-models-llms.md>), [liger](<https://devfeed.tech/tags/liger.md>), [llm](<https://devfeed.tech/tags/llm.md>), [optimization](<https://devfeed.tech/tags/optimization.md>), [partnership](<https://devfeed.tech/tags/partnership.md>), [performance](<https://devfeed.tech/tags/performance.md>), [reinforcement-learning](<https://devfeed.tech/tags/reinforcement-learning.md>), [rlhf](<https://devfeed.tech/tags/rlhf.md>), [trl](<https://devfeed.tech/tags/trl.md>)

### AI overview

This article explains how Group Relative Policy Optimization (GRPO) can reduce the resource requirements of reinforcement learning fine-tuning for language models. It presents a TRL optimization based on chunked GRPO loss that reduces peak memory usage by 40% and discusses scaling GRPO across multiple GPUs and nodes while preserving performance and correctness.

### Source excerpt

We're on a journey to advance and democratize artificial intelligence through open source and open science.

## PipelineRL

DevFeed: [PipelineRL](<https://devfeed.tech/articles/pipelinerl-7056.md>)

Original publisher: [Read original article](<https://huggingface.co/blog/ServiceNow/pipelinerl>)

Author: Alex Piche; Rafael Pardinas; Ehsan Kamalloo; Bahdanau

Published: 2025-04-25T22:37:16Z

Content type: article

Language: en

Sources: [Hugging Face - Blog](<https://devfeed.tech/sources/hugging-face-blog.md>)

Topics: [grpo](<https://devfeed.tech/topics/grpo.md>), [model-serving](<https://devfeed.tech/topics/model-serving.md>), [gradient\_accumulation](<https://devfeed.tech/topics/gradient-accumulation.md>), [GPU](<https://devfeed.tech/topics/gpu.md>), [datasets](<https://devfeed.tech/topics/datasets.md>), [math](<https://devfeed.tech/topics/math.md>), [Optimization](<https://devfeed.tech/topics/optimization.md>)

Tags: [batch](<https://devfeed.tech/tags/batch.md>), [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [blog-post](<https://devfeed.tech/tags/blog-post.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [grpo](<https://devfeed.tech/tags/grpo.md>), [optimization](<https://devfeed.tech/tags/optimization.md>), [performance](<https://devfeed.tech/tags/performance.md>), [rl](<https://devfeed.tech/tags/rl.md>), [training](<https://devfeed.tech/tags/training.md>)

### AI overview

This blog post presents PipelineRL, a modular reinforcement-learning architecture for training language models. It uses inflight weight updates to preserve efficient inference batching while keeping collected data on-policy or near on-policy. The article reports competitive results against Open-Reasoner-Zero on AIME 2024 and MATH 500 using a simpler GRPO-based implementation.

### Source excerpt

In this blog post, we show that 1) inflight weight updates do not harm the training process and 2) PipelineRL achieves competitive results compared to Open-Reasoner-Zero, while using a simpler RL algorithm. We also present the modular PipelineRL architecture that facilitates trying new inference / trainer combinations. In conventional RL approaches (Figure 1a), there is a trade-off between high throughput inference and on-policy data collection.

## Open-R1: Update #1

DevFeed: [Open-R1: Update #1](<https://devfeed.tech/articles/open-r1-update-1-7420.md>)

Original publisher: [Read original article](<https://huggingface.co/blog/open-r1/update-1>)

Author: Leandro von Werra; Lewis Tunstall; Quentin Gallouédec; Guilherme Penedo; Edward Beeching; Anton Lozhkov; Brigitte Tousignant; Daniel van Strien

Published: 2025-02-02T00:04:28Z

Content type: article

Language: en

Sources: [Hugging Face - Blog](<https://devfeed.tech/sources/hugging-face-blog.md>)

Topics: [deepseek](<https://devfeed.tech/topics/deepseek.md>), [synthetic-data](<https://devfeed.tech/topics/synthetic-data.md>), [datasets](<https://devfeed.tech/topics/datasets.md>), [grpo](<https://devfeed.tech/topics/grpo.md>), [trl](<https://devfeed.tech/topics/trl.md>), [Benchmark](<https://devfeed.tech/topics/benchmark.md>), [vllm](<https://devfeed.tech/topics/vllm.md>), [GPU](<https://devfeed.tech/topics/gpu.md>)

Tags: [benchmark](<https://devfeed.tech/tags/benchmark.md>), [deepseek](<https://devfeed.tech/tags/deepseek.md>), [follow](<https://devfeed.tech/tags/follow.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [grpo](<https://devfeed.tech/tags/grpo.md>), [leaderboard](<https://devfeed.tech/tags/leaderboard.md>), [synthetic-data](<https://devfeed.tech/tags/synthetic-data.md>), [trl](<https://devfeed.tech/tags/trl.md>), [vllm](<https://devfeed.tech/tags/vllm.md>)

### AI overview

This update reports early progress on the open-r1 project to reproduce the DeepSeek-R1 training pipeline and dataset. It covers matching MATH-500 evaluation results, the unusually long model responses and their GPU-memory implications, a public evaluation leaderboard, and the integration of GRPO into TRL with DeepSpeed and vLLM support.

### Source excerpt

It's been two weeks since the release of DeepSeek R1 and just a week since we started the open-r1 project to replicate the missing pieces, namely the training pipeline and the synthetic data. This post summarizes: - the progress of Open-R1 to replicate the DeepSeek-R1 pipeline and dataset - what we learned about DeepSeek-R1 and discussions around it - cool projects the community has built since the release of DeepSeek-R1 It should serve both as an update on the project and as a collection of...

## Mini-R1: Reproduce Deepseek R1 "aha moment" a RL tutorial

DevFeed: [Mini-R1: Reproduce Deepseek R1 "aha moment" a RL tutorial](<https://devfeed.tech/articles/mini-r1-reproduce-deepseek-r1-aha-moment-a-rl-tutorial-7418.md>)

Original publisher: [Read original article](<https://huggingface.co/blog/open-r1/mini-r1-contdown-game>)

Author: Philipp Schmid

Published: 2025-01-31T10:29:40Z

Content type: tutorial

Language: en

Sources: [Hugging Face - Blog](<https://devfeed.tech/sources/hugging-face-blog.md>)

Topics: [deepseek](<https://devfeed.tech/topics/deepseek.md>), [grpo](<https://devfeed.tech/topics/grpo.md>), [Reinforcement learning](<https://devfeed.tech/topics/reinforcement-learning.md>), [distributed-training](<https://devfeed.tech/topics/distributed-training.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>), [vllm](<https://devfeed.tech/topics/vllm.md>), [NCCL](<https://devfeed.tech/topics/nccl.md>), [Nvidia](<https://devfeed.tech/topics/nvidia.md>), [qwen](<https://devfeed.tech/topics/qwen.md>), [OpenAI](<https://devfeed.tech/topics/openai.md>)

Tags: [blog](<https://devfeed.tech/tags/blog.md>), [blog-post](<https://devfeed.tech/tags/blog-post.md>), [deepseek](<https://devfeed.tech/tags/deepseek.md>), [development](<https://devfeed.tech/tags/development.md>), [distributed-training](<https://devfeed.tech/tags/distributed-training.md>), [grpo](<https://devfeed.tech/tags/grpo.md>), [llms](<https://devfeed.tech/tags/llms.md>), [model](<https://devfeed.tech/tags/model.md>), [nvidia](<https://devfeed.tech/tags/nvidia.md>), [open](<https://devfeed.tech/tags/open.md>), [qwen](<https://devfeed.tech/tags/qwen.md>), [reasoning](<https://devfeed.tech/tags/reasoning.md>), [reinforcement-learning](<https://devfeed.tech/tags/reinforcement-learning.md>), [rl](<https://devfeed.tech/tags/rl.md>), [train](<https://devfeed.tech/tags/train.md>), [training](<https://devfeed.tech/tags/training.md>), [tutorial](<https://devfeed.tech/tags/tutorial.md>), [vllm](<https://devfeed.tech/tags/vllm.md>)

### AI overview

This tutorial reproduces a small version of DeepSeek-R1's reported reinforcement-learning "aha moment" by training an open model with Group Relative Policy Optimization (GRPO) to solve the Countdown numbers game. It covers self-verification, search behavior, distributed training with DeepSpeed and vLLM, and a four-NVIDIA-H100-GPU setup.

### Source excerpt

The release of Deepseek R1 shocked the industry. Why? Well, DeepSeek-R1 is an open model that rivals OpenAI's o1 in complex reasoning tasks, introduced using Group Relative Policy Optimization (GRPO) and RL-focused multi-stage training approach. They not only released the model, but also a research paper on how they did it. In the paper they described an "aha moment" when using pure RL to train the model.

## DeepSeek's reported efficiency and Nvidia's stock decline

DevFeed: [DeepSeek's reported efficiency and Nvidia's stock decline](<https://devfeed.tech/articles/nvidia-deepseek-ai-24007.md>)

Original publisher: [Read original article](<https://habr.com/ru/companies/redmadrobot/articles/877416/>)

Author: redmadrobot (red\_mad\_robot)

Published: 2025-01-28T16:53:47Z

Content type: opinion

Language: ru

Sources: [Redmadrobot EN](<https://devfeed.tech/sources/redmadrobot-en.md>), [Redmadrobot RU](<https://devfeed.tech/sources/redmadrobot-ru.md>)

Topics: [deepseek](<https://devfeed.tech/topics/deepseek.md>), [Nvidia](<https://devfeed.tech/topics/nvidia.md>), [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [GPU](<https://devfeed.tech/topics/gpu.md>), [moe](<https://devfeed.tech/topics/moe.md>), [grpo](<https://devfeed.tech/topics/grpo.md>), [ChatGPT](<https://devfeed.tech/topics/chatgpt.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [chatgpt](<https://devfeed.tech/tags/chatgpt.md>), [deepseek](<https://devfeed.tech/tags/deepseek.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [grpo](<https://devfeed.tech/tags/grpo.md>), [moe](<https://devfeed.tech/tags/moe.md>), [nvidia](<https://devfeed.tech/tags/nvidia.md>), [openai](<https://devfeed.tech/tags/openai.md>), [tag-13e2af703774](<https://devfeed.tech/tags/tag-13e2af703774.md>), [tag-61cd5a476b1d](<https://devfeed.tech/tags/tag-61cd5a476b1d.md>)

### AI overview

The article argues that DeepSeek's reported lower training and inference costs contributed to concern about Nvidia's GPU-demand business model and coincided with a sharp Nvidia stock decline. It attributes the claimed efficiency to reinforcement learning with GRPO and a Mixture of Experts architecture.

### Source excerpt

Акции NVIDIA рухнули, потеряв за одну ночь $600 миллиардов рыночной капитализации -- крупнейший дневной обвал на рынке ценных бумаг в истории США. Это событие стало шоком для рынка GPU, будущее которого, как казалось, предопределено на годы вперед. Причиной стал запуск китайской нейросети DeepSeek, которая не только превзошла западных конкурентов, таких как ChatGPT, но и стоила в разработке в тысячу раз дешевле. Этот прорыв поставил под угрозу бизнес-модель NVIDIA, основанную на стабильном спросе на их GPU. Разбираемся, как так вышло, вместе с Валерой Ковальским, CEO NDT by red_mad_robot. Читать далее