# DACA-GRPO: Denoising-Aware Credit Assignment for Reinforcement Learning in Diffusion Language Models

DevFeed: [DACA-GRPO: Denoising-Aware Credit Assignment for Reinforcement Learning in Diffusion Language Models](<https://devfeed.tech/articles/daca-grpo-denoising-aware-credit-assignment-for-reinforcement-learning-in-diffusion-language-models-31489.md>)

Original publisher: [Read original article](<https://machinelearning.apple.com/research/denoising-aware-credit-assignment>)

Published: 2026-09-16T00:00:00Z

Content type: article

Language: en

Sources: [Apple Machine Learning Research](<https://devfeed.tech/sources/apple-machine-learning-research.md>)

Topics: [grpo](<https://devfeed.tech/topics/grpo.md>), [Reinforcement learning](<https://devfeed.tech/topics/reinforcement-learning.md>), [large-language-models](<https://devfeed.tech/topics/large-language-models.md>), [LLM Techniques](<https://devfeed.tech/topics/llm-techniques.md>), [Code](<https://devfeed.tech/topics/code.md>)

Tags: [code-generation](<https://devfeed.tech/tags/code-generation.md>), [denoising](<https://devfeed.tech/tags/denoising.md>), [diffusion](<https://devfeed.tech/tags/diffusion.md>), [grpo](<https://devfeed.tech/tags/grpo.md>), [json-schema](<https://devfeed.tech/tags/json-schema.md>), [optimization](<https://devfeed.tech/tags/optimization.md>), [reinforcement-learning](<https://devfeed.tech/tags/reinforcement-learning.md>)

## AI overview

The article introduces DACA-GRPO, a denoising-aware enhancement for GRPO-style reinforcement-learning trainers used with diffusion language models. It uses per-token denoising progress scores and stratified masking likelihood to improve credit assignment and reduce bias in likelihood estimates. Across seven benchmarks, the method reports gains in mathematical reasoning, code generation, constraint satisfaction, and JSON Schema adherence.

## Source excerpt

Diffusion large language models are a compelling alternative to autoregressive models, yet existing RL methods for diffusion treat all denoising steps as equally important and rely on biased, high-variance likelihood estimates. We identify two fundamental weaknesses: the absence of temporal credit assignment across the denoising trajectory, and the systematic bias of mean-field likelihood estimates used for policy optimization. To address these, we propose Denoising-Aware Credit Assignment for GRPO (DACA-GRPO), a lightweight, plug-and-play enhancement for any GRPO-style trainer. DACA-GRPO...