# Near Minimax Rate for Quantile Distributional Reinforcement Learning

DevFeed: [Near Minimax Rate for Quantile Distributional Reinforcement Learning](<https://devfeed.tech/articles/near-minimax-rate-for-quantile-distributional-reinforcement-learning-94961.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2610.29595>)

Author: Zijie Cheng, Zhihua Zhang

Published: 2026-10-22T04:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Reinforcement learning](<https://devfeed.tech/topics/reinforcement-learning.md>), [Machine learning](<https://devfeed.tech/topics/machine-learning.md>)

Tags: [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [learning](<https://devfeed.tech/tags/learning.md>), [minimax](<https://devfeed.tech/tags/minimax.md>), [quantile](<https://devfeed.tech/tags/quantile.md>), [reinforcement-learning](<https://devfeed.tech/tags/reinforcement-learning.md>), [stat-ml](<https://devfeed.tech/tags/stat-ml.md>)

## AI overview

Assuming access to a generative model, the paper estimates the quantile Bellman fixed point from an empirical Markov decision process and proves non-asymptotic Wasserstein error bounds. With reward densities bounded away from zero, it obtains a root-n high-probability rate uniformly over quantile resolutions and Wasserstein orders from 1 to infinity. If densities vanish at support endpoints, continuity and endpoint monotonicity retain that resolution-independent rate for orders up to 2; for p>2, the root-n bound has an m^(1/2−1/p) coefficient, alongside a resolution-independent n^(−1/p) bound for finite p. It also gives resolution-independent W-infinity consistency governed by reward-quantile continuity. At each fixed resolution, the estimator attains the characterized local asymptotic minimax risk. That risk can be resolution-independent in a regular one-state case, but grows with resolution in an endpoint-vanishing two-state example for p>2.

## Source excerpt

arXiv:2610.29595v1 Announce Type: cross Abstract: In this paper, we study quantile distributional policy evaluation, where the return distribution under a given policy is approximated by the quantile Bellman fixed point $\bm\eta_m$. We investigate the statistical accuracy of estimating $\bm\eta_m$ across quantile resolutions under Wasserstein distances. Assuming access to a generative model, we construct a quantile fixed point estimator $\bm\eta_m^{(n)}$ from an empirical Markov decision process and establish non-asymptotic error bounds under Wasserstein distances. When reward densities are uniformly bounded away from zero, we obtain a root-$n$ high-probability bound simultaneously over all resolutions $m$ and all Wasserstein orders $1\le p\le\infty$. When reward densities vanish at the support endpoints, we show that continuity and endpoint monotonicity suffice to preserve the same resolution-independent rate for $1\le p\le2$. For $p>2$, we establish a root-$n$ bound with coefficient $m^{1/2-1/p}$, together with a slower, resolution-independent $n^{-1/p}$ bound for every finite $p>2$. We further obtain resolution-independent $W_\infty$ consistency governed by the continuity modulus of reward quantile functions. To investigate the sharpness of these bounds, we characterize the exact local asymptotic minimax risk at each fixed resolution as the expected Wasserstein-coordinate norm of the efficient Gaussian limit and prove that our estimator attains it. We then examine the resolution dependence of this optimal local risk. A regular one-state instance has resolution-independent local risk constants, whereas a single endpoint-vanishing two-state instance exhibits growth of order $m^{1/2-1/p}$, up to logarithmic factors along a resolution subsequence, for every $p>2$.