# Data-Driven Exploration for a Class of Continuous-Time Indefinite Linear--Quadratic Reinforcement Learning Problems

DevFeed: [Data-Driven Exploration for a Class of Continuous-Time Indefinite Linear--Quadratic Reinforcement Learning Problems](<https://devfeed.tech/articles/data-driven-exploration-for-a-class-of-continuous-time-indefinite-linear-quadratic-reinforcement-learning-problems-138718.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2507.00358>)

Author: Yilie Huang, Xun Yu Zhou

Published: 2026-11-17T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Reinforcement learning](<https://devfeed.tech/topics/reinforcement-learning.md>), [Machine learning](<https://devfeed.tech/topics/machine-learning.md>)

Tags: [continuous](<https://devfeed.tech/tags/continuous.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [cs-sy](<https://devfeed.tech/tags/cs-sy.md>), [data](<https://devfeed.tech/tags/data.md>), [data-driven](<https://devfeed.tech/tags/data-driven.md>), [eess-sy](<https://devfeed.tech/tags/eess-sy.md>), [exploration](<https://devfeed.tech/tags/exploration.md>), [learning](<https://devfeed.tech/tags/learning.md>), [linear](<https://devfeed.tech/tags/linear.md>), [math-oc](<https://devfeed.tech/tags/math-oc.md>), [problems](<https://devfeed.tech/tags/problems.md>), [reinforcement-learning](<https://devfeed.tech/tags/reinforcement-learning.md>), [time](<https://devfeed.tech/tags/time.md>)

## AI overview

For scalar-state continuous-time stochastic LQ control with state- and control-dependent volatility and no running control rewards, the paper proposes model-free exploration that adapts entropy regularization through the critic and policy variance through the actor. The authors say this responds to fixed or deterministic schedules that require tuning and do not account for learning progress. They report a sublinear regret bound matching the best-known model-free result for this problem class, previously obtained only with fixed schedules. Numerical experiments are reported to show faster convergence and better regret performance than non-adaptive model-free and model-based counterparts. The supplied text is an abstract, so it does not provide implementation details or experimental specifics.

## Source excerpt

arXiv:2507.00358v3 Announce Type: replace Abstract: We study reinforcement learning (RL) for a class of continuous-time stochastic linear-quadratic (LQ) control problems, where volatilities depend on both states and controls while states are scalar-valued and there are no running control rewards, the last feature also leading to the so-called indefinite LQ controls. We propose a model-free, data-driven exploration mechanism that adaptively adjusts entropy regularization by the critic and policy variance by the actor. Unlike the constant or deterministic exploration schedules employed in the existing literature, which require extensive tuning for implementations and ignore learning progress during iterations, our adaptive exploratory approach boosts learning efficiency with minimal tuning. Despite its flexibility, our method achieves a sublinear regret bound that matches the best-known model-free results for this class of LQ problems, which were previously derived only with fixed exploration schedules. Numerical experiments demonstrate that adaptive explorations accelerate convergence and improve regret performance compared to the non-adaptive model-free and model-based counterparts.