Paper: arXiv 2512.14991
Authors: Hanqing Jin, Renyuan Xu, Yanzhao Yang
Abstract
We study reinforcement learning for controlled diffusion processes with unbounded continuous state spaces, bounded continuous actions, and polynomially growing rewards: settings that arise naturally in finance, economics, and operations research. To overcome the challenges of continuous and high-dimensional domains, we introduce a model-based algorithm that adaptively partitions the joint state-action space. The algorithm maintains estimators of drift, volatility, and rewards within each partition, refining the discretization whenever estimation bias exceeds statistical confidence. This adaptive scheme balances exploration and approximation, enabling efficient learning in unbounded domains. Our analysis establishes regret bounds that depend on the problem horizon, state dimension, reward growth order, and a newly defined notion of zooming dimension tailored to unbounded diffusion processes. The bounds recover existing results for bounded settings as a special case, while extending theoretical guarantees to a broader class of diffusion-type problems. Finally, we validate the effectiveness of our approach through numerical experiments, including applications to high-dimensional problems such as multi-asset mean-variance portfolio selection.
Complexity vs Empirical Score
- Math Complexity: 8.5/10
- Empirical Rigor: 5.0/10
- Quadrant: Holy Grail — high math complexity, high empirical rigor
Why this score: The paper employs advanced stochastic calculus and high-dimensional adaptive partitioning theory with complex regret bound derivations, yet it includes numerical experiments (e.g., multi-asset portfolio selection) that demonstrate practical feasibility.
Research Flowchart
flowchart TD A["Research Goal:<br>RL for Stochastic Control<br>in Unbounded Diffusion Processes"] --> B["Methodology:<br>Model-Based Adaptive Partitioning"] B --> C["Input Data:<br>Unbounded Continuous States<br>Bounded Actions & Polynomial Rewards"] C --> D["Computational Process:<br>Estimate Drift, Volatility & Rewards<br>Refine Partitions on Bias Threshold"] D --> E["Outcome:<br>Regret Bounds with<br>Zooming Dimension &<br>Validation via Portfolio Selection"]
Evidence Card
What the authors claim, on what data, and who has checked it. Read from the paper's own text by a language model: every quote is verbatim, nothing was re-run. How to read this card
Proposes
The paper proposes a model‑based reinforcement learning algorithm that adaptively partitions the joint state‑action space, maintains drift, volatility and reward estimators within each partition, and selects actions via an upper confidence bound on the constructed Q‑function.
Source sentence
To overcome the challenges of continuous and high-dimensional domains, we introduce a model-based algorithm that adaptively partitions the joint state-action space. The algorithm maintains estimators of drift, volatility, and rewards within each partition, refining the discretization whenever estimation bias exceeds statistical confidence.
Assumes 6
- The drift µ_h, volatility σ_h and mean reward \bar R_h are locally Lipschitz continuous in state and action. model
Source sentence
Assume there exists constants ℓµ, ℓσ > 0, m ∈N and L0 > 0 such that for all h ∈[H −1], x1, x2 ∈RdS, and a1, a2 ∈A, it holds that: ∥µh(x1, a1) −µh(x2, a2)∥ ≤ ℓµ ∥x1 −x2∥+ ∥a1 −a2∥ , ∥σh(x1, a1) −σh(x1, a2)∥ ≤ ℓσ ∥x1 −x2∥+ ∥a1 −a2∥ , max h∈[H−1]{∥µh(0, 0)∥, ∥σh(0, 0)∥} ≤ L0.
- The reward distributions have sub‑Gaussian tail decay. statistical
Source sentence
In addition, assume that the reward distribution has sub-Gaussian tail decay, i.e., there exists a known constant θ > 0 such that ∀x ∈RdS, a ∈A, λ1 ∈R, and h ∈[H]: Erh∼Rh(x,a) h exp λ1(rh −¯Rh(x, a)) i ≤e θλ2 1 2 .
- The volatility matrices satisfy a uniform ellipticity condition (σ_h σ_h^⊤ ≻ λ I). model
Source sentence
In addition, assume the following elliptic condition, i.e., there exists a constant λ > 0 such that ∀x ∈RdS, a ∈A, and h ∈[H −1]: σh(x, a)σh(x, a)⊤≻λIdS.
- The initial state distribution has finite p‑th moment with p^2 larger than a specified polynomial in dimensions and reward order. data
Source sentence
Assume that there exists p ∈N with p2 > (m + 1)2(dS + dA + 2) + (m + 1)(2dS + 2m + 4), such that the initial state X1 = ξ of the diffusion process in (2.1) satisfies: Eξ∼Ξ[∥ξ∥p] < +∞, The assumption that p2 > (m+1)2(dS +dA +2)+(m+1)(2dS +2m+4) ensures that the initial distribution is well behaved.
Show 2 more
- The state transition follows a discretized controlled diffusion: X_{h+1} - X_h = µ_h Δ + σ_h B_h √Δ with Gaussian noise B_h. model
Source sentence
Mathematically, for h ∈[H −1], the state process follows: Xh+1 −Xh = µh(Xh, Ah)∆+ σh(Xh, Ah)Bh √ ∆, (2.1) X1 = ξ, where ∆> 0 is the time-increment between two consecutive time stamps, Bh are i.i.d. samples from the multi-variate Gaussian distribution N(0, IdS) and ξ is independently sampled from an initial distribution Ξ. Clearly, for non-degenerate σh(x, a), we have Th(·|x, a) = N µh(x, a)∆, Σh(x, a)∆ , where
- The state‑action space is initially partitioned into hypercubes of fixed diameter D > 0. other
Source sentence
We first partition the entire state-action space RdS ×A into (closed) hypercubes of fixed diameter D > 0.
Data
- synthetic high‑dimensional multi‑asset mean‑variance portfolio selection problems synthetic unknown
Source sentence
Finally, we validate the effectiveness of our approach through numerical experiments, including applications to high-dimensional problems such as multi-asset mean-variance portfolio selection.
Costs
Not applicable
Finds 3
- The algorithm achieves a regret bound of order \tilde O(H K^{1-p/2-(m+1)^2(z_{max,c}+2)^{- (m+1)(2d_S+2m+4)} p(p+m+1)(z_{max,c}+2)+p(2d_S+2m+4)}) where H is horizon, K episodes, p moments, m reward growth order, d_S state dimension and z_{max,c} worst‑case zoo theoretical
Source sentence
Mathematically, we show that the proposed algorithm achieves a regret of order ˜O(HK 1−p2−(m+1)2(zmax,c+2)−(m+1)(2dS +2m+4) p(p+m+1)(zmax,c+2)+p(2dS +2m+4) ), with H the horizon of the problem, K the number of episodes, p the highest bounded moments for the initial state distribution, m + 1 the order of reward polynomial growth, dS the dimension of state space and zmax,c the worst-case zooming dimension over the
- The regret bounds specialize to existing results for bounded settings, showing the method extends classical rates to unbounded diffusion processes. theoretical
Source sentence
The bounds recover existing results for bounded settings as a special case, while extending theoretical guarantees to a broader class of diffusion-type problems.
- Numerical experiments on high‑dimensional problems such as multi‑asset mean‑variance portfolio selection demonstrate the effectiveness of the adaptive partitioning approach. performance
Source sentence
Finally, we validate the effectiveness of our approach through numerical experiments, including applications to high-dimensional problems such as multi-asset mean-variance portfolio selection.
Checked by others
No other paper in the archive has been found to support or contradict these findings (compared with 8 related papers). Coverage is partial, so this is neither replication nor refutation.
Code
No repository by the authors found in the paper.
Extracted 2026-11-14 · same card as JSON · A language model chose these items: check the source sentence before relying on one.