Paper: arXiv 2501.06275
Authors: Sebastien Lleo, Wolfgang Runggaldier
Abstract
We investigate exploratory randomization for an extended linear-exponential-quadratic-Gaussian (LEQG) control problem in discrete time. This extended control problem is related to the structure of risk-sensitive investment management applications. We introduce exploration through a randomization of the control. Next, we apply the duality between free energy and relative entropy to reduce the LEQG problem to an equivalent risk-neutral LQG control problem with an entropy regularization term, see, e.g. Dai Pra et al. (1996), for which we present a solution approach based on Dynamic Programming. Our approach, based on the energy-entropy duality may also be considered as leading to a justification for the use, in the literature, of an entropy regularization when applying a randomized control.
Complexity vs Empirical Score
- Math Complexity: 9.0/10
- Empirical Rigor: 3.0/10
- Quadrant: Lab Rats — high math complexity, limited empirical testing
Why this score: The paper is highly theoretical, featuring advanced stochastic control theory, duality theorems, and recursive Dynamic Programming solutions without any empirical validation or data. It presents a mathematical framework for exploration in control but lacks backtesting or implementation details.
Research Flowchart
flowchart TD A["Research Goal<br>Explore randomized control for discrete-time<br>Linear-Exponential-Quadratic-Gaussian (LEQG) problem"] --> B["Key Methodology<br>Introduce exploratory randomization<br>Apply free energy & relative entropy duality"] B --> C["Mathematical Transformation<br>Reduce LEQG problem to equivalent<br>risk-neutral LQG with entropy regularization"] C --> D["Computation<br>Solve via Dynamic Programming<br>obtain optimal state-feedback policy"] D --> E["Key Findings/Outcomes<br>1. New optimal control solution for LEQG<br>2. Theoretical justification for entropy regularization<br>3. Application to risk-sensitive investment management"]
Evidence Card
What the authors claim, on what data, and who has checked it. Read from the paper's own text by a language model: every quote is verbatim, nothing was re-run. How to read this card
Proposes
The paper proposes to introduce exploratory randomization via additive Gaussian perturbations of deterministic controls and to reduce the discrete-time risk‑sensitive LEQG problem to an equivalent risk‑neutral LQG stochastic game with an entropy regularization term using the free‑energy/relative‑ent
Source sentence
Next, we apply the duality between free energy and relative entropy to reduce the LEQG problem to an equivalent risk-neutral LQG control problem with an entropy regularization term, see, e.g. Dai Pra et al. (1996), for which we present a solution approach based on Dynamic Programming. Our main contribution is to solve the randomized LEQG problem using a duality between free energy and entropy Dai Pra et al. (1996).
Assumes 6
- Controls are randomized by adding an independent Gaussian noise with mean equal to the deterministic control and a variance that models exploration. model
Source sentence
Here, the mean is the deter-ministic control that captures exploitation, while the variance, which provides an additional (antagonizing) control, models exploration.
- The state noise is i.i.d. Gaussian with possibly time‑varying mean and covariance. model
Source sentence
Gaussian with mean ¯γt, which may depend on time, and covariance Λt, namely w¯γ t ∼N (¯γt, Λt).
- The randomized control noise is i.i.d. Gaussian with mean \(\bar\eta_t\) and covariance \(\Xi_t\). model
Source sentence
Gaussian with mean ¯ηt and covariance Ξt.
- The problem is set in discrete time, which avoids the technical challenges of continuous‑time additive randomization. model
Source sentence
While implementing additive randomization in continuous-time settings poses significant technical challenges1, the discrete-time framework allows us to focus more directly on the exploration aspect.
Show 2 more
- A relative‑entropy penalty is applied to the risk‑neutral reformulation, arising from both the randomized controls and the risk‑sensitive to risk‑neutral transformation. statistical
Source sentence
This penalization comes from two sources: (i) the randomized controls and (ii) the transition from risk-sensitive to risk-neutral. This risk-neutral randomized stochastic control problem is penalized by an appropriate relative entropy term and formulated as a stochastic game.
- Assumption 3.2 (positivity of certain matrices) holds, ensuring a unique saddle point for the LQG game. model
Source sentence
As B(1) t > 0, t = 1, . . . , T, Assumption 3.2(i) is satisfied. To check this assumption, we use Sylvester’s criterion.
Data
No dataset named in the text.
Costs
Not applicable
Finds 3
- The optimal policy and value function are obtained in closed form and are parametrized by the model’s initial data via a backward recursion (Theorem 3.5). theoretical
Source sentence
Both turn out to be parametrized by the initial data of the model via a backward recursion, as stated in Theorem 3.5. We can now state our main Theorem 3.5.
- The randomized LEQG problem admits a unique saddle point at \((u^*,\gamma^*,\eta^*)\) under the stated positivity conditions. theoretical
Source sentence
Nevertheless, we will see below that Assumption 3.2 is satisfied, and so, by Proposition 3.7, the game has a unique saddle point at (u∗, γ∗, η∗). To check this assumption, we use Sylvester’s criterion.
- The reduction to a risk‑neutral LQG stochastic game with entropy penalization is established through the energy‑entropy duality. theoretical
Source sentence
risk-sensitive LEQG problem, set with re-spect to an initial measure P, a risk-neutral randomized stochastic control problem, set with respect to a transformed measure. This risk-neutral randomized stochastic control problem is penalized by an appropriate relative entropy term and formulated as a stochastic game.
Checked by others
No other paper in the archive has been found to support or contradict these findings (compared with 10 related papers). Coverage is partial, so this is neither replication nor refutation.
1 paper in the archive
Code
No repository by the authors found in the paper.
Extracted 2026-11-14 · same card as JSON · A language model chose these items: check the source sentence before relying on one.