Paper: arXiv 2602.12030

Authors: Federico Cacciamani, Roberto Daluiso, Marco Pinciroli, Michele Trapletti, Edoardo Vittori

Abstract

In finance, sequential decision problems are often faced, for which reinforcement learning (RL) emerges as a promising tool for optimisation without the need of analytical tractability. However, the objective of classical RL is the expected cumulated reward, while financial applications typically require a trade-off between return and risk. In this work, we focus on settings where one cares about the time split of the total return, ruling out most risk-aware generalisations of RL which optimise a risk measure defined on the latter. We notice that a preference for homogeneous splits, which we found satisfactory for hedging, can be unfit for other problems, and therefore propose a new risk metric which still penalises uncertainty of the single rewards, but allows for an arbitrary planning of their target levels. We study the properties of the resulting objective and the generalisation of learning algorithms to optimise it. Finally, we show numerical results on toy examples.

Complexity vs Empirical Score

  • Math Complexity: 6.5/10
  • Empirical Rigor: 3.5/10
  • Quadrant: Lab Rats — theoretically deep, empirically untested

Why this score: The paper presents a novel and mathematically sound time-inhomogeneous reward volatility framework for risk-aware RL, bridging an important gap in financial planning objectives. However, empirical verification is limited to toy examples and lacks rigorous market backtesting or code availability.

Research Flowchart

  flowchart TD
    A["Research Question: How to optimize RL for time-split return and risk trade-offs in finance?"] --> B["Define Metric: Time-Inhomogeneous Volatility Aversion (penalizes single-reward uncertainty)"]
    B --> C["Inputs: Target reward trajectories, financial environment states, and historical/simulated data"]
    C --> D["Algorithmic Generalization: Modify RL algorithms (Q-learning/Policy Gradient) for the new objective"]
    D --> E["Numerical Experiments: Train and evaluate agent on financial toy examples & hedging tasks"]
    E --> F["Key Findings: Successfully balances risk-return while allowing arbitrary target planning across time"]