Paper: arXiv 2509.12764

Authors: Yuming Ma

Abstract

Myopic optimization (MO) outperforms reinforcement learning (RL) in portfolio management: RL yields lower or negative returns, higher variance, larger costs, heavier CVaR, lower profitability, and greater model risk. We model execution/liquidation frictions with mark-to-market accounting. Using Malliavin calculus (Clark-Ocone/BEL), we derive policy gradients and risk shadow price, unifying HJB and KKT. This gives dual gap and convergence results: geometric MO vs. RL floors. We quantify phantom profit in RL via Malliavin policy-gradient contamination analysis and define a control-affects-dynamics (CAD) premium of RL indicating plausibly positive.

Complexity vs Empirical Score

  • Math Complexity: 9.5/10
  • Empirical Rigor: 4.0/10
  • Quadrant: Lab Rats — high math complexity, limited empirical testing

Why this score: The paper is intensely mathematical, featuring advanced Malliavin calculus, stochastic SDEs, HJB/KKT unification, and dual gap analysis, indicating high complexity. However, the empirical evidence is largely conceptual and comparative (citing other works’ backtest failures), lacking original datasets, code, or detailed implementation metrics, resulting in lower rigor.

Research Flowchart

  flowchart TD
  A["Research Goal:<br>Why RL loses money in<br>Portfolio Management vs. Myopic Optimization"] --> B

  subgraph B ["Methodology & Inputs"]
      direction LR
      B1["Model Execution/Liquidation<br>Frictions via Mark-to-Market"]
      B2["Malliavin Calculus<br>Policy Gradients & Risk Shadow Price"]
      B3["HJB & KKT Unification<br>Dual Gap Convergence"]
  end

  B --> C["Computational Process:<br>Myopic vs. RL Strategy Comparison"]

  subgraph D ["Key Findings / Outcomes"]
      D1["Myopic Optimization:<br>Superior Returns, Low Variance"]
      D2["Reinforcement Learning:<br>Negative Returns, High CVaR, Model Risk"]
      D3["Phantom Profit Analysis:<br>Control-Affects-Dynamics (CAD) Premium"]
  end

  C --> D

Evidence Card

What the authors claim, on what data, and who has checked it. Read from the paper's own text by a language model: every quote is verbatim, nothing was re-run. How to read this card

Proposes

The paper proposes that myopic optimization (MO), implemented as a sequence of one‑period convex programs with execution/liquidation frictions, is sufficient for dynamic portfolio management and unifies policy‑gradient reinforcement learning (RL) via Malliavin calculus to derive risk shadow prices a

Source sentence
Myopic optimization (MO) outperforms reinforcement learning (RL) in portfolio management: RL yields lower or negative returns, higher variance, larger costs, heavier CVaR, lower profitability, and greater model risk. This gives dual gap and convergence results: geometric MO vs. RL floors.

Assumes 6

  • Market dynamics are action‑independent (AIM) and follow the specified stochastic processes. market
    Source sentence
    1 Introduction In this article we consider dynamic portfolio optimization for the buyside and the sellside mainly focusing on action-independent market (AIM) dynamics and standard execution/liquidation frictions with cases in control-affects-dynamics (CAD) market condition attached. In this regime, we show that myopic op­ timization (MO)-a sequence of one-period convex programs solved online-is dynamically
  • Execution and liquidation frictions are modeled with mark‑to‑market accounting. execution
    Source sentence
    We model execution/liquidation frictions with mark-to-market accounting.
  • Proportional trading costs are positive ( > 0 ). execution
    Source sentence
    With proportional trading costs >. > 0, optimal policies exhibit no-trade wedges and partial adjustment; small-cost band width 5 ex >.½, welfare-loss ex >.i [25, 43, 44, 26, 82].
  • Control‑affects‑dynamics (CAD) market condition may be attached to the regime. model
    Source sentence
    1 Introduction In this article we consider dynamic portfolio optimization for the buyside and the sellside mainly focusing on action-independent market (AIM) dynamics and standard execution/liquidation frictions with cases in control-affects-dynamics (CAD) market condition attached.
Show 2 more
  • The objective functions are convex (or locally smooth) so that myopic convex programs are well‑posed. model
    Source sentence
    Myopic optimization (MO) treats dynamic programs as a sequence of one­ period convex programs solved at each time t, whereas the RL objective embeds each action's influence on future states and conducts long-horizon credit assign­ ment via a Bellman recursion or cross-time gradient propagation. In frictionless settings, myopic optimality holds under isoelastic preferences [73, 60, 59].
  • Convergence analysis relies on smoothness and Polyak‑Łojasiewicz (PL) curvature of the risk functional. statistical
    Source sentence
    Convergence statements rely on smoothness and PL-type curvature; ill­ conditioned or sharply nonconvex objectives weaken constants.

Data

No dataset named in the text.

Costs

Not applicable The paper reports no trading or execution result; the cost the extractor found is a model parameter.

The paper-text extraction found a cost parameter inside a model (Standard execution/liquidation frictions with proportional costs are incorporated via mark‑to‑market accounting.); the classifier found no reported trading result, so costs are not applicable.

Finds 5

  • Myopic optimization outperforms reinforcement learning: RL yields lower or negative returns, higher variance, larger costs, heavier CVaR, lower profitability, and greater model risk. performance
    Source sentence
    Myopic optimization (MO) outperforms reinforcement learning (RL) in portfolio management: RL yields lower or negative returns, higher variance, larger costs, heavier CVaR, lower profitability, and greater model risk.
  • Geometric convergence of MO is established, while RL exhibits variance floors that limit convergence. theoretical
    Source sentence
    This gives dual gap and convergence results: geometric MO vs. RL floors. Viewed as op­ timization flows, myopic schemes contract geometrically under PL geometry, while policy-gradient RL carries non-vanishing variance floors that stall con­ vergence unless variance is aggressively reduced [69, 9].
  • A Malliavin policy‑gradient contamination analysis quantifies phantom profit in RL and defines a CAD premium that can be plausibly positive. theoretical
    Source sentence
    We quantify phantom profit in RL via Malliavin policy-gradient contamination analysis and define a control-affects-dynamics (CAD) premium of RL indicating plausibly positive.
  • Empirically, in most liquid AIM‑like environments RL adds at most 0‑1 bps per decision day; in rare CAD‑dominated niches it may reach low single‑digit bps per day. performance
    Source sentence
    Nevertheless, empirically, in most liq­ uid AIM-like environments this nets out to 0-1 bps per decision day; in rare, 33 strongly CAD-dominated niches it can reach low single-digit bps/day, but those cases require carefully validated state feedback and tight leakage controls in RL solvers.
  • When market dynamics are AIM and portfolios are valued by prudent MtM convention, treating taker and maker execution as convex programs makes MO sufficient in practice. theoretical
    Source sentence
    3 Conclusion and Discussion Across both taker and maker settings, when market dynamics are AIM and port­ folios are valuated by prudent MtM convention with spreads, impact, funding, and block discounts, by treating taker and maker execution as a sequence of con­ vex programs solved on decision ticks, MO is sufficient in practice.

Checked by others

Supported by

  • Benchmarking Robustness of Deep Reinforcement Learning approaches to Online Portfolio Management (2023)
    On the question: Do reinforcement learning portfolio management strategies provide robust and profitable performance?
    Compare the findings

    This paper: Myopic optimization outperforms reinforcement learning: RL yields lower or negative returns, higher variance, larger costs, heavier CVaR, lower profitability, and greater model risk.

    That paper: Most deep reinforcement learning algorithms were not robust, with strategies generalizing poorly and degrading quickly during backtesting.

    Both findings agree that RL methods perform poorly: A reports lack of robustness and quick degradation, while B shows lower/negative returns, higher variance, costs, CVaR, and model risk.

Code

No repository by the authors found in the paper.

Extracted 2026-11-14 · same card as JSON · A language model chose these items: check the source sentence before relying on one.