MLP-Initialized Reinforcement Learning for Exit Timing in SPX Short-Straddle Portfolios

Paper: arXiv 2610.13126

Authors: Jianning Wang, Robert Ślepaczuk, Paweł Sakowski

Abstract

This paper studies exit timing for SPX short-straddle portfolios using a two-stage supervised and reinforcement-learning framework. Historical episodes first generate ex-post optimal hold/exit labels for training a multilayer perceptron (MLP) using only contemporaneously available state variables at deployment. The trained MLP then initializes reinforcement-learning actors that are fine-tuned with a risk-aware trading reward. Out-of-sample evaluation over 2024-2026 shows that the supervised MLP substantially outperforms passive holding and representative fixed-time and profit-and-loss-based exit rules. Across seven independent seeds, MLP-initialized Actor-Critic averages 37.54% cumulative return and a Sharpe ratio of 1.705, compared with 35.01% and 1.651 for the supervised MLP. Actor-Critic trained from scratch averages only 0.18% return. These findings show that oracle-guided supervised learning provides a stable foundation for reward-based policy fine-tuning in a sparse and path-dependent option-exit problem.

Complexity vs Empirical Score

• Math Complexity: 6.5/10
• Empirical Rigor: 8.0/10
• Quadrant: Holy Grail — high math complexity, high empirical rigor

Why this score: The paper demonstrates strong empirical rigor through extensive out-of-sample testing, multiple control policies, and detailed portfolio construction. Its novelty lies in the two-stage MLP-initialized RL approach for a challenging options exit timing problem. The mathematical complexity is moderate, combining supervised learning and reinforcement learning frameworks.

Research Flowchart

  flowchart TD
    A[Research Goal: Optimize SPX Short-Straddle Exit Timing] --> B{Data: Historical SPX Options & Market Data};
    B --> C[Stage 1: Supervised Learning (MLP)];
    C -- Ex-post optimal hold/exit labels --> D{Computational Process: Train MLP};
    D -- Initializes --> E[Stage 2: Reinforcement Learning (Actor-Critic)];
    E -- Fine-tuned with risk-aware trading reward --> F{Outcomes: Performance Evaluation};
    F --> G[Key Findings: MLP-Initialized RL Outperforms Others];
    G -- MLP-Initialized RL: 37.54% Return, Sharpe 1.705 --> H[Impact: Stable Foundation for Reward-Based Policy Fine-Tuning];