Return Prediction for Mean-Variance Portfolio Selection: How Decision-Focused Learning Shapes Forecasting Models

ArXiv ID: 2409.09684 “View on arXiv”

Authors: Unknown

Abstract

Markowitz laid the foundation of portfolio theory through the mean-variance optimization (MVO) framework. However, the effectiveness of MVO is contingent on the precise estimation of expected returns, variances, and covariances of asset returns, which are typically uncertain. Machine learning models are becoming useful in estimating uncertain parameters, and such models are trained to minimize prediction errors, such as mean squared errors (MSE), which treat prediction errors uniformly across assets. Recent studies have pointed out that this approach would lead to suboptimal decisions and proposed Decision-Focused Learning (DFL) as a solution, integrating prediction and optimization to improve decision-making outcomes. While studies have shown DFL’s potential to enhance portfolio performance, the detailed mechanisms of how DFL modifies prediction models for MVO remain unexplored. This study investigates how DFL adjusts stock return prediction models to optimize decisions in MVO. Theoretically, we show that DFL’s gradient can be interpreted as tilting the MSE-based prediction errors by the inverse covariance matrix, effectively incorporating inter-asset correlations into the learning process, while MSE treats each asset’s error independently. This tilting mechanism leads to systematic prediction biases where DFL overestimates returns for assets included in portfolios while underestimating excluded assets. Our findings reveal why DFL achieves superior portfolio performance despite higher prediction errors. The strategic biases are features, not flaws.

Keywords: decision-focused learning, mean-variance optimization, gradient tilting, inverse covariance matrix, machine learning, Equities

Complexity vs Empirical Score

• Math Complexity: 8.5/10
• Empirical Rigor: 4.0/10
• Quadrant: Lab Rats
• Why: The paper is mathematically dense with detailed derivations of gradients, inverse covariance matrices, and optimization theory, placing it in the high math category. While it presents empirical results (like Table 1), the summary suggests a focus on theoretical mechanisms rather than extensive backtesting or implementation-heavy datasets, aligning with lab-oriented research.

  flowchart TD
    A["Research Goal<br>How does DFL modify prediction models<br>to optimize MVO decisions?"] --> B["Data & Inputs<br>Asset return data for MVO<br>ML model for return prediction"]
    B --> C["Methodology<br>Compare: MSE-based (baseline) vs.<br>Decision-Focused Learning (DFL)"]
    C --> D["Computational Process<br>Backpropagation: Theoretical analysis<br>of gradients via chain rule"]
    D --> E["Key Finding 1: Gradient Mechanism<br>MSE Error ∝ Inverse Covariance<br>(Incorrelates inter-asset correlation)"]
    D --> F["Key Finding 2: Strategic Biasing<br>DFL systematically overestimates<br>returns for included assets"]
    E --> G["Outcome<br>DFL yields superior portfolio performance<br>despite higher MSE prediction errors"]
    F --> G
    style A fill:#e1f5fe
    style G fill:#f1f8e9