Efficient Solution of Portfolio Optimization Problems via Dimension Reduction and Sparsification

ArXiv ID: 2306.12639 “View on arXiv”

Authors: Unknown

Abstract

The Markowitz mean-variance portfolio optimization model aims to balance expected return and risk when investing. However, there is a significant limitation when solving large portfolio optimization problems efficiently: the large and dense covariance matrix. Since portfolio performance can be potentially improved by considering a wider range of investments, it is imperative to be able to solve large portfolio optimization problems efficiently, typically in microseconds. We propose dimension reduction and increased sparsity as remedies for the covariance matrix. The size reduction is based on predictions from machine learning techniques and the solution to a linear programming problem. We find that using the efficient frontier from the linear formulation is much better at predicting the assets on the Markowitz efficient frontier, compared to the predictions from neural networks. Reducing the covariance matrix based on these predictions decreases both runtime and total iterations. We also present a technique to sparsify the covariance matrix such that it preserves positive semi-definiteness, which improves runtime per iteration. The methods we discuss all achieved similar portfolio expected risk and return as we would obtain from a full dense covariance matrix but with improved optimizer performance.

Keywords: Markowitz portfolio optimization, covariance matrix reduction, machine learning predictions, sparsity, linear programming

Complexity vs Empirical Score

• Math Complexity: 7.0/10
• Empirical Rigor: 6.0/10
• Quadrant: Holy Grail
• Why: The paper is mathematically dense, featuring formal optimization formulations and linear algebra concepts like positive semi-definiteness. It also demonstrates high empirical rigor by testing on real financial data (S&P 500) with specific runtime improvements, though it lacks the full backtest/trading implementation details of a ‘Street Trader’.

  flowchart TD
    A["Research Goal:<br>Efficiently solve large-scale<br>portfolio optimization problems<br>overcoming dense covariance matrix issues"] --> B["Key Methodology:<br>Dimension Reduction & Sparsification<br>using ML predictions & LP solutions"]

    B --> C["Input Data:<br>Portfolio assets data<br>Expected returns &<br>full covariance matrix"]

    C --> D["Core Process:<br>Predict asset importance<br>via Linear Programming<br>& Neural Networks"]

    D --> E["Covariance Matrix<br>Transformation:<br>1. Reduce size (select assets)<br>2. Sparsify (preserve PSD)"]

    E --> F["Compute Outcomes:<br>Risk (Variance) & Return<br>Optimization Runtime & Iterations"]

    F --> G["Key Findings:<br>LP predictions outperform Neural Networks<br>Reduced matrices yield similar risk/return<br>Runtime & iterations significantly improved"]