Paper: arXiv 2306.12639
Authors: Cassidy K. Buhler, Hande Y. Benson
Abstract
The Markowitz mean-variance portfolio optimization model aims to balance expected return and risk when investing. However, there is a significant limitation when solving large portfolio optimization problems efficiently: the large and dense covariance matrix. Since portfolio performance can be potentially improved by considering a wider range of investments, it is imperative to be able to solve large portfolio optimization problems efficiently, typically in microseconds. We propose dimension reduction and increased sparsity as remedies for the covariance matrix. The size reduction is based on predictions from machine learning techniques and the solution to a linear programming problem. We find that using the efficient frontier from the linear formulation is much better at predicting the assets on the Markowitz efficient frontier, compared to the predictions from neural networks. Reducing the covariance matrix based on these predictions decreases both runtime and total iterations. We also present a technique to sparsify the covariance matrix such that it preserves positive semi-definiteness, which improves runtime per iteration. The methods we discuss all achieved similar portfolio expected risk and return as we would obtain from a full dense covariance matrix but with improved optimizer performance.
Complexity vs Empirical Score
- Math Complexity: 7.0/10
- Empirical Rigor: 6.0/10
- Quadrant: Holy Grail — high math complexity, high empirical rigor
Why this score: The paper is mathematically dense, featuring formal optimization formulations and linear algebra concepts like positive semi-definiteness. It also demonstrates high empirical rigor by testing on real financial data (S&P 500) with specific runtime improvements, though it lacks the full backtest/trading implementation details of a ‘Street Trader’.
Research Flowchart
flowchart TD A["Research Goal:<br>Efficiently solve large-scale<br>portfolio optimization problems<br>overcoming dense covariance matrix issues"] --> B["Key Methodology:<br>Dimension Reduction & Sparsification<br>using ML predictions & LP solutions"] B --> C["Input Data:<br>Portfolio assets data<br>Expected returns &<br>full covariance matrix"] C --> D["Core Process:<br>Predict asset importance<br>via Linear Programming<br>& Neural Networks"] D --> E["Covariance Matrix<br>Transformation:<br>1. Reduce size (select assets)<br>2. Sparsify (preserve PSD)"] E --> F["Compute Outcomes:<br>Risk (Variance) & Return<br>Optimization Runtime & Iterations"] F --> G["Key Findings:<br>LP predictions outperform Neural Networks<br>Reduced matrices yield similar risk/return<br>Runtime & iterations significantly improved"]
Evidence Card
What the authors claim, on what data, and who has checked it. Read from the paper's own text by a language model: every quote is verbatim, nothing was re-run. How to read this card
Proposes
The paper proposes dimension reduction and increased sparsity of the covariance matrix to improve large‑scale portfolio optimization.
Source sentence
We propose dimension reduction and increased sparsity as remedies for the covariance matrix.
Assumes 4
- Predictive models (ML or LP) will sufficiently match the assets in the optimal Markowitz portfolios. model
Source sentence
Our hypothesis is that the predictive models will give a sufficient match to the investments in the optimal portfolios.
- Correlations close to zero can be set to zero while preserving positive semidefiniteness of the covariance matrix. model
Source sentence
Correlations close to zero can be replaced by zero, which yields a sparse matrix. Care must be taken to retain positive semidefiniteness.
- Large correlations and variances are sufficient to determine the optimal portfolios and represent their risk. model
Source sentence
The main hypothesis for this technique is that the variance and large correlations are sufficient to determine the optimal portfolios and represent their risk.
- Daily closing prices of S&P 500 constituents provide a representative dataset for testing the methods. data
Source sentence
Our financial data, which consists of daily closing prices for individual stocks in the S&P 500 index.
Data
- S&P 500 daily closing prices (374 stocks) market-prices unknown
Source sentence
The efficient frontier for 374 stocks selected from the S&P500 using the…
- S&P 500 daily closing prices (497 assets, 502 days) market-prices unknown
Source sentence
We repeated the above experiment with a shorter dataset that has more stocks: 497 assets for 502 days starting November 21 2017 to November 19th 2019.
Yahoo Finance
Costs
Not modeled
Finds 3
- Reducing the covariance matrix based on ML or LP predictions decreases both runtime and total iterations while preserving portfolio risk and return. performance
Source sentence
Reducing the covariance matrix based on these predictions decreases both runtime and total iterations.
- All three proposed methods achieve similar expected risk and return as the full dense covariance matrix but with improved optimizer performance. performance
Source sentence
The methods we discuss all achieved similar portfolio expected risk and return as we would obtain from a full dense covariance matrix but with improved optimizer performance.
- Sparsification by correlation reduces runtime per iteration while increasing total runtime per iteration, and higher sparsity leads to fewer optimizer iterations. performance
Source sentence
Reducing the size of the covariance matrix decreased the number of iterations and CPU run‑time, yet increased the runtime per iteration.
Checked by others
No other paper in the archive has been found to support or contradict these findings (compared with 18 related papers). Coverage is partial, so this is neither replication nor refutation.
Extracted 2026-11-14 · same card as JSON · A language model chose these items: check the source sentence before relying on one.