Paper: arXiv 2507.07159
Authors: Nishan Ranabhat, Behnam Javanparast, David Goerz, Estelle Inack
Abstract
Portfolio optimization is a routine asset management operation conducted in financial institutions around the world. However, under real-world constraints such as turnover limits and transaction costs, its formulation becomes a mixed-integer nonlinear program that current mixed-integer optimizers often struggle to solve. We propose mapping this problem onto a classical Ising-like Hamiltonian and solving it with Variational Neural Annealing (VNA), via its classical formulation implemented using autoregressive neural networks. We demonstrate that VNA can identify near-optimal solutions for portfolios comprising more than 2,000 assets and yields performance comparable to that of state-of-the-art optimizers, such as Mosek, while exhibiting faster convergence on hard instances. Finally, we present a dynamical finite-size scaling analysis applied to the S&P 500, Russell 1000, and Russell 3000 indices, revealing universal behavior and polynomial annealing time scaling of the VNA algorithm on portfolio optimization problems.
Complexity vs Empirical Score
- Math Complexity: 8.5/10
- Empirical Rigor: 8.0/10
- Quadrant: Holy Grail — high math complexity, high empirical rigor
Why this score: The paper introduces high-level mathematical concepts such as Ising Hamiltonian mapping, variational neural annealing, and dynamical finite-size scaling, demonstrating substantial theoretical depth. It validates its approach with large-scale backtests on major indices (S&P 500, Russell 1000/3000) and benchmarks against commercial solvers like Mosek, showing strong empirical implementation and performance analysis.
Research Flowchart
flowchart TD
A["Research Goal<br>Optimize portfolios with constraints<br>over 2,000 assets"] --> B["Methodology<br>Variational Neural Annealing<br>Autoregressive Neural Networks"]
B --> C["Formulation<br>Map to Ising-like Hamiltonian<br>Mixed-Integer Nonlinear Program"]
C --> D["Data Sources<br>S&P 500, Russell 1000,<br>Russell 3000 Indices"]
D --> E["Computational Process<br>Dynamical Finite-Size Scaling Analysis"]
E --> F{"Findings"}
F --> G["Near-optimal solutions<br>Comparable to Mosek<br>Faster on hard instances"]
F --> H["Scaling Laws<br>Universal behavior revealed<br>Polynomial annealing time"]
Evidence Card
What the authors claim, on what data, and who has checked it. Read from the paper's own text by a language model: every quote is verbatim, nothing was re-run. How to read this card
Proposes
The paper proposes mapping large-scale portfolio optimization onto a classical Ising-like Hamiltonian and solving it with Variational Neural Annealing (VNA) using autoregressive neural networks.
Source sentence
We propose mapping this problem onto a classical Ising-like Hamiltonian and solving it with Variational Neural Annealing (VNA), via its classical formulation implemented using autoregressive neural networks.
Assumes 6
- The covariance matrix C is positive semi‑definite, making the continuous‑valued objective convex. model
Source sentence
The covariance matrix C is positive semi-definite by construction, and the objective function is therefore convex.
- Asset returns are estimated from monthly closing prices using logarithmic returns over a five‑year look‑back window. data
Source sentence
Closing prices with monthly granularity are used to compute expected returns and risk terms. µ is computed as a logarithm returns, since it is time-additive, and the covariance matrix C is built with a five-year look-back window.
- Only assets that are present throughout the entire ten‑year trading period are included. execution
Source sentence
Only assets that are consistently present during the entire trading period are considered.
- Transaction costs consist of a variable cost proportional to trade volume and a fixed cost proportional to the number of trades. market
Source sentence
The transaction cost can be of two types: the first is variable cost, which is proportional to the volume of the trade, and the second is fixed cost, which is proportional to the number of trades.
Show 2 more
- The VNA algorithm’s runtime scales linearly with the number of assets per gradient step (O(N)). statistical
Source sentence
Other factors include the computational complexity of VNA, which is of O(N) for each gradient descent step when implemented with a RNN.
- The normalization penalty term can be adjusted to incorporate transaction costs, assuming the penalty parameters can be made arbitrarily large. model
Source sentence
the normalization penalty term is altered to include the transaction costs, ρn(1T x + Costv + Costf −1)2.
Data
- S&P 500 historical prices market-prices public
Source sentence
Our dataset is made up of historical prices over ten years of the S&P 500, Russell 1000 and Russell 3000 indices.
- Russell 1000 historical prices market-prices public
Source sentence
Our dataset is made up of historical prices over ten years of the S&P 500, Russell 1000 and Russell 3000 indices.
- Russell 3000 historical prices market-prices public
Source sentence
Our dataset is made up of historical prices over ten years of the S&P 500, Russell 1000 and Russell 3000 indices.
Costs
Modeled
Finds 5
- VNA identifies near‑optimal solutions for portfolios with more than 2,000 assets and achieves performance comparable to state‑of‑the‑art mixed‑integer solvers such as Mosek. performance
Source sentence
We demonstrate that VNA can identify near-optimal solutions for portfolios comprising more than 2,000 assets and yields performance comparable to that of state-of-the-art optimizers, such as Mosek, while exhibiting faster convergence on hard instances.
- On hard instances VNA reaches a relative gap of 0.01 % in roughly half the time required by Mosek, even when Mosek’s stopping criterion is relaxed to 1 %. performance
Source sentence
Increasing the relative gap requirement by two orders of magnitude still resulted in Mosek having twice the time-to-solution as VNA.
- The average Sharpe ratios obtained by VNA are statistically indistinguishable from those obtained by Mosek across S&P 500, Russell 1000 and Russell 3000 datasets. performance
Source sentence
The Sharpe ratio difference is minimal, indicating that VNA can yield comparable results to Mosek across different trading periods, sometimes with slightly better results.
- Finite‑size scaling analysis reveals universal polynomial annealing‑time scaling with critical exponents µ = −0.58 ± 0.08 and κ = 0.16 ± 0.09. theoretical
Source sentence
The critical exponents extracted from this collapse are µ = −0.58 ± 0.08 and κ = 0.16 ± 0.09.
- VNA’s runtime per gradient step scales linearly with the number of assets (O(N)), enabling scalability to thousands of assets. performance
Source sentence
Other factors include the computational complexity of VNA, which is of O(N) for each gradient descent step when implemented with a RNN.
Checked by others
No other paper in the archive has been found to support or contradict these findings (compared with 42 related papers). Coverage is partial, so this is neither replication nor refutation.
Code
No repository by the authors found in the paper.
Extracted 2026-11-14 · same card as JSON · A language model chose these items: check the source sentence before relying on one.
Cited References (99)
- Spin glass theory and beyond: an introduction to the replica method and its applications (1987)
- A measure of data collapse for scaling (2001) — link
- Solvable model of a spin-glass (1975) — link
- Spin glasses: a computational challenge for the 21st century (2002) — link
- Rational decisions, random matrices and spin glasses (1998) — link
- Portfolio selection (1952) — link
- Tax-Aware Portfolio Construction via Convex Optimization (2021) — link
- Ising formulations of many NP problems (2014) — link
and 91 more
- On the computational complexity of Ising spin glass models (1982) — link
- Computational study of a family of mixed-integer quadratic programming problems (1996)
- Mixed-integer nonlinear optimization (2013)
- Structure and binding of Lennard–Jones clusters: $13\leq N\leq 147$ (1987) — link
- Thermodynamics and the global optimization of Lennard-Jones clusters (1998) — link
- Optimization in computational systems biology (2008) — link
- Optimization Algorithms for Computational Systems Biology (2017) — link
- Application of combinatorial optimization strategies in synthetic biology (2020) — link
- A survey of job shop scheduling problem: The types and models (2022) — link
- Healthcare scheduling in optimization context: a review (2021) — link
- The flexible job shop scheduling problem: A review (2024) — link
- Optimization by simulated annealing (1983) — link
- A Survey of simulated annealing applications to operations research problems (1994) — link
- A tutorial survey of theory and applications of simulated annealing (1985) — link
- Optimization by simulated annealing: an experimental evaluation; Part II, graph coloring and number partitioning (1991) — link
- Robust design of multimachine power system stabilizers using simulated annealing (2000) — link
- Strategic portfolio optimization using simulated, digital, and quantum annealing (2022) — link
- Universal nonequilibrium quantum dynamics in imaginary time (2011) — link
- Dynamic scaling at classical phase transitions approached through nonequilibrium quenching (2014) — link
- A quantum approximate optimization algorithm (2014) — link
- A variational eigenvalue solver on a photonic quantum processor (2014) — link
- A review on Quantum Approximate Optimization Algorithm and its variants (2024) — link
- Challenges and opportunities in quantum optimization (2024) — link
- Branch-and-bound digitized counterdiabatic quantum optimization (2025) — link
- From portfolio optimization to quantum blockchain and security: A systematic review of quantum computing in finance — → 2307.01155
- Quantum computational finance: quantum algorithm for portfolio optimization (2024) — link
- Wasserstein solution quality and the Quantum Approximate Optimization Algorithm: a portfolio optimization case study (2022) — link
- Benchmarking the performance of portfolio optimization with QAOA (2022) — link
- Best practices for portfolio optimization by quantum computing, experimented on real quantum devices (2023) — link
- Quantum walk-based portfolio optimisation (2021) — link
- Quantum computing for finance: overview and prospects (2019) — link
- Quantum computing for finance: state-of-the-art and future prospects (2020) — link
- Quantum computing for finance (2023) — link
- Reverse quantum annealing approach to portfolio optimization problems (2019) — link
- Quantum annealing with manufactured spins (2011) — link
- A coherent quantum annealer with Rydberg atoms (2017) — link
- Solving the optimal trading trajectory problem using a quantum annealer (2015) — link
- Quantum portfolio optimization with investment bands and target volatility (2021) — link
- Portfolio optimization of 60 stocks using classical and quantum algorithms (2020) — link
- Quantum critical dynamics in a 5,000-qubit programmable spin glass (2023) — link
- Scaling advantage in approximate optimization with quantum annealing (2025) — link
- Pushing the boundary of quantum advantage in hard combinatorial optimization with probabilistic computers (2025) — link
- Supplementing recurrent neural networks with annealing to solve combinatorial optimization problems (2023) — link
- Lattice protein folding with variational annealing (2025) — link
- Sparse autoregressive neural networks for classical spin systems (2024) — link
- Message passing variational autoregressive network for solving intractable Ising models (2024) — link
- A method for quantifying the generalization capabilities of generative models for solving Ising models (2024)
- Efficient optimization of variational autoregressive networks with natural gradient (2025) — link
- Variational neural annealing (2021) — link
- Enhancing combinatorial optimization with classical and quantum generative models (2024) — link
- Solving statistical mechanics using variational autoregressive networks (2019) — link
- Recurrent neural network wave functions (2020) — link
- Iterative retraining of quantum spin models using recurrent neural networks (2020) — link
- The density-matrix renormalization group in the age of matrix product states (2011) — link
- Quantum-inspired meta-heuristic approaches for a constrained portfolio optimization problem (2024) — link
- Dynamic portfolio optimization with real datasets using quantum processors and quantum-inspired tensor networks (2022) — link
- Decomposition pipeline for large-scale portfolio optimization with applications to near-term quantum computing — → 2409.10301
- Ising-model optimizer with parallel-trial bit-sieve engine (2018)
- Physics-inspired optimization for quadratic unconstrained problems using a digital annealer (2019) — link
- Solving the optimal trading trajectory problem using simulated bifurcation (2020) — link
- Real-time trading system based on selections of potentially profitable, uncorrelated, and balanced stocks by NP-hard combinatorial optimization — → 2307.06339
- Index tracking using Shapley additive explanations and one-dimensional pointwise convolutional autoencoders (2024) — link
- Deep learning for portfolio optimization (2020) — link
- StockGPT: a GenAI model for stock prediction and trading — → 2404.05101
- MILLION: a general multi-objective framework with controllable risk for portfolio management — → 2412.03038
- Graph neural networks for deep portfolio optimization (2023) — link
- Neural networks for portfolio analysis with cardinality constraints (2024) — link
- Hybrid data-driven and deep learning based portfolio optimization (2024) — link
- l1-regularization in portfolio selection with machine learning (2022) — link
- Enhancing Markowitz’s portfolio selection paradigm with machine learning (2024) — link
- End-to-End learning for stochastic optimization: a bayesian perspective (2023)
- E2EAI: End-to-End deep learning framework for active investing — → 2305.16364
- Generative AI for End-to-End limit order book modelling: a token-level autoregressive generative model of message flow using a deep state space network — → 2309.00638
- End-to-end risk budgeting portfolio optimization with neural networks (2024) — link
- Deep reinforcement learning for optimal portfolio allocation: a comparative study with mean-variance optimization (2023) — link
- Advancing investment frontiers: industry-grade deep reinforcement learning for portfolio optimization (2024) — link
- Deep reinforcement learning and mean-variance strategies for responsible portfolio optimization (2024) — link
- FreQuant: a reinforcement-learning based adaptive portfolio optimization with multi-frequency decomposition (2024) — link
- Deep learning in finance: a survey of applications and techniques (2024) — link
- Large Language Models in finance: a survey — → 2311.10723
- From deep learning to LLMs: a survey of AI in quantitative investment — → 2503.21422
- Recurrent neural network wave functions for Rydberg atom arrays on kagome lattice (2024) — link
- Investigating topological order using recurrent neural networks (2023) — link
- Leveraging recurrence in neural network wavefunctions for large-scale simulations of Heisenberg antiferromagnets: the triangular lattice (2025) — link
- Nonlinear programming (1997)
- Numerical Optimization (2006) — link
- Learning Constrained Optimization with Deep Augmented Lagrangian Methods (2024) — link
- An adaptive augmented Lagrangian method for training physics and equality constrained artificial neural networks (2023) — link
- Gurobi Optimizer
- IBM ILOG CPLEX Optimizer
- MOSEK Optimization Suite
9 of 99 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2596 linked papers.