Paper: arXiv 2309.00073
Authors: Kelvin J. L. Koa, Yunshan Ma, Ritchie Ng, Tat-Seng Chua
Abstract
Multi-step stock price prediction over a long-term horizon is crucial for forecasting its volatility, allowing financial institutions to price and hedge derivatives, and banks to quantify the risk in their trading books. Additionally, most financial regulators also require a liquidity horizon of several days for institutional investors to exit their risky assets, in order to not materially affect market prices. However, the task of multi-step stock price prediction is challenging, given the highly stochastic nature of stock data. Current solutions to tackle this problem are mostly designed for single-step, classification-based predictions, and are limited to low representation expressiveness. The problem also gets progressively harder with the introduction of the target price sequence, which also contains stochastic noise and reduces generalizability at test-time. To tackle these issues, we combine a deep hierarchical variational-autoencoder (VAE) and diffusion probabilistic techniques to do seq2seq stock prediction through a stochastic generative process. The hierarchical VAE allows us to learn the complex and low-level latent variables for stock prediction, while the diffusion probabilistic model trains the predictor to handle stock price stochasticity by progressively adding random noise to the stock data. Our Diffusion-VAE (D-Va) model is shown to outperform state-of-the-art solutions in terms of its prediction accuracy and variance. More importantly, the multi-step outputs can also allow us to form a stock portfolio over the prediction length. We demonstrate the effectiveness of our model outputs in the portfolio investment task through the Sharpe ratio metric and highlight the importance of dealing with different types of prediction uncertainties.
Complexity vs Empirical Score
- Math Complexity: 7.0/10
- Empirical Rigor: 6.0/10 (capped from 7.5: transaction costs not modelled; see penalty flags)
- Quadrant: Holy Grail — high math complexity, high empirical rigor (effective empirical rigor capped from 7.5)
Why this score: The paper employs advanced mathematics including hierarchical variational autoencoders, diffusion probabilistic models, and denoising score-matching, with some theoretical derivations in the latent space. It demonstrates strong empirical rigor through extensive experiments on public datasets, code availability, ablation studies, and a practical portfolio evaluation using Sharpe ratio metrics.
Research Flowchart
flowchart TD
A["Research Goal: Multi-Step Stock Price Prediction"] --> B["Methodology: Diffusion VAE"]
B --> C["Hierarchical VAE: Learn Low-Level Latent Variables"]
B --> D["Diffusion Probabilistic Model: Handle Stochasticity"]
C & D --> E["Training Process: Seq2Seq Prediction"]
E --> F{"Key Findings / Outcomes"}
F --> G["SOTA Prediction Accuracy & Low Variance"]
F --> H["Improved Portfolio Construction via Sharpe Ratio"]
F --> I["Effective Uncertainty Quantification"]
Evidence Card
What the authors claim, on what data, and who has checked it. Read from the paper's own text by a language model: every quote is verbatim, nothing was re-run. How to read this card
Proposes
The paper proposes the Diffusion‑VAE (D‑Va) model, an end‑to‑end framework that combines a hierarchical VAE, a stochastic diffusion process and a denoising component for multi‑step stock price prediction.
Source sentence
We propose a solution that integrates a hierarchical VAE model, a stochastic diffusion process and a denoising component and implement it in an end-to-end model for stock prediction.
Assumes 3
- Gaussian noise is added to input and target sequences via a diffusion process with a variance schedule. model
Source sentence
In the diffusion probabilistic model, we define a Markov chain of diffusion steps that slowly adds random Gaussian noise to the input sequence X to obtain the noisy samples X1, X2, · · · , X𝑁, where 𝑁 is the number of diffusion steps.
- Risk‑free interest rate is set to zero when computing Sharpe ratios. market
Source sentence
we set the risk-free interest rate to 0
- Historical price data from Yahoo Finance is accurate and sufficient for training and evaluation. data
Source sentence
The data is collected from Yahoo Finance3, and is processed in the same manner as the ACL18 dataset.
Data
- ACL18 StockNet dataset market-prices public
Source sentence
It contains the historical prices of 88 high trade volume stocks from the U.S. market, which represents the top 8-10 stocks in capital size across 9 major industries. The duration of data ranges between 01/01/2014 to 01/01/2017.
- Extended StockNet dataset market-prices public
Source sentence
We extend this dataset by collecting updated U.S. stock data from 01/01/2017 to 01/01/2023, taking the latest top 10 stocks from the 11 major industries
Costs
Not modeled
Finds 3
- D‑Va outperforms state‑of‑the‑art methods in multi‑step prediction accuracy and variance. predictive-accuracy
Source sentence
Our Diffusion‑VAE (D‑Va) model is shown to outperform state-of-the-art solutions in terms of its prediction accuracy and variance.
- Using D‑Va predictions with covariance regularization yields the highest Sharpe ratios across datasets and prediction horizons. performance
Source sentence
using the predictions from D‑Va, together with covariance regularization, we are able to form portfolios that consistently achieve the best Sharpe ratios, even across difference sequence lengths 𝑇.
- Regularization and D‑Va together achieve the best Sharpe ratio performance, surpassing an equal‑weight baseline despite using only historical price data. performance
Source sentence
using D‑Va as the prediction model and applying regularization both help to improve the Sharpe ratio results, and combining both methods allow us to obtain the best Sharpe ratio performance.
Checked by others
No other paper in the archive has been found to support or contradict these findings (compared with 24 related papers). Coverage is partial, so this is neither replication nor refutation.
13 papers in the archive : FinCast: A Foundation Model for Financial Time-Series … ; RiskMiner: Discovering Formulaic Alphas via Risk Seeking … ; From Factor Models to Deep Learning: Machine Learning in …
Extracted 2026-11-14 · same card as JSON · A language model chose these items: check the source sentence before relying on one.