Paper: arXiv 2309.00073

Authors: Kelvin J. L. Koa, Yunshan Ma, Ritchie Ng, Tat-Seng Chua

Abstract

Multi-step stock price prediction over a long-term horizon is crucial for forecasting its volatility, allowing financial institutions to price and hedge derivatives, and banks to quantify the risk in their trading books. Additionally, most financial regulators also require a liquidity horizon of several days for institutional investors to exit their risky assets, in order to not materially affect market prices. However, the task of multi-step stock price prediction is challenging, given the highly stochastic nature of stock data. Current solutions to tackle this problem are mostly designed for single-step, classification-based predictions, and are limited to low representation expressiveness. The problem also gets progressively harder with the introduction of the target price sequence, which also contains stochastic noise and reduces generalizability at test-time. To tackle these issues, we combine a deep hierarchical variational-autoencoder (VAE) and diffusion probabilistic techniques to do seq2seq stock prediction through a stochastic generative process. The hierarchical VAE allows us to learn the complex and low-level latent variables for stock prediction, while the diffusion probabilistic model trains the predictor to handle stock price stochasticity by progressively adding random noise to the stock data. Our Diffusion-VAE (D-Va) model is shown to outperform state-of-the-art solutions in terms of its prediction accuracy and variance. More importantly, the multi-step outputs can also allow us to form a stock portfolio over the prediction length. We demonstrate the effectiveness of our model outputs in the portfolio investment task through the Sharpe ratio metric and highlight the importance of dealing with different types of prediction uncertainties.

Complexity vs Empirical Score

  • Math Complexity: 7.0/10
  • Empirical Rigor: 6.0/10 (capped from 7.5: transaction costs not modelled; see penalty flags)
  • Quadrant: Holy Grail — high math complexity, high empirical rigor (effective empirical rigor capped from 7.5)

Why this score: The paper employs advanced mathematics including hierarchical variational autoencoders, diffusion probabilistic models, and denoising score-matching, with some theoretical derivations in the latent space. It demonstrates strong empirical rigor through extensive experiments on public datasets, code availability, ablation studies, and a practical portfolio evaluation using Sharpe ratio metrics.

Research Flowchart

  flowchart TD
  A["Research Goal: Multi-Step Stock Price Prediction"] --> B["Methodology: Diffusion VAE"]
  B --> C["Hierarchical VAE: Learn Low-Level Latent Variables"]
  B --> D["Diffusion Probabilistic Model: Handle Stochasticity"]
  C & D --> E["Training Process: Seq2Seq Prediction"]
  E --> F{"Key Findings / Outcomes"}
  F --> G["SOTA Prediction Accuracy & Low Variance"]
  F --> H["Improved Portfolio Construction via Sharpe Ratio"]
  F --> I["Effective Uncertainty Quantification"]

Evidence Card

What the authors claim, on what data, and who has checked it. Read from the paper's own text by a language model: every quote is verbatim, nothing was re-run. How to read this card

Proposes

The paper proposes the Diffusion‑VAE (D‑Va) model, an end‑to‑end framework that combines a hierarchical VAE, a stochastic diffusion process and a denoising component for multi‑step stock price prediction.

Source sentence
We propose a solution that integrates a hierarchical VAE model, a stochastic diffusion process and a denoising component and implement it in an end-to-end model for stock prediction.

Assumes 3

  • Gaussian noise is added to input and target sequences via a diffusion process with a variance schedule. model
    Source sentence
    In the diffusion probabilistic model, we define a Markov chain of diffusion steps that slowly adds random Gaussian noise to the input sequence X to obtain the noisy samples X1, X2, · · · , X𝑁, where 𝑁 is the number of diffusion steps.
  • Risk‑free interest rate is set to zero when computing Sharpe ratios. market
    Source sentence
    we set the risk-free interest rate to 0
  • Historical price data from Yahoo Finance is accurate and sufficient for training and evaluation. data
    Source sentence
    The data is collected from Yahoo Finance3, and is processed in the same manner as the ACL18 dataset.

Data

  • ACL18 StockNet dataset · ACL18 market-prices public
    01/01/2014 to 01/01/2017 · 88 high trade volume U.S. stocks across 9 major industries
    Source sentence
    It contains the historical prices of 88 high trade volume stocks from the U.S. market, which represents the top 8-10 stocks in capital size across 9 major industries. The duration of data ranges between 01/01/2014 to 01/01/2017.
  • Extended StockNet dataset · Yahoo Finance market-prices public
    01/01/2017 to 01/01/2023 · 110 top U.S. stocks from 11 major industries
    Source sentence
    We extend this dataset by collecting updated U.S. stock data from 01/01/2017 to 01/01/2023, taking the latest top 10 stocks from the 11 major industries

Recognised sources: Yahoo Finance

Costs

Not modeled A trading or execution result is reported without costs.

Finds 3

  • D‑Va outperforms state‑of‑the‑art methods in multi‑step prediction accuracy and variance. predictive-accuracy
    Source sentence
    Our Diffusion‑VAE (D‑Va) model is shown to outperform state-of-the-art solutions in terms of its prediction accuracy and variance.
  • Using D‑Va predictions with covariance regularization yields the highest Sharpe ratios across datasets and prediction horizons. performance
    Source sentence
    using the predictions from D‑Va, together with covariance regularization, we are able to form portfolios that consistently achieve the best Sharpe ratios, even across difference sequence lengths 𝑇.
  • Regularization and D‑Va together achieve the best Sharpe ratio performance, surpassing an equal‑weight baseline despite using only historical price data. performance
    Source sentence
    using D‑Va as the prediction model and applying regularization both help to improve the Sharpe ratio results, and combining both methods allow us to obtain the best Sharpe ratio performance.

Checked by others

No other paper in the archive has been found to support or contradict these findings (compared with 24 related papers). Coverage is partial, so this is neither replication nor refutation.

Cited by 13 papers in the archive (9 by independent authors): FinCast: A Foundation Model for Financial Time-Series … (used as a baseline); RiskMiner: Discovering Formulaic Alphas via Risk Seeking … (uses its method or data); From Factor Models to Deep Learning: Machine Learning in … (uses its method or data)

Code

https://github.com/koa-fin/dva — the authors' own repository

Extracted 2026-11-14 · same card as JSON · A language model chose these items: check the source sentence before relying on one.