Paper: arXiv 2307.00459

Authors: Eugene W. Park

Abstract

This paper presents a method for predicting stock returns using principal component analysis (PCA) and the hidden Markov model (HMM) and tests the results of trading stocks based on this approach. Principal component analysis is applied to the covariance matrix of stock returns for companies listed in the S&P 500 index, and interpreting principal components as factor returns, we apply the HMM model on them. Then we use the transition probability matrix and state conditional means to forecast the factors returns. Reverting the factor returns forecasts to stock returns using eigenvectors, we obtain forecasts for the stock returns. We find that, with the right hyperparameters, our model yields a strategy that outperforms the buy-and-hold strategy in terms of the annualized Sharpe ratio.

Complexity vs Empirical Score

  • Math Complexity: 7.5/10
  • Empirical Rigor: 6.0/10 (capped from 7.0: transaction costs not modelled; see penalty flags)
  • Quadrant: Holy Grail — high math complexity, high empirical rigor (effective empirical rigor capped from 7.0)

Why this score: The paper employs advanced mathematical concepts like spectral theorem for symmetric matrices, eigen-decomposition, and Gaussian HMMs, resulting in high math complexity. It also demonstrates strong empirical rigor by applying the method to S&P 500 data, implementing trading strategies, and reporting performance metrics like annualized Sharpe ratio against a benchmark.

Research Flowchart

  flowchart TD
  Goal["Research Goal: Forecast Stock Returns & Outperform Buy-and-Hold"] -->|"Input"| Data["Data: S&P 500 Historical Returns"]
  Data -->|"Process 1"| PCA["PCA: Decompose Covariance Matrix into Principal Components (Factors)"]
  PCA -->|"Output"| Factors["Factor Returns (Eigenvectors)"]
  Factors -->|"Process 2"| HMM["HMM: Model Factor Dynamics & Forecast Returns"]
  HMM -->|"Output"| Forecast["Forecasted Stock Returns"]
  Forecast -->|"Process 3"| Strategy["Trading Strategy: Reconstruct & Trade"]
  Strategy -->|"Result"| Outcome["Outcome: Annualized Sharpe Ratio > Buy-and-Hold"]

Evidence Card

What the authors claim, on what data, and who has checked it. Read from the paper's own text by a language model: every quote is verbatim, nothing was re-run. How to read this card

Proposes

The paper proposes a method that combines principal component analysis and hidden Markov models to forecast stock returns and build trading strategies.

Source sentence
This paper presents a method for predicting stock returns using principal component analysis (PCA) and the hidden Markov model (HMM) and tests the results of trading stocks based on this approach.

Assumes 6

  • Emission probabilities in each HMM state follow a Gaussian distribution. model
    Source sentence
    we assume the emission probability in each state in the state space, S, to have have a Gaussian distribution.
  • The underlying Markov chain is equally likely to start in any state. model
    Source sentence
    we are assuming that it is equally likely for the underlying Markov chain to start in any state
  • State transition probabilities are equally likely between any pair of states. model
    Source sentence
    it is equally likely for the Markov chain to transit from one state to another
  • Trading can be executed at the exact closing price with no transaction costs. execution
    Source sentence
    we assume the following: 1. we are able to trade at exactly the closing time 2. we can exactly match the n assets with equal weight in our portfolio 3. and we do not incur any trading costs.
Show 2 more
  • Risk‑free rate is zero. model
    Source sentence
    We will assume the risk-free rate to be zero.
  • Survivor bias is mitigated by requiring at least 400 return observations per period. data
    Source sentence
    we are subject to survivor bias, but minimize this effect by ensuring that we have at least 400 data points of company returns for each time period.

Data

  • Weekly returns of S&P 500 companies · finance.yahoo.com market-prices public
    July 23, 2004 to July 8, 2016 · S&P 500 constituents
    Source sentence
    We use data for the weekly returns of companies listed in the S&P500 to train our model. The returns are calculated using the closing stock prices, and each return sequence comprise the columns of our dataset, denoted X.

Costs

Not modeled A trading or execution result is reported without costs.

Source sentence
we do not incur any trading costs.

Finds 5

  • Strategy 2 with p = 15% achieves an annualized Sharpe ratio of 1.36, more than 50% higher than the buy‑and‑hold benchmark. performance
    Source sentence
    the strategy with the highest Sharpe ratio is strategy 2 with p = 15%. This outperforms
  • Forecast accuracy (directional) is only slightly above 50%, with Strategy 1 yielding just above 50% winning probability. predictive-accuracy
    Source sentence
    Strategy 1 clearly outperforms strategy 2 in forecasting the direction of the returns. Yet, strategy 1 yield just above 50%...
  • Sharpe ratios are highly sensitive to the hyperparameter p, with a spread of up to 0.91 between the best and worst configurations. statistical
    Source sentence
    ratios depend heavily on the hyperparameter p. The difference between the highest and lowest ratios of strategy is as high as 0.91.
  • The PCA+HMM model provides forecast accuracy slightly above 50% and can outperform the market when the right p is chosen. performance
    Source sentence
    yields model-based trading strategies with forecast accuracy slightly above 50%.
  • Strategy 2 (p = 15%) shows lower cumulative return than buy‑and‑hold but is more stable, avoiding major downturns in late‑2014 and early‑2016. performance
    Source sentence
    it rides out two ocassions of downturns, during late-2014 and early-2016, without much losses, contrary to the other strategies.

Checked by others

No other paper in the archive has been found to support or contradict these findings (compared with 36 related papers). Coverage is partial, so this is neither replication nor refutation.

Code

No repository by the authors found in the paper.

Extracted 2026-11-14 · same card as JSON · A language model chose these items: check the source sentence before relying on one.