Improving Portfolio Performance Using a Novel Method for Predicting Financial Regimes

ArXiv ID: 2310.04536 “View on arXiv”

Authors: Unknown

Abstract

This work extends a previous work in regime detection, which allowed trading positions to be profitably adjusted when a new regime was detected, to ex ante prediction of regimes, leading to substantial performance improvements over the earlier model, over all three asset classes considered (equities, commodities, and foreign exchange), over a test period of four years. The proposed new model is also benchmarked over this same period against a hidden Markov model, the most popular current model for financial regime prediction, and against an appropriate index benchmark for each asset class, in the case of the commodities model having a test period cost-adjusted cumulative return over four times higher than that expected from the index. Notably, the proposed model makes use of a contrarian trading strategy, not uncommon in the financial industry but relatively unexplored in machine learning models. The model also makes use of frequent short positions, something not always desirable to investors due to issues of both financial risk and ethics; however, it is discussed how further work could remove this reliance on shorting and allow the construction of a long-only version of the model.

Keywords: regime detection, regime prediction, hidden Markov model, contrarian trading, short positions, Equities, Commodities, and Foreign Exchange

Complexity vs Empirical Score

• Math Complexity: 6.0/10
• Empirical Rigor: 8.0/10
• Quadrant: Holy Grail
• Why: The paper employs advanced statistical and machine learning concepts (Markov switching models, random forests, feature selection with BorutaShap) and presents a rigorous backtest with multiple asset classes, realistic transaction costs, and benchmark comparisons.

  flowchart TD
    A["Research Goal: Ex Ante Prediction of<br>Financial Regimes to Improve Portfolio Performance"] --> B["Data Inputs: Equities, Commodities, Forex<br>4-Year Test Period"]
    B --> C["Methodology: Novel ML Model<br>with Contrarian Strategy & Short Positions"]
    C --> D["Computational Process: Compare against<br>Hidden Markov Model & Benchmark Indices"]
    D --> E["Key Findings: Substantial Performance<br>Improvement over HMM & Benchmarks"]
    E --> F["Outcome: Over 4x Cumulative Return<br>vs Benchmark in Commodities"]