RegimeFolio: A Regime Aware ML System for Sectoral Portfolio Optimization in Dynamic Markets

ArXiv ID: 2510.14986 “View on arXiv”

Authors: Yiyao Zhang, Diksha Goel, Hussain Ahmad, Claudia Szabo

Abstract

Financial markets are inherently non-stationary, with shifting volatility regimes that alter asset co-movements and return distributions. Standard portfolio optimization methods, typically built on stationarity or regime-agnostic assumptions, struggle to adapt to such changes. To address these challenges, we propose RegimeFolio, a novel regime-aware and sector-specialized framework that, unlike existing regime-agnostic models such as DeepVol and DRL optimizers, integrates explicit volatility regime segmentation with sector-specific ensemble forecasting and adaptive mean-variance allocation. This modular architecture ensures forecasts and portfolio decisions remain aligned with current market conditions, enhancing robustness and interpretability in dynamic markets. RegimeFolio combines three components: (i) an interpretable VIX-based classifier for market regime detection; (ii) regime and sector-specific ensemble learners (Random Forest, Gradient Boosting) to capture conditional return structures; and (iii) a dynamic mean-variance optimizer with shrinkage-regularized covariance estimates for regime-aware allocation. We evaluate RegimeFolio on 34 large cap U.S. equities from 2020 to 2024. The framework achieves a cumulative return of 137 percent, a Sharpe ratio of 1.17, a 12 percent lower maximum drawdown, and a 15 to 20 percent improvement in forecast accuracy compared to conventional and advanced machine learning benchmarks. These results show that explicitly modeling volatility regimes in predictive learning and portfolio allocation enhances robustness and leads to more dependable decision-making in real markets.

Keywords: portfolio optimization, volatility regimes, mean-variance allocation, ensemble learning, covariance shrinkage, Equities

Complexity vs Empirical Score

• Math Complexity: 4.5/10
• Empirical Rigor: 8.0/10
• Quadrant: Street Traders
• Why: The paper uses standard ML models (RF, GBDT) and classic portfolio optimization (mean-variance with shrinkage) rather than heavy advanced mathematics; the implementation details are straightforward. However, it presents strong empirical results with specific backtest metrics (137% return, 1.17 Sharpe) over a clear 2020-2024 dataset of 34 equities, demonstrating high backtest readiness.

  flowchart TD
    Goal["Research Goal<br>Develop a Regime-Aware ML System<br>for Sectoral Portfolio Optimization"]

    Data["Data & Inputs<br>34 Large Cap U.S. Equities<br>2020-2024<br>VIX & Sector Data"]

    Proc1["Methodology: Regime Detection<br>Interpretable VIX-based Classifier<br>Identifies Volatility Regimes"]

    Proc2["Methodology: Forecasting<br>Regime & Sector-Specific Ensemble<br>RF/GBM for Conditional Returns"]

    Proc3["Methodology: Optimization<br>Dynamic Mean-Variance<br>Shrinkage-Regularized Covariance"]

    Out["Key Outcomes & Findings<br>Cumulative Return: 137%<br>Sharpe Ratio: 1.17<br>Max Drawdown: -12%<br>Forecast Accuracy: +15-20%"]

    Goal --> Data
    Data --> Proc1
    Proc1 --> Proc2
    Proc2 --> Proc3
    Proc3 --> Out