Longitudinal market structure detection using a dynamic modularity-spectral algorithm

ArXiv ID: 2407.04500 “View on arXiv”

Authors: Unknown

Abstract

In this paper, we introduce the Dynamic Modularity-Spectral Algorithm (DynMSA), a novel approach to identify clusters of stocks with high intra-cluster correlations and low inter-cluster correlations by combining Random Matrix Theory with modularity optimisation and spectral clustering. The primary objective is to uncover hidden market structures and find diversifiers based on return correlations, thereby achieving a more effective risk-reducing portfolio allocation. We applied DynMSA to constituents of the S&P 500 and compared the results to sector- and market-based benchmarks. Besides the conception of this algorithm, our contributions further include implementing a sector-based calibration for modularity optimisation and a correlation-based distance function for spectral clustering. Testing revealed that DynMSA outperforms baseline models in intra- and inter-cluster correlation differences, particularly over medium-term correlation look-backs. It also identifies stable clusters and detects regime changes due to exogenous shocks, such as the COVID-19 pandemic. Portfolios constructed using our clusters showed higher Sortino and Sharpe ratios, lower downside volatility, reduced maximum drawdown and higher annualised returns compared to an equally weighted market benchmark.

Keywords: spectral clustering, random matrix theory, portfolio optimization, modularity optimisation, correlation clustering, Equities (S&P 500)

Complexity vs Empirical Score

• Math Complexity: 7.0/10
• Empirical Rigor: 8.5/10
• Quadrant: Holy Grail
• Why: The paper employs advanced mathematical techniques including Random Matrix Theory, modularity optimization, and spectral clustering, with detailed derivations in the methodology section. It demonstrates high empirical rigor through extensive backtesting on S&P 500 constituents, showing outperformance against benchmarks with specific metrics (Sharpe/Sortino ratios, drawdowns) and sensitivity to regime changes like COVID-19.

  flowchart TD
    A["Research Goal: Identify Stable Stock Clusters<br>for Risk-Reduced Portfolio Allocation"] --> B["Methodology: DynMSA Algorithm"]
    B --> C["Random Matrix Theory Filtering<br>Remove Market Noise"]
    C --> D["Modularity Optimisation<br>Sector-based Calibration"]
    D --> E["Spectral Clustering<br>Correlation-based Distance"]
    E --> F["Apply to S&P 500 Data"]
    F --> G{"Key Outcomes"}
    G --> H["Outperformed Benchmarks<br>Higher Sharpe/Sortino Ratios"]
    G --> I["Detected Regime Shifts<br>e.g., COVID-19 Pandemic"]