Paper: arXiv 2609.08106

Authors: Kunhan Guo

Abstract

MASTER’s inter-stock multi-head attention – the module responsible for modeling cross-sectional stock relationships – accounts for 42.5% of model parameters and 25% of predictive value. We systematically decompose this module and uncover a surprising structure: the learned attention is near-uniform (perplexity 278/300), yet forcing exact uniformity eliminates all cross-sectional discrimination. Spectral analysis resolves this paradox: the deviation from uniformity is low-rank (effective rank ~65, top-10 modes capture 96.5% of energy), explaining why sparse approximations consistently fail while Nystrom low-rank attention (m=32 landmarks) matches full O(N^2) attention at O(mN) cost – certified equivalent via TOST at both N=300 (5 seeds, Rank IC p=0.003) and N=800 (10 seeds, Rank IC p=0.034). Additional findings include: (i) attention anti-correlates with return similarity (Spearman rho = -0.614; on the industry-labeled subset, -0.645 unconditionally and -0.627 after controlling for industry, beta, and volatility), suggesting complementarity-seeking rather than correlation mining; (ii) all graph-based alternatives degrade performance, with hard masking worse than complete module removal; and (iii) at N ~ 3,500 with adapted architectures, no cross-stock module (GCN, Nystrom, or MASTER-style pipeline) significantly outperforms a per-stock LSTM baseline (n=4 seeds), indicating that the benefits observed at smaller scales do not trivially transfer. These results establish that the inter-stock attention’s value resides in a compressible, dynamic, near-global redistribution that rewards low-rank approximation but resists sparsification.

Complexity vs Empirical Score

  • Math Complexity: 7.5/10
  • Empirical Rigor: 8.5/10
  • Quadrant: Holy Grail — high math complexity, high empirical rigor

Why this score: This paper presents a rigorous empirical analysis of a complex deep learning model in quantitative finance, offering novel insights into its internal workings and proposing an efficient approximation. The findings are well-supported by statistical tests and multi-seed validations, demonstrating both strong empirical rigor and significant novelty.

Research Flowchart

  flowchart TD
    A[Research Goal: Understand MASTER's Cross-Sectional Attention] --> B{Key Methodology: Decompose & Analyze MASTER's Attention Module};
    B --> C{Data/Inputs: Stock Market Data};
    C --> D[Computational Processes: Spectral Analysis, TOST, Correlation Analysis, Alternative Comparisons];
    D --> E[Outcome 1: Near-Uniform, Low-Rank Attention (Effective Rank ~65)];
    E --> F[Outcome 2: Nystrom Low-Rank Attention Matches Full Attention (N=300 & N=800)];
    F --> G[Outcome 3: Attention Anti-Correlates with Return Similarity];
    G --> H[Outcome 4: Graph-based Alternatives Degrade Performance];
    H --> I[Outcome 5: No Cross-Stock Benefit at N~3,500];
    I --> J[Conclusion: Inter-stock attention is compressible, dynamic, near-global, rewards low-rank approx. but resists sparsification];