Interpretable Hypothesis-Driven Trading:A Rigorous Walk-Forward Validation Framework for Market Microstructure Signals

ArXiv ID: 2512.12924 “View on arXiv”

Authors: Gagan Deep, Akash Deep, William Lamptey

Abstract

We develop a rigorous walk-forward validation framework for algorithmic trading designed to mitigate overfitting and lookahead bias. Our methodology combines interpretable hypothesis-driven signal generation with reinforcement learning and strict out-of-sample testing. The framework enforces strict information set discipline, employs rolling window validation across 34 independent test periods, maintains complete interpretability through natural language hypothesis explanations, and incorporates realistic transaction costs and position constraints. Validating five market microstructure patterns across 100 US equities from 2015 to 2024, the system yields modest annualized returns (0.55%, Sharpe ratio 0.33) with exceptional downside protection (maximum drawdown -2.76%) and market-neutral characteristics (beta = 0.058). Performance exhibits strong regime dependence, generating positive returns during high-volatility periods (0.60% quarterly, 2020-2024) while underperforming in stable markets (-0.16%, 2015-2019). We report statistically insignificant aggregate results (p-value 0.34) to demonstrate a reproducible, honest validation protocol that prioritizes interpretability and extends naturally to advanced hypothesis generators, including large language models. The key empirical finding reveals that daily OHLCV-based microstructure signals require elevated information arrival and trading activity to function effectively. The framework provides complete mathematical specifications and open-source implementation, establishing a template for rigorous trading system evaluation that addresses the reproducibility crisis in quantitative finance research. For researchers, practitioners, and regulators, this work demonstrates that interpretable algorithmic trading strategies can be rigorously validated without sacrificing transparency or regulatory compliance.

Keywords: Algorithmic Trading, Walk-forward Validation, Market Microstructure, Reinforcement Learning, Overfitting

Complexity vs Empirical Score

• Math Complexity: 7.0/10
• Empirical Rigor: 9.0/10
• Quadrant: Holy Grail
• Why: The paper employs advanced mathematical frameworks for walk-forward validation and reinforcement learning, but its core contribution is a rigorous, implementation-heavy empirical protocol with extensive backtesting data, strict information discipline, and open-source code.

  flowchart TD
    A["Research Goal:<br>Develop Rigorous Validation<br>for Microstructure Trading"] --> B["Methodology: Walk-Forward Framework<br>Signal Gen & Strict OOS Testing"]
    B --> C["Data Input:<br>5 MS Patterns, 100 US Equities<br>2015-2024"]
    C --> D["Computational Process:<br>34 Rolling Windows +<br>Transaction Costs +<br>RL Optimization"]
    D --> E["Outcome 1: Modest Returns<br>0.55% Annualized, Sharpe 0.33"]
    D --> F["Outcome 2: Regime Dependence<br>+0.60% High Vol vs -0.16% Stable"]
    D --> G["Outcome 3: Stat. Insignificant<br>P-value 0.34<br>Reproducible Template"]