Walk-Forward Validation

XGBoost Forecasting of NEPSE Index Log Returns with Walk Forward Validation ArXiv ID: 2601.08896 “View on arXiv” Authors: Sahaj Raj Malla, Shreeyash Kayastha, Rumi Suwal, Harish Chandra Bhandari, Rajendra Adhikari Abstract This study develops a robust machine learning framework for one-step-ahead forecasting of daily log-returns in the Nepal Stock Exchange (NEPSE) Index using the XGBoost regressor. A comprehensive feature set is engineered, including lagged log-returns (up to 30 days) and established technical indicators such as short- and medium-term rolling volatility measures and the 14-period Relative Strength Index. Hyperparameter optimization is performed using Optuna with time-series cross-validation on the initial training segment. Out-of-sample performance is rigorously assessed via walk-forward validation under both expanding and fixed-length rolling window schemes across multiple lag configurations, simulating real-world deployment and avoiding lookahead bias. Predictive accuracy is evaluated using root mean squared error, mean absolute error, coefficient of determination (R-squared), and directional accuracy on both log-returns and reconstructed closing prices. Empirical results show that the optimal configuration, an expanding window with 20 lags, outperforms tuned ARIMA and Ridge regression benchmarks, achieving the lowest log-return RMSE (0.013450) and MAE (0.009814) alongside a directional accuracy of 65.15%. While the R-squared remains modest, consistent with the noisy nature of financial returns, primary emphasis is placed on relative error reduction and directional prediction. Feature importance analysis and visual inspection further enhance interpretability. These findings demonstrate the effectiveness of gradient boosting ensembles in modeling nonlinear dynamics in volatile emerging market time series and establish a reproducible benchmark for NEPSE Index forecasting. ...

Interpretable Hypothesis-Driven Trading:A Rigorous Walk-Forward Validation Framework for Market Microstructure Signals ArXiv ID: 2512.12924 “View on arXiv” Authors: Gagan Deep, Akash Deep, William Lamptey Abstract We develop a rigorous walk-forward validation framework for algorithmic trading designed to mitigate overfitting and lookahead bias. Our methodology combines interpretable hypothesis-driven signal generation with reinforcement learning and strict out-of-sample testing. The framework enforces strict information set discipline, employs rolling window validation across 34 independent test periods, maintains complete interpretability through natural language hypothesis explanations, and incorporates realistic transaction costs and position constraints. Validating five market microstructure patterns across 100 US equities from 2015 to 2024, the system yields modest annualized returns (0.55%, Sharpe ratio 0.33) with exceptional downside protection (maximum drawdown -2.76%) and market-neutral characteristics (beta = 0.058). Performance exhibits strong regime dependence, generating positive returns during high-volatility periods (0.60% quarterly, 2020-2024) while underperforming in stable markets (-0.16%, 2015-2019). We report statistically insignificant aggregate results (p-value 0.34) to demonstrate a reproducible, honest validation protocol that prioritizes interpretability and extends naturally to advanced hypothesis generators, including large language models. The key empirical finding reveals that daily OHLCV-based microstructure signals require elevated information arrival and trading activity to function effectively. The framework provides complete mathematical specifications and open-source implementation, establishing a template for rigorous trading system evaluation that addresses the reproducibility crisis in quantitative finance research. For researchers, practitioners, and regulators, this work demonstrates that interpretable algorithmic trading strategies can be rigorously validated without sacrificing transparency or regulatory compliance. ...

Discovery of a 13-Sharpe OOS Factor: Drift Regimes Unlock Hidden Cross-Sectional Predictability ArXiv ID: 2511.12490 “View on arXiv” Authors: Mainak Singha Abstract We document a high-performing cross-sectional equity factor that achieves out-of-sample Sharpe ratios above 13 through regime-conditional signal activation. The strategy combines value and short-term reversal signals only during stock-specific drift regimes, defined as periods when individual stocks show more than 60 percent positive days in trailing 63-day windows. Under these conditions, the factor delivers annualized returns of 158.6 percent with 12.0 percent volatility and a maximum drawdown of minus 11.9 percent. Using rigorous walk-forward validation across 20 years of S&P 500 data (2004 to 2024), we show performance roughly 13 times stronger than market benchmarks on a risk-adjusted basis, produced entirely out-of-sample with frozen parameters. The factor passes extensive robustness tests, including 1,000 randomization trials with p-values below 0.001, and maintains Sharpe ratios above 7 even under 30 percent parameter perturbations. Exposure to standard risk factors is negligible, with total R-squared values below 3 percent. We provide mechanistic evidence that drift regimes reshape market microstructure by amplifying behavioral biases, altering liquidity patterns, and creating conditions where cross-sectional price discovery becomes systematically exploitable. Conservative capacity estimates indicate deployable capital of 100 to 500 million dollars before noticeable performance degradation. ...

LSTM-ARIMA as a Hybrid Approach in Algorithmic Investment Strategies ArXiv ID: 2406.18206 “View on arXiv” Authors: Unknown Abstract This study focuses on building an algorithmic investment strategy employing a hybrid approach that combines LSTM and ARIMA models referred to as LSTM-ARIMA. This unique algorithm uses LSTM to produce final predictions but boosts the results of this RNN by adding the residuals obtained from ARIMA predictions among other inputs. The algorithm is tested across three equity indices (S&P 500, FTSE 100, and CAC 40) using daily frequency data from January 2000 to August 2023. The testing architecture is based on the walk-forward procedure for the hyperparameter tunning phase that uses Random Search and backtesting the algorithms. The selection of the optimal model is determined based on adequately selected performance metrics focused on risk-adjusted return measures. We considered two strategies for each algorithm: Long-Only and Long-Short to present the situation of two various groups of investors with different investment policy restrictions. For each strategy and equity index, we compute the performance metrics and visualize the equity curve to identify the best strategy with the highest modified information ratio. The findings conclude that the LSTM-ARIMA algorithm outperforms all the other algorithms across all the equity indices which confirms the strong potential behind hybrid ML-TS (machine learning - time series) models in searching for the optimal algorithmic investment strategies. ...