Paper: arXiv 2505.06383
Authors: Andrew Paskaramoorthy, Terence van Zyl, Tim Gebbie
Abstract
Backtests on historical data are the basis for practical evaluations of portfolio selection rules, but their reliability is often limited by reliance on a single sample path. This can lead to high estimation variance. Resampling techniques offer a potential solution by increasing the effective sample size, but can disrupt the temporal ordering inherent in financial data and introduce significant bias. This paper investigates the critical questions: First, How large is this bias for Sharpe Ratio estimates?, and then, second: What are its primary drivers?. We focus on the canonical rolling-window mean-variance portfolio rule. Our contributions are identifying the bias mechanism, and providing a practical heuristic for gauging bias severity. We show that the bias arises from the disruption of train-test dependence linked to the return auto-covariance structure and derive bounds for the bias which show a strong dependence on the observable first-lag autocorrelation. Using simulations to confirm these findings, it is revealed that the resulting Sharpe Ratio bias is often a fraction of a typical backtest’s estimation noise, benefiting from partial offsetting of component biases. Empirical analysis further illustrates that differences between IID-resampled and standard backtests align qualitatively with these drivers. Surprisingly, our results suggest that while IID resampling can disrupt temporal dependence, its resulting bias can often be tolerable. However, we highlight the need for structure-preserving resampling methods.
Complexity vs Empirical Score
- Math Complexity: 7.5/10
- Empirical Rigor: 6.0/10
- Quadrant: Holy Grail — high math complexity, high empirical rigor
Why this score: The paper employs advanced mathematical derivations to establish bias bounds and analyze the impact of temporal dependence on IID resampling, placing it on the higher end of math complexity. It also uses simulations and real-world data for validation, though it lacks extensive backtesting code or large-scale datasets, keeping empirical rigor moderate to high.
Research Flowchart
flowchart TD
A["Research Goal:<br>Quantify Bias of IID Resampling<br>in Rolling-Window MV Backtests"] --> B["Key Methodology:<br>Theoretical Analysis + Simulations"]
B --> C["Inputs:<br>Historical Return Data<br>Auto-covariance Structure"]
C --> D["Computational Process:<br>1. Standard Rolling-Window MV<br>2. IID Resampled Rolling-Window MV<br>3. Compare Sharpe Ratio Estimates"]
D --> E["Key Findings:<br>Bias Mechanism: Disrupted<br>Train-Test Dependence<br>Bias Proportional to First-Lag<br>Autocorrelation"]
E --> F["Outcome:<br>Resampling Bias is Often<br>Tolerable vs. Estimation Noise<br>Recommends Structure-Preserving<br>Methods"]
F --> G(("End"))
Evidence Card
What the authors claim, on what data, and who has checked it. Read from the paper's own text by a language model: every quote is verbatim, nothing was re-run. How to read this card
Proposes
The paper explains the bias mechanism of IID‑resampled backtests for rolling‑window mean‑variance portfolios and derives analytical bounds plus a practical heuristic for bias severity.
Source sentence
Our first main contribution is the explanation of the bias mechanism. We demonstrate analytically and through simulation that IID resampling introduces bias primarily by disrupting the first-order autocovariance structure of asset returns.
Assumes 5
- The realised return process is stationary and ergodic. statistical
Source sentence
We assume that the realised return process is stationary and ergodic.
- Portfolio rule’s out‑of‑sample returns are stationary. statistical
Source sentence
We assume that the portfolio rule’s out‑of‑sample returns are stationary.
- Standard backtest Sharpe Ratio is unbiased and consistent under stationarity. statistical
Source sentence
Assuming stationarity, the backtested Sharpe Ratio is an un‑biased and consistent estimator of the true Sharpe Ratio.
- IID resampling is performed without replacement (shuffling) on the historical sample path. model
Source sentence
An IID Resampled backtest involves applying IID resampling without repetition (i.e., shuffling) to the historical sample path.
Show 1 more
- The data‑generating process can be characterised by first‑lag return autocorrelation (and optionally GARCH effects). model
Source sentence
The bounds confirm that the bias magnitude is strongly linked to first‑lag return autocorrelation, which is observable.
Data
JKP Global Factor Data
Costs
Unclear
Finds 5
- The bias of the IID‑resampled Sharpe Ratio is largely driven by the first‑lag autocovariance and is often a fraction of the standard error of the original backtest. theoretical
Source sentence
The bias of the IID resampled Sharpe Ratio is actually linearly related to the autocovariance, where the gradient of the line is related to the inverse of the asset’s Sharpe Ratio.
- Standardised bias of the IID‑resampled Sharpe Ratio has a 95th percentile of only 0.33, indicating a modest bias relative to estimation noise. statistical
Source sentence
The bias from IIS resampling appears to be modest relative to the standard error of the standard backtest, with a 95th percentile of the standardised biases of 0.33 for the Sharpe Ratio.
- Including GARCH effects increases the standardised bias for mean, variance, and Sharpe Ratio estimates. statistical
Source sentence
The standardised biases for the mean, variance, and Sharpe Ratio all increase by including GARCH effects.
- Bias magnitude grows with portfolio dimension; for M=40 the standardised Sharpe Ratio bias is about three times that for M=1. theoretical
Source sentence
The standardised bias grows with the dimension as seen in Figure 4, where the standardised bias of the Sharpe Ratio for M = 40 is approximately three times as large as M = 1.
- When the underlying asset’s Sharpe Ratio is low, the IID‑resampled Sharpe Ratio bias is larger, reflecting greater disruption of temporal dependence. theoretical
Source sentence
IID resampled Sharpe Ratios for the MV portfolio are most affected by disruption in the temporal dependence when the underlying asset’s Sharpe Ratio is small.
Checked by others
No other paper in the archive has been found to support or contradict these findings (compared with 47 related papers). Coverage is partial, so this is neither replication nor refutation.
Code
No repository by the authors found in the paper.
Extracted 2026-11-14 · same card as JSON · A language model chose these items: check the source sentence before relying on one.
Cited References (18)
- Optimal Versus Naive Diversification: How Inefficient is the 1/N Portfolio Strategy? (2009) — link
- Tree-Based Models for Correlated Data (2022) — link
- Cross-Validation for Correlated Data (2022) — link
- A note on the validity of cross-validation for evaluating autoregressive time series prediction (2018) — link
- Advances in financial machine learning (2018)
- The Statistics of Sharpe Ratios (2002) — link
- Comparing Sharpe ratios: so where are the p-values? (2007) — link
- Is there a replication crisis in finance? (2023) — link
and 10 more
- Factor Momentum Everywhere (2019) — link
- Robust performance hypothesis testing with the Sharpe ratio (2008) — link
- Time-varying risk premium in large cross-sectional equity data sets (2016) — link
- Regression-based estimation of dynamic asset pricing models (2015) — link
- Have risk premia vanished? (2022) — link
- The equity premium and structural breaks (2001) — link
- Empirical properties of asset returns: stylized facts and statistical issues (2001) — link
- SERIAL CORRELATION, PERIODICITY AND SCALING OF EIGENMODES IN AN EMERGING MARKET (2008) — link
- The performance of hedge funds and mutual funds in emerging markets (2010)
- The Impact of Bootstrap Methods on Time Series Analysis (2003)
0 of 18 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2596 linked papers.