{"paper_id":"2505.06383","title":"The bias of IID resampled backtests for rolling-window mean-variance portfolios","authors":["Andrew Paskaramoorthy","Terence van Zyl","Tim Gebbie"],"paper_date":"2025-05-09","paper_type":"empirical","paper_url":"https://arxiv.org/pdf/2505.06383v1","page_url":"https://thequant.space/flowcharts/the-bias-of-iid-resampled-backtests-for-rolling-window-mean/","scores":{"empirical_rigor":6,"math_complexity":7.5,"quadrant":"Holy Grail"},"proposes":{"status":"available","statement":"The paper explains the bias mechanism of IID‑resampled backtests for rolling‑window mean‑variance portfolios and derives analytical bounds plus a practical heuristic for bias severity.","quote":"Our first main contribution is the explanation of the bias mechanism. We demonstrate analytically and through simulation that IID resampling introduces bias primarily by disrupting the first-order autocovariance structure of asset returns.","summary":"Backtests on historical data are the basis for practical evaluations of portfolio selection rules, but their reliability is often limited by reliance on a single sample path. This can lead to high estimation variance. Resampling techniques offer a potential solution by increasing the effective…","strategy":null},"assumptions":{"status":"available","items":[{"assumption":"The realised return process is stationary and ergodic.","type":"statistical","stated":"explicit","quote":"We assume that the realised return process is stationary and ergodic."},{"assumption":"Portfolio rule’s out‑of‑sample returns are stationary.","type":"statistical","stated":"explicit","quote":"We assume that the portfolio rule’s out‑of‑sample returns are stationary."},{"assumption":"Standard backtest Sharpe Ratio is unbiased and consistent under stationarity.","type":"statistical","stated":"explicit","quote":"Assuming stationarity, the backtested Sharpe Ratio is an un‑biased and consistent estimator of the true Sharpe Ratio."},{"assumption":"IID resampling is performed without replacement (shuffling) on the historical sample path.","type":"model","stated":"explicit","quote":"An IID Resampled backtest involves applying IID resampling without repetition (i.e., shuffling) to the historical sample path."},{"assumption":"The data‑generating process can be characterised by first‑lag return autocorrelation (and optionally GARCH effects).","type":"model","stated":"implied","quote":"The bounds confirm that the bias magnitude is strongly linked to first‑lag return autocorrelation, which is observable."}]},"datasets":{"status":"available","items":[],"known":[{"id":"jkp","name":"JKP Global Factor Data","provider":"Jensen, Kelly and Pedersen","kind":"market-prices","access":"public","quote":"Data We use JKP Factor data [9] to specify realistic parameters for our assumed DGP.","mentions":4,"used_in":1}],"sample":{"start":null,"end":null,"market":null,"asset_class":null,"frequency":null},"data_resolution":"unclear"},"transaction_costs":{"status":"not_extracted","modeled":"unclear","headline_net_of_costs":null,"detail":null,"quote":null,"source":"none","rule":"R0","candidates":{"front_matter":"unclear","evidence_card":"unclear","evidence_card_quoted":false},"meaning":"the classifier was not confident"},"results":{"status":"available","evaluation_type":"unclear","findings":[{"finding":"The bias of the IID‑resampled Sharpe Ratio is largely driven by the first‑lag autocovariance and is often a fraction of the standard error of the original backtest.","kind":"theoretical","quote":"The bias of the IID resampled Sharpe Ratio is actually linearly related to the autocovariance, where the gradient of the line is related to the inverse of the asset’s Sharpe Ratio."},{"finding":"Standardised bias of the IID‑resampled Sharpe Ratio has a 95th percentile of only 0.33, indicating a modest bias relative to estimation noise.","kind":"statistical","quote":"The bias from IIS resampling appears to be modest relative to the standard error of the standard backtest, with a 95th percentile of the standardised biases of 0.33 for the Sharpe Ratio."},{"finding":"Including GARCH effects increases the standardised bias for mean, variance, and Sharpe Ratio estimates.","kind":"statistical","quote":"The standardised biases for the mean, variance, and Sharpe Ratio all increase by including GARCH effects."},{"finding":"Bias magnitude grows with portfolio dimension; for M=40 the standardised Sharpe Ratio bias is about three times that for M=1.","kind":"theoretical","quote":"The standardised bias grows with the dimension as seen in Figure 4, where the standardised bias of the Sharpe Ratio for M = 40 is approximately three times as large as M = 1."},{"finding":"When the underlying asset’s Sharpe Ratio is low, the IID‑resampled Sharpe Ratio bias is larger, reflecting greater disruption of temporal dependence.","kind":"theoretical","quote":"IID resampled Sharpe Ratios for the MV portfolio are most affected by disruption in the temporal dependence when the underlying asset’s Sharpe Ratio is small."}],"reported_performance":null,"out_of_sample":{"method":"unclear","headline_out_of_sample":null}},"independent_evidence":{"status":"none_found","supports":[],"contradicts":[],"qualifies":[],"compared_with":47,"citations":{"cited_by_in_corpus":0,"by_independent_authors":0,"how_cited":{"used_as_baseline":0,"uses_method_or_data":0,"extends":0,"discusses_findings":0,"mentions":0,"unclassified":0},"independent_papers":[],"scope":"Citations between papers in this corpus only; citations from outside it are not counted."}},"implementation":{"status":"available","public":false,"code_url":null,"note":"No repository presented by the authors as their own was found in the paper. Absence here is not proof that no code exists."},"replication":{"status":"none_found","card":null,"note":"No replication of this paper has been run by thequant.space yet."},"extraction":{"text_source":"fulltext","model":"openai/gpt-oss-120b","version":1,"extracted_at":"2026-11-14T17:38:50Z"},"provenance":{"claims":"Everything here is what the authors state in the paper. Nothing was re-run or reproduced by the platform.","quotes":"Assumptions, datasets, cost detail and findings were extracted by a language model. Every quote is verbatim paper text. Where an item has `support`, \"full\" means every figure in it appears in the quoted sentences and \"partial\" means a figure is stated elsewhere in the paper; items with a figure found nowhere in the paper were discarded.","datasets_known":"datasets.known lists well-known vendors and public datasets by a stable id, found by a dictionary over the full text where the sentence says the data is used; it is independent of the model and covers only names in the dictionary.","flags":"transaction_costs.modeled, results.evaluation_type and results.out_of_sample.method are platform classifications (structured-decision models); see https://thequant.space/score-guide/#penalty-flags","comparisons":"supports / contradicts / qualifies are a language model's reading of two papers' quoted findings, restricted to closely related papers that share no author. They are pointers to read, not verified replications, and coverage is partial.","implementation":"code_url is a repository the paper itself presents as its own (rule-scored, ambiguous cases judged by a model), checked for liveness when it was recorded."},"docs":"https://thequant.space/agents/"}