The problem. Replication done right costs one to four weeks per paper. Your queue of candidates is effectively infinite. Choosing well is therefore a portfolio problem: maximize what you learn per week — about markets, about a literature’s reliability, and about strategies you might run — rather than chasing the single most exciting result.
The expected-information frame
A replication’s value = (probability the outcome changes what you do) × (size of that change) ÷ (weeks of effort). This immediately demotes two popular choices: papers you’re sure are right (confirmation teaches little — replicate the canonical result only as pipeline calibration, a different purpose) and papers you’re sure are wrong (debunking a paper nobody was going to trade is entertainment). The information-rich zone is genuine uncertainty about a result you’d act on — a strategy you might run, a method you might adopt, a data source you might trust.
The five selection criteria
Score each candidate 0–2 on all five; replicate from the top:
1. Actionability (×2 weight). Would a successful replication change your allocation, your pipeline, or your priors about a whole literature? Passing the implementability tests scores 2; pure curiosity scores 0.
2. Feasibility. Data obtainable at your tier, compute within budget, methods within skill. A perfect candidate you can’t finish is a 0 — and the most common planning error. Check code availability here too: an official repo halves the cost, a good third-party repo helps and adds audit obligations.
3. Informativeness of the literature, not just the paper. Some papers are load-bearing: dozens of follow-ups inherit their construction (FI-2010 benchmarks, a zoo factor’s original sort, a widely-forked implementation). Replicating a load-bearing paper prices an entire branch of a research map at once. Our maps and hub rankings are built to spot these hubs-within-hubs.
4. Age sweet spot. Too new (< ~1 year): no post-publication data, decay unmeasurable, and the authors’ own robustness process may still be running. Too old with heavy citation: probably already replicated — search for existing replications first (the critique search); reading five is cheaper than running one. The sweet spot: 2–6 years old, cited, unreplicated, with post-sample data now available — replication and out-of-sample test in one effort.
5. Suspicion asymmetry. Prefer papers where your prior and the paper’s claim diverge — high rigor score but implausible magnitude, or modest claims from a famously careful group. Divergence is where updating happens. A red-flag-free paper claiming SR 3 is exactly the profile that pays.
Build a replication portfolio, not a pick
Across a quarter, mix three types: one calibration replication (a settled result, to validate your harness and data — expected outcome: match); one decision replication (the strategy or method you might actually adopt — the actionability play); and one audit replication (a load-bearing or suspicious paper — the public-good play, and the source of the field’s scarcest artifact: published replication notes). The mix hedges the portfolio: calibrations always pay something, decisions pay privately, audits pay reputationally — our T-KAN review was an audit replication, and it repriced a result.
Questions to ask before committing a month
- What specific decision changes on success? On failure? (If the answers match, don’t run it.)
- Has someone already done this? (Twenty minutes of critique-searching first, always.)
- What’s the minimum version — one table, one market, one subperiod — that delivers 80% of the information?
- Will I publish the notes either way? (Precommit; failed replications are the ones the field needs most.)
How to use this site for the selection
This is the workflow the archive was built for: filter a topic hub by rigor, cross-reference the research map for load-bearing branch positions, run the ten-minute read on candidates, score them on the five criteria — and when you finish a replication, leave the notes in the paper’s discussion section, where the next selector will find them.
The execution manual: reproduction checklist → · The triage upstream: how to read papers → · backtest evaluation →