Definition. A factor-investing paper claims that a characteristic — value, momentum, quality, or one of several hundred published successors — predicts the cross-section of returns: sort assets by it, go long the top bucket and short the bottom, and earn a premium. The genre is the most published and least replicable in empirical finance, which makes the generic backtest checks necessary but not sufficient. These are the factor-specific ones.
The worked example: construction is the hidden factor
The same signal can produce a t-stat of 1 or 4 depending on choices that rarely make the abstract. A characteristic sorted into deciles, equal-weighted, including micro-caps, rebalanced monthly, 1963–2000 might show 8%/yr of spread; the identical characteristic in quintiles, value-weighted, NYSE-breakpoints, ex-micro-caps often shows a third of that. Nothing was “wrong” — each cell in that grid is a separate trial, and the paper you’re reading is the cell that worked. This is why the multiple-testing hurdle for new factors sits near t ≈ 3.0, not 2.0.
The factor-paper checklist
- ☐ Weighting and breakpoints: value-weighted results with NYSE breakpoints reported? Equal-weighted deciles are a micro-cap bet wearing a factor costume — and micro-caps are where survivorship and untradeable spreads concentrate.
- ☐ The long and short legs separated: many premia live entirely in the short leg — expensive to borrow, sometimes impossible. A “premium” you can only earn by shorting unborrowable names is a museum piece.
- ☐ Post-formation lag: characteristic measured with announcement-date information only? Accounting signals used at fiscal period-end are leaking.
- ☐ Turnover and capacity stated: monthly-rebalanced factors can turn over 100–400%/yr; multiply by realistic costs before believing net claims.
- ☐ Subperiods, including post-2003: decimalization, and later the publication of the factor itself, mark regime lines. A factor alive only pre-2000 is history, not a strategy — the decay literature is unambiguous.
- ☐ Spanning tests: is the new factor priced after controlling for the standard set, or is it old exposure with a new name?
Questions to ask while reading
- How many characteristic/construction combinations does the paper’s own appendix reveal were examined?
- Would the result survive value-weighting minus micro-caps? (If the paper doesn’t show it, assume no.)
- What is the net-of-cost premium at the fund size the authors imply?
- Does the factor exist in international samples the authors didn’t optimize on?
- If published five-plus years ago: what happened after publication?
What our scoring captures — and misses
The rigor axis rewards exactly what this genre most often lacks: cost accounting, out-of-sample sections, and disclosed robustness grids, so high-rigor factor papers are disproportionately trustworthy. What the score cannot see is the unpublished search space — a clean-looking paper that quietly tried 200 constructions scores the same as one that pre-registered a single design. That correction is yours to apply, via the deflated Sharpe ratio with an honest N; our score is a floor on quality, not a ceiling on skepticism.
Companions: Interpreting a factor-zoo paper → · What makes a factor tradable? → · The backtest checklist →