Scoring Methodology

To navigate the high-volume landscape of quantitative finance, thequant.space utilizes a dual-axis scoring framework. This system decomposes complex ArXiv research into two primary dimensions: Theoretical Complexity and Empirical Rigor.


The Classification Framework

1. Math Complexity

This metric quantifies the “theoretical overhead” required to parse the paper. We analyze the density of mathematical notation and the sophistication of the underlying machinery.

  • High Score (7–10): Heavy reliance on Stochastic Calculus, Partial Differential Equations (PDEs), bespoke optimization proofs, or advanced topology.
  • Low Score (1–3): Primarily descriptive statistics, high-level conceptual frameworks, or basic linear algebra.

2. Empirical Rigor

This metric assesses the “path to implementation.” We look for signals that the strategy has been stress-tested against the realities of the market.

  • High Score (7–10): Extensive backtesting on high-fidelity data (Tick-level, WRDS, Bloomberg), clear mention of transaction costs, and rigorous out-of-sample validation.
  • Low Score (1–3): Use of “toy” datasets, synthetic data, or purely logical derivations without historical verification.

Penalty flags

Practitioners trust a backtest only as far as its execution assumptions go, so every empirical paper page carries four flags above the summary:

FlagValuesWhat it means
Transaction costsModeled · Partial · Not modeledModeled: an explicit cost model (commission, spread or slippage in bps, or a market-impact model) is applied to the reported results. Partial: costs are only discussed, tested in a sensitivity or break-even table, or a fee is charged without spread or slippage for a high-turnover strategy.
Out-of-sampleHold-out · Walk-forward · Cross-validation · NoneHow the reported results were validated on data the model never saw.
DataL2 order book · Tick · Intraday bars · Daily · Weekly / monthly · Synthetic onlyThe finest resolution of market data used in the evaluation.
Codeavailable · not foundThe authors’ own public repository, checked to be live (papers with code).

The cap. Empirical Rigor is capped at 6.0 when a paper reports trading or execution results without explicitly modelling transaction costs, or reports trading, execution or forecasting results without any out-of-sample test. The page then shows the capped score, the score before the cap, and the reason. The cap never applies to theory, surveys, pricing or calibration papers, or to descriptive empirical studies.

Claims that are too good. The same 6.0 cap applies when a paper’s own numbers should not be taken at face value: a reported Sharpe ratio above 3 on daily or lower-frequency data, or headline trading or forecasting results on data the paper itself says carries survivorship or look-ahead bias. Order-book, tick and intraday studies are exempt from the Sharpe test, because high ratios are ordinary at those horizons. The figures are the authors’ claims as extracted from the paper, so a cap of this kind means “not yet believable without independent confirmation”, not “wrong”.

How the flags are set. A structured-decision model reads the paper’s data, method, results and conclusion sections and answers each question with a calibrated probability. When it isn’t sure, a second model is asked. A flag the models still aren’t confident about is shown as Unclear, and an unclear flag never triggers the cap. The setup was chosen on a hand-checked set of 40 papers, where it capped no paper wrongly. Flags are automated, so if one is wrong, say so in the comments on the paper’s page.


Strategic Quadrants

By plotting these scores (split at 5.0 on each 0–10 axis using the effective, post-cap Empirical Rigor and Math Complexity), we group papers into four research profiles:

QuadrantDesignationResearch Profile
Math ≥ 5 & Rigor ≥ 5Holy GrailHigh mathematical complexity paired with stronger empirical evaluation (data, benchmarks, and out-of-sample checks).
Math ≥ 5 & Rigor < 5Lab RatsMathematically intensive theory or modeling with limited empirical validation in the paper.
Math < 5 & Rigor ≥ 5Street TradersEmpirical and applied work with lighter mathematical machinery.
Math < 5 & Rigor < 5PhilosophersConceptual discussion, surveys, or framing pieces with lower mathematical and empirical scores.

What scores do not measure: These scores and quadrant labels classify theoretical overhead and empirical methodology only. They do not measure expected profitability, tradable alpha, economic significance, strategy capacity, execution viability, or production readiness. Where a methodological penalty applies (such as an unmodeled-cost or missing-OOS cap), Empirical Rigor is capped at 6.0 and the effective score determines the paper’s quadrant and ranking.


Current Library Distribution

QuadrantPapersShare
Holy Grail2,50044%
Lab Rats1,83832%
Street Traders92216%
Philosophers4768%

The Strategic Map

Color up to 3 topics:

Each dot is one paper. Scores come in half-point steps, so dots are spread slightly within their cell; hover shows the exact score and how many papers share it. Click a dot to open the paper.

Reading the map: Papers in Lab Rats present formal models that have not yet been tested empirically in the source paper; papers in Street Traders focus on empirical tests using simpler mathematical tools. Always inspect the Evidence Card and penalty flags before relying on any reported performance figure.


Disclaimer: Scoring is performed via automated semantic analysis. If you believe a paper has been misclassified, please open an issue.