{"paper_id":"2403.00009","title":"Randomized Control in Performance Analysis and Empirical Asset Pricing","authors":["Cyril Bachelard","Apostolos Chalkis","Vissarion Fisikopoulos","Elias Tsigaridas"],"paper_date":"2024-02-14","paper_type":"empirical","paper_url":"https://arxiv.org/pdf/2403.00009v1","page_url":"https://thequant.space/flowcharts/randomized-control-in-performance-analysis-and-empirical-ass/","scores":{"empirical_rigor":8,"math_complexity":7.5,"quadrant":"Holy Grail"},"proposes":{"status":"available","statement":"The paper proposes geometric random walks, a class of continuous MCMC methods, to generate random portfolios under high‑dimensional constraints for asset‑pricing and performance analysis.","quote":"Specifically, we introduce geometric random walks, a class of continuous MCMC methods tailored for high-dimensional constrained scenarios from computational geometry, as an instrument to create a randomized control which is useful to evaluate asset pricing anomalies within constrained setups.","summary":"The present article explores the application of randomized control techniques in empirical asset pricing and performance evaluation. It introduces geometric random walks, a class of Markov chain Monte Carlo methods, to construct flexible control groups in the form of random portfolios adhering to…","strategy":null},"assumptions":{"status":"available","items":[{"assumption":"The distribution of no‑skill performance can be approximated through sampling of random portfolios.","type":"statistical","stated":"explicit","quote":"However, it relies on the assumption that the distribution of no-skill performance, crucial for hypothesis system evaluation, can be objectively approximated through sampling."},{"assumption":"Analytical solutions for random‑portfolio performance are generally unavailable, requiring numerical techniques.","type":"model","stated":"explicit","quote":"In most cases of practical relevance, analytical solutions are not feasible, and we must resort to numerical techniques to obtain the desired results."},{"assumption":"Stocks must have a continuous five‑year price history without gaps larger than two weeks.","type":"data","stated":"explicit","quote":"Furthermore, to be included in our study, stocks need a consistent price history of five years without any gaps larger than two weeks."},{"assumption":"Only liquid stocks with median daily trading volume over the prior 252 days above USD 1.5 million are retained.","type":"execution","stated":"explicit","quote":"As a threshold, we require a median trading volume over the previous 252 trading days to be above USD 1.5 million."},{"assumption":"All returns are computed from discrete daily total returns using closing prices in USD.","type":"data","stated":"explicit","quote":"returns using closing prices denoted in USD, covering the period 01.01.1995 to 31.12.2022."}]},"datasets":{"status":"available","items":[{"name":"CRSP daily stock file","provider":"Center for Research in Securities Prices (CRSP)","kind":"market-prices","access":"commercial","period":"1995-01-01 to 2022-12-31","universe":"U.S. large‑ and mid‑cap equities covering ~85% of market cap","quote":"We suggest utilizing data from the Center for Research in Securities Prices (CRSP) as an alternative data source. We select a subset of stocks listed on the NYSE, AMEX, and NASDAQ indexes, focusing solely on ordinary common shares of companies incorporated in the U.S.","known_id":"crsp"},{"name":"MSCI USA index constituents (proxy)","provider":"MSCI (proxied via CRSP filtering)","kind":"benchmark","access":"proprietary","period":null,"universe":"Large and mid‑cap U.S. equities","quote":"we found that a good approximation to mimick the MSCI universe is to filter for the largest stocks listed on the three exchanges such that in aggregate they cover 85% of total market capitalization.","known_id":"crsp"}],"known":[{"id":"jkp","name":"JKP Global Factor Data","provider":"Jensen, Kelly and Pedersen","kind":"market-prices","access":"public","quote":"ies (at the 5% level) except for strategy Q. This finding appears robust to the choice of factors, as evidenced by the qualitative consistency of alphas obtained from a regression on the 13 factor themes suggested by Jensen, Kelly, and Pedersen (JKP) [61].","mentions":6,"used_in":3},{"id":"crsp","name":"CRSP","provider":"Center for Research in Security Prices","kind":"market-prices","access":"commercial","quote":"As the composition of the MSCI USA index is proprietary, we suggest utilizing data from the Center for Research in Securities Prices (CRSP) as an alternative data source.","mentions":3,"used_in":1}],"sample":{"start":null,"end":null,"market":null,"asset_class":null,"frequency":null},"data_resolution":"daily"},"transaction_costs":{"status":"not_extracted","modeled":"unclear","headline_net_of_costs":null,"detail":null,"quote":null,"source":"none","rule":"R0","candidates":{"front_matter":"unclear","evidence_card":"unclear","evidence_card_quoted":false},"meaning":"the classifier was not confident"},"results":{"status":"available","evaluation_type":"trading-pnl","findings":[{"finding":"Random portfolios constructed under realistic constraints are unsuitable for formal hypothesis testing of skill but can be useful for visual and descriptive performance analysis.","kind":"negative","quote":"We explained that commonly employed random portfolios do not offer a statistically acceptable experimental design for performance comparisons and cautioned the reader against such a use case with the aid of an illustrative example."},{"finding":"A backtest of a nonsensical “letter Z” strategy outperformed the S&P 500 by 6.2% annualized, illustrating spurious skill signals from random portfolios.","kind":"performance","quote":"the results show an annualized outperformance of 6.2% (based on daily geometric returns) and the Sharpe ratio test of Ledoit and Wolf"},{"finding":"Factor loadings of the letter‑Z strategies are significant, with positive exposures to size, value, profitability and investment, and negative to momentum, indicating that apparent outperformance is driven by known factor tilts.","kind":"statistical","quote":"one finds that the majority of factor loadings are statistically significant. Coefficients are predominantely positive for size and value as well as for the two quality‑related factors –profitability and investment– while all coefficients for momentum are negative."},{"finding":"The authors provide an open‑source C++/R implementation (GeomScale) that reproduces the MSCI Diversified Multi‑Factor index guidelines and enables reproducible experiments.","kind":"theoretical","quote":"an efficient open‑source implementation in C++ with an interface in R"}],"reported_performance":null,"out_of_sample":{"method":"unclear","headline_out_of_sample":null}},"independent_evidence":{"status":"none_found","supports":[],"contradicts":[],"qualifies":[],"compared_with":14,"citations":{"cited_by_in_corpus":0,"by_independent_authors":0,"how_cited":{"used_as_baseline":0,"uses_method_or_data":0,"extends":0,"discusses_findings":0,"mentions":0,"unclassified":0},"independent_papers":[],"scope":"Citations between papers in this corpus only; citations from outside it are not counted."}},"implementation":{"status":"available","public":false,"code_url":null,"note":"No repository presented by the authors as their own was found in the paper. Absence here is not proof that no code exists."},"replication":{"status":"none_found","card":null,"note":"No replication of this paper has been run by thequant.space yet."},"extraction":{"text_source":"fulltext","model":"openai/gpt-oss-120b","version":1,"extracted_at":"2026-11-14T18:14:53Z"},"provenance":{"claims":"Everything here is what the authors state in the paper. Nothing was re-run or reproduced by the platform.","quotes":"Assumptions, datasets, cost detail and findings were extracted by a language model. Every quote is verbatim paper text. Where an item has `support`, \"full\" means every figure in it appears in the quoted sentences and \"partial\" means a figure is stated elsewhere in the paper; items with a figure found nowhere in the paper were discarded.","datasets_known":"datasets.known lists well-known vendors and public datasets by a stable id, found by a dictionary over the full text where the sentence says the data is used; it is independent of the model and covers only names in the dictionary.","flags":"transaction_costs.modeled, results.evaluation_type and results.out_of_sample.method are platform classifications (structured-decision models); see https://thequant.space/score-guide/#penalty-flags","comparisons":"supports / contradicts / qualifies are a language model's reading of two papers' quoted findings, restricted to closely related papers that share no author. They are pointers to read, not verified replications, and coverage is partial.","implementation":"code_url is a repository the paper itself presents as its own (rule-scored, ambiguous cases judged by a model), checked for liveness when it was recorded."},"docs":"https://thequant.space/agents/"}