Paper: arXiv 2508.13174
Authors: Hongjun Ding, Binqi Chen, Jinsheng Huang, Taian Guo, Zhengyang Mao, Guoyi Shao, Lutong Zou, Luchen Liu, Ming Zhang
Abstract
Formula alpha mining, which generates predictive signals from financial data, is critical for quantitative investment. Although various algorithmic approaches-such as genetic programming, reinforcement learning, and large language models-have significantly expanded the capacity for alpha discovery, systematic evaluation remains a key challenge. Existing evaluation metrics predominantly include backtesting and correlation-based measures. Backtesting is computationally intensive, inherently sequential, and sensitive to specific strategy parameters. Correlation-based metrics, though efficient, assess only predictive ability and overlook other crucial properties such as temporal stability, robustness, diversity, and interpretability. Additionally, the closed-source nature of most existing alpha mining models hinders reproducibility and slows progress in this field. To address these issues, we propose AlphaEval, a unified, parallelizable, and backtest-free evaluation framework for automated alpha mining models. AlphaEval assesses the overall quality of generated alphas along five complementary dimensions: predictive power, stability, robustness to market perturbations, financial logic, and diversity. Extensive experiments across representative alpha mining algorithms demonstrate that AlphaEval achieves evaluation consistency comparable to comprehensive backtesting, while providing more comprehensive insights and higher efficiency. Furthermore, AlphaEval effectively identifies superior alphas compared to traditional single-metric screening approaches. All implementations and evaluation tools are open-sourced to promote reproducibility and community engagement.
Complexity vs Empirical Score
- Math Complexity: 4.0/10
- Empirical Rigor: 6.0/10 (capped from 8.5: transaction costs not modelled; see penalty flags)
- Quadrant: Street Traders — empirical and applied, lower math complexity (effective empirical rigor capped from 8.5)
Why this score: The paper presents a pragmatic framework for alpha evaluation with modest mathematical complexity, but demonstrates high empirical rigor through open-source code, large-scale benchmarks, and specific performance metrics against traditional backtesting.
Research Flowchart
flowchart TD
A["Research Goal: Create AlphaEval<br>to address evaluation challenges in alpha mining"] --> B["Key Methodology: 5-Dimensional<br>Backtest-Free Evaluation Framework"]
B --> C["Computational Process:<br>Parallelizable Assessment Pipeline"]
C --> D["Data/Input: Representative Alpha Mining<br>Algorithms & Diverse Datasets"]
D --> E{"Key Findings & Outcomes"}
E --> F["✅ Evaluation Consistency<br>~ Backtesting"]
E --> G["✅ Higher Efficiency &<br>Comprehensive Insights"]
E --> H["✅ Superior Alpha Identification<br>vs. Single-Metric Screening"]
Evidence Card
What the authors claim, on what data, and who has checked it. Read from the paper's own text by a language model: every quote is verbatim, nothing was re-run. How to read this card
Proposes
The paper proposes AlphaEval, a unified, backtest‑free, parallelizable framework for evaluating automated alpha mining models.
Source sentence
we propose AlphaEval, a unified, parallelizable, and backtest-free evaluation framework
Assumes 1
- The risk‑free rate is assumed to be zero for simplicity. market
Source sentence
The risk-free rate is assumed to be zero for simplicity.
Data
- A‑share dataset market-prices public
Source sentence
We get the A-share dataset and S&P 500 dataset from Qlib. The time ranges for Train/Valid/Test on A-Share dataset are 2010-01-01 – 2019-12-31/2020-01-01 – 2020-12-31/2021-01-01 – 2024-12-31.
- S&P 500 dataset market-prices public
Source sentence
The time ranges for Train/Valid/Test on S&P 500 dataset are 2010-01-01 – 2015-12-31/2016-01-01 – 2016-12-31/2017-01-01 – 2020-12-31.
Qlib datasets
Costs
Unclear - sources disagree
Finds 5
- AlphaEval scores are highly consistent with precision backtesting outcomes while providing broader diagnostic insight and faster evaluation. performance
Source sentence
The results show that AlphaEval scores are highly consistent with precision backtesting outcomes while offering broader diagnostic insight and significantly faster evaluation.
- AlphaEval demonstrates superior alpha selection performance compared to conventional single‑metric filtering approaches such as IC alone. performance
Source sentence
AlphaEval demonstrates superior alpha selection performance compared to conventional single-metric filtering approaches (e.g., by IC alone).
- AlphaEval reduces relative evaluation time by more than 25% thanks to its backtest‑free, parallelizable design. performance
Source sentence
AlphaEval achieves a significant speedup, reducing relative evaluation time by more than 25%.
- Relative Rank Entropy (RRE) exhibits a strong negative correlation with annualized turnover, indicating more stable alphas lead to lower trading activity. statistical
Source sentence
RRE exhibits a strong and statistically significant negative linear correlation with turnover.
- Alphas with Perturbation Fidelity Score (PFS) ≥ 0.9 show significantly lower maximum drawdown, confirming PFS as a reliable risk‑screening metric. statistical
Source sentence
Alphas with higher Perturbation Fidelity Score (PFS ≥0.9) show significantly lower maximum drawdown (MaxDD), indicating greater robustness to market perturbations.
Checked by others
No other paper in the archive has been found to support or contradict these findings (compared with 29 related papers). Coverage is partial, so this is neither replication nor refutation.
5 papers in the archive
Extracted 2026-11-14 · same card as JSON · A language model chose these items: check the source sentence before relying on one.
Cited References (55)
- LightGBM : A Highly Efficient Gradient Boosting Decision Tree (2017) — link
- XGBoost : A Scalable Tree Boosting System (2016) — link
- AlphaEvolve : A Learning Framework to Discover Novel Alphas in Quantitative Investment (2021) — link
- AutoAlpha : an Efficient Hierarchical Evolutionary Algorithm for Mining Alpha Factors in Quantitative Investment (2020) — link
- Alphaforge: A framework to mine and dynamically combine formulaic alpha factors — → 2406.18394
- Can Large Language Models Mine Interpretable Financial Factors More Effectively? A Neural-Symbolic Factor Mining Agent Model (2024) — link
- AlphaQCM: Alpha Discovery in Finance with Distributional Reinforcement Learning (2025)
- A five-factor asset pricing model (2015)
and 47 more
- An empirical study of financial factor mining based on gene expression programming (2021)
- 101 formulaic alphas (2016)
- Qlib: An AI-oriented Quantitative Investment Platform (2020) — link
- $ \textbackslash text\ Alpha \ \textasciicircum 2$: Discovering Logical Formulaic Alphas using Deep Reinforcement Learning (2024) — link
- RiskMiner : Discovering Formulaic Alphas via Risk Seeking Monte Carlo Tree Search — → 2402.07080
- QuantFactor REINFORCE : Mining Steady Formulaic Alpha Factors with Variance -bounded REINFORCE — → 2409.05144
- Beyond the Black Box: Interpretability of LLMs in Finance — → 2505.24650
- Stop Explaining Black Box Machine Learning Models for High Stakes Decisions and Use Interpretable Models Instead (2019) — link
- Explainable Artificial Intelligence (XAI) approaches for transparency and accountability in financial decision-making (2023)
- Explainable AI (XAI) for Credit Scoring and Loan Approvals (2025)
- Enhancing explainability in real-world scenarios: towards a robust stability measure for local interpretability (2025) — link
- Can i trust the explanations? investigating explainable machine learning methods for monotonic models — → 2309.13246
- Cumulated gain-based evaluation of IR techniques (2002)
- A Theoretical Analysis of NDCG Type Ranking Measures (2013) — link
- QuantBench: Benchmarking AI Methods for Quantitative Investment — → 2504.18600
- Learning gflownets from partial episodes for improved convergence and stability (2023)
- A theory of continuous generative flow networks (2023)
- Let the flows tell: Solving graph combinatorial problems with gflownets (2023)
- Biological sequence design with gflownets (2022)
- Markov chain monte carlo and variational inference: Bridging the gap (2015)
- A tutorial on energy-based learning (2006)
- Open graph benchmark: Datasets for machine learning on graphs (2020)
- DistDGL: Distributed graph neural network training for billion-scale graphs (2020)
- Cluster-gcn: An efficient algorithm for training deep and large graph convolutional networks (2019)
- Distance encoding: Design provably more powerful neural networks for graph representation learning (2020)
- Geometric deep learning: Grids, groups, graphs, geodesics, and gauges (2021) — link
- Neural message passing for quantum chemistry (2017)
- How Powerful are Graph Neural Networks?
- Graph attention networks (2017) — link
- Inductive representation learning on large graphs (2017)
- Multilayer perceptrons for classification and regression (1991)
- Graph convolutional networks for text classification (2019)
- The graph neural network model (2008)
- Flow network based generative models for non-iterative diverse candidate generation (2021)
- Trajectory balance: Improved credit assignment in gflownets (2022)
- Gflownet foundations (2023)
- Modeling relational data with graph convolutional networks (2018)
- Learning from Expert Factors: Trajectory-level Reward Shaping for Formulaic Alpha Mining — → 2507.20263
- 101 Formulaic Alphas (2016) — link
- R&D-Agent-Quant: A Multi-Agent Framework for Data-Centric Factors and Model Joint Optimization — → 2505.15155
- AlphaEval: A Comprehensive and Efficient Evaluation Framework for Formula Alpha Mining (2025) — link
- Navigating the Alpha Jungle: An LLM-Powered MCTS Framework for Formulaic Factor Mining (2025) — link
- Alphaagent: Llm-driven alpha mining with regularized exploration to counteract alpha decay (2025)
- Chain-of-Alpha: Unleashing the Power of Large Language Models for Alpha Mining in Quantitative Trading (2025) — link
- Generating Synergistic Formulaic Alpha Collections via Reinforcement Learning — → 2306.12964
- An empirical study of financial factor mining based on gene expression programming (2021)
- From Linear to Hierarchical : Evolving Tree -structured Thoughts for Efficient Alpha Mining (2025) — link
9 of 55 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2596 linked papers.