Paper: arXiv 2508.13174

Authors: Hongjun Ding, Binqi Chen, Jinsheng Huang, Taian Guo, Zhengyang Mao, Guoyi Shao, Lutong Zou, Luchen Liu, Ming Zhang

Abstract

Formula alpha mining, which generates predictive signals from financial data, is critical for quantitative investment. Although various algorithmic approaches-such as genetic programming, reinforcement learning, and large language models-have significantly expanded the capacity for alpha discovery, systematic evaluation remains a key challenge. Existing evaluation metrics predominantly include backtesting and correlation-based measures. Backtesting is computationally intensive, inherently sequential, and sensitive to specific strategy parameters. Correlation-based metrics, though efficient, assess only predictive ability and overlook other crucial properties such as temporal stability, robustness, diversity, and interpretability. Additionally, the closed-source nature of most existing alpha mining models hinders reproducibility and slows progress in this field. To address these issues, we propose AlphaEval, a unified, parallelizable, and backtest-free evaluation framework for automated alpha mining models. AlphaEval assesses the overall quality of generated alphas along five complementary dimensions: predictive power, stability, robustness to market perturbations, financial logic, and diversity. Extensive experiments across representative alpha mining algorithms demonstrate that AlphaEval achieves evaluation consistency comparable to comprehensive backtesting, while providing more comprehensive insights and higher efficiency. Furthermore, AlphaEval effectively identifies superior alphas compared to traditional single-metric screening approaches. All implementations and evaluation tools are open-sourced to promote reproducibility and community engagement.

Complexity vs Empirical Score

  • Math Complexity: 4.0/10
  • Empirical Rigor: 6.0/10 (capped from 8.5: transaction costs not modelled; see penalty flags)
  • Quadrant: Street Traders — empirical and applied, lower math complexity (effective empirical rigor capped from 8.5)

Why this score: The paper presents a pragmatic framework for alpha evaluation with modest mathematical complexity, but demonstrates high empirical rigor through open-source code, large-scale benchmarks, and specific performance metrics against traditional backtesting.

Research Flowchart

  flowchart TD
  A["Research Goal: Create AlphaEval<br>to address evaluation challenges in alpha mining"] --> B["Key Methodology: 5-Dimensional<br>Backtest-Free Evaluation Framework"]
  B --> C["Computational Process:<br>Parallelizable Assessment Pipeline"]
  C --> D["Data/Input: Representative Alpha Mining<br>Algorithms & Diverse Datasets"]
  D --> E{"Key Findings & Outcomes"}
  E --> F["✅ Evaluation Consistency<br>~ Backtesting"]
  E --> G["✅ Higher Efficiency &<br>Comprehensive Insights"]
  E --> H["✅ Superior Alpha Identification<br>vs. Single-Metric Screening"]

Evidence Card

What the authors claim, on what data, and who has checked it. Read from the paper's own text by a language model: every quote is verbatim, nothing was re-run. How to read this card

Proposes

The paper proposes AlphaEval, a unified, backtest‑free, parallelizable framework for evaluating automated alpha mining models.

Source sentence
we propose AlphaEval, a unified, parallelizable, and backtest-free evaluation framework

Assumes 1

  • The risk‑free rate is assumed to be zero for simplicity. market
    Source sentence
    The risk-free rate is assumed to be zero for simplicity.

Data

  • A‑share dataset · Qlib market-prices public
    2010-01-01 – 2024-12-31 (train/valid/tes
    Source sentence
    We get the A-share dataset and S&P 500 dataset from Qlib. The time ranges for Train/Valid/Test on A-Share dataset are 2010-01-01 – 2019-12-31/2020-01-01 – 2020-12-31/2021-01-01 – 2024-12-31.
  • S&P 500 dataset · Qlib market-prices public
    2010-01-01 – 2020-12-31 (train/valid/tes
    Source sentence
    The time ranges for Train/Valid/Test on S&P 500 dataset are 2010-01-01 – 2015-12-31/2016-01-01 – 2016-12-31/2017-01-01 – 2020-12-31.

Recognised sources: Qlib datasets

Costs

Unclear - sources disagree The platform classifier and the paper-text extraction give different answers, so no value is asserted.

Platform classifier: Not modeled

Paper text extraction: Not applicable

Finds 5

  • AlphaEval scores are highly consistent with precision backtesting outcomes while providing broader diagnostic insight and faster evaluation. performance
    Source sentence
    The results show that AlphaEval scores are highly consistent with precision backtesting outcomes while offering broader diagnostic insight and significantly faster evaluation.
  • AlphaEval demonstrates superior alpha selection performance compared to conventional single‑metric filtering approaches such as IC alone. performance
    Source sentence
    AlphaEval demonstrates superior alpha selection performance compared to conventional single-metric filtering approaches (e.g., by IC alone).
  • AlphaEval reduces relative evaluation time by more than 25% thanks to its backtest‑free, parallelizable design. performance
    Source sentence
    AlphaEval achieves a significant speedup, reducing relative evaluation time by more than 25%.
  • Relative Rank Entropy (RRE) exhibits a strong negative correlation with annualized turnover, indicating more stable alphas lead to lower trading activity. statistical
    Source sentence
    RRE exhibits a strong and statistically significant negative linear correlation with turnover.
  • Alphas with Perturbation Fidelity Score (PFS) ≥ 0.9 show significantly lower maximum drawdown, confirming PFS as a reliable risk‑screening metric. statistical
    Source sentence
    Alphas with higher Perturbation Fidelity Score (PFS ≥0.9) show significantly lower maximum drawdown (MaxDD), indicating greater robustness to market perturbations.

Checked by others

No other paper in the archive has been found to support or contradict these findings (compared with 29 related papers). Coverage is partial, so this is neither replication nor refutation.

Cited by 5 papers in the archive (3 by independent authors)

Code

https://github.com/dhj74/AlphaEval — the authors' own repository

Extracted 2026-11-14 · same card as JSON · A language model chose these items: check the source sentence before relying on one.

Cited References (55)

  1. LightGBM : A Highly Efficient Gradient Boosting Decision Tree (2017) — link
  2. XGBoost : A Scalable Tree Boosting System (2016) — link
  3. AlphaEvolve : A Learning Framework to Discover Novel Alphas in Quantitative Investment (2021) — link
  4. AutoAlpha : an Efficient Hierarchical Evolutionary Algorithm for Mining Alpha Factors in Quantitative Investment (2020) — link
  5. Alphaforge: A framework to mine and dynamically combine formulaic alpha factors — → 2406.18394
  6. Can Large Language Models Mine Interpretable Financial Factors More Effectively? A Neural-Symbolic Factor Mining Agent Model (2024) — link
  7. AlphaQCM: Alpha Discovery in Finance with Distributional Reinforcement Learning (2025)
  8. A five-factor asset pricing model (2015)
and 47 more
  1. An empirical study of financial factor mining based on gene expression programming (2021)
  2. 101 formulaic alphas (2016)
  3. Qlib: An AI-oriented Quantitative Investment Platform (2020) — link
  4. $ \textbackslash text\ Alpha \ \textasciicircum 2$: Discovering Logical Formulaic Alphas using Deep Reinforcement Learning (2024) — link
  5. RiskMiner : Discovering Formulaic Alphas via Risk Seeking Monte Carlo Tree Search — → 2402.07080
  6. QuantFactor REINFORCE : Mining Steady Formulaic Alpha Factors with Variance -bounded REINFORCE — → 2409.05144
  7. Beyond the Black Box: Interpretability of LLMs in Finance — → 2505.24650
  8. Stop Explaining Black Box Machine Learning Models for High Stakes Decisions and Use Interpretable Models Instead (2019) — link
  9. Explainable Artificial Intelligence (XAI) approaches for transparency and accountability in financial decision-making (2023)
  10. Explainable AI (XAI) for Credit Scoring and Loan Approvals (2025)
  11. Enhancing explainability in real-world scenarios: towards a robust stability measure for local interpretability (2025) — link
  12. Can i trust the explanations? investigating explainable machine learning methods for monotonic models — → 2309.13246
  13. Cumulated gain-based evaluation of IR techniques (2002)
  14. A Theoretical Analysis of NDCG Type Ranking Measures (2013) — link
  15. QuantBench: Benchmarking AI Methods for Quantitative Investment — → 2504.18600
  16. Learning gflownets from partial episodes for improved convergence and stability (2023)
  17. A theory of continuous generative flow networks (2023)
  18. Let the flows tell: Solving graph combinatorial problems with gflownets (2023)
  19. Biological sequence design with gflownets (2022)
  20. Markov chain monte carlo and variational inference: Bridging the gap (2015)
  21. A tutorial on energy-based learning (2006)
  22. Open graph benchmark: Datasets for machine learning on graphs (2020)
  23. DistDGL: Distributed graph neural network training for billion-scale graphs (2020)
  24. Cluster-gcn: An efficient algorithm for training deep and large graph convolutional networks (2019)
  25. Distance encoding: Design provably more powerful neural networks for graph representation learning (2020)
  26. Geometric deep learning: Grids, groups, graphs, geodesics, and gauges (2021) — link
  27. Neural message passing for quantum chemistry (2017)
  28. How Powerful are Graph Neural Networks?
  29. Graph attention networks (2017) — link
  30. Inductive representation learning on large graphs (2017)
  31. Multilayer perceptrons for classification and regression (1991)
  32. Graph convolutional networks for text classification (2019)
  33. The graph neural network model (2008)
  34. Flow network based generative models for non-iterative diverse candidate generation (2021)
  35. Trajectory balance: Improved credit assignment in gflownets (2022)
  36. Gflownet foundations (2023)
  37. Modeling relational data with graph convolutional networks (2018)
  38. Learning from Expert Factors: Trajectory-level Reward Shaping for Formulaic Alpha Mining — → 2507.20263
  39. 101 Formulaic Alphas (2016) — link
  40. R&D-Agent-Quant: A Multi-Agent Framework for Data-Centric Factors and Model Joint Optimization — → 2505.15155
  41. AlphaEval: A Comprehensive and Efficient Evaluation Framework for Formula Alpha Mining (2025) — link
  42. Navigating the Alpha Jungle: An LLM-Powered MCTS Framework for Formulaic Factor Mining (2025) — link
  43. Alphaagent: Llm-driven alpha mining with regularized exploration to counteract alpha decay (2025)
  44. Chain-of-Alpha: Unleashing the Power of Large Language Models for Alpha Mining in Quantitative Trading (2025) — link
  45. Generating Synergistic Formulaic Alpha Collections via Reinforcement Learning — → 2306.12964
  46. An empirical study of financial factor mining based on gene expression programming (2021)
  47. From Linear to Hierarchical : Evolving Tree -structured Thoughts for Efficient Alpha Mining (2025) — link

9 of 55 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2596 linked papers.