AlphaRJM: Reward-Jump Memory for Stochastic Return-Guided Alpha Discovery

Paper: arXiv 2609.08581

Authors: Sayan Dhan, Selvaraju Natarajan

Abstract

Formulaic alpha discovery is a pool-dependent symbolic search problem in which informative feedback is observed primarily when a complete expression is evaluated. This delayed feedback creates two coupled difficulties: the retained alpha pool does not preserve the full history of realized evaluation feedback, and the value of an intermediate construction action is uncertain because its consequence depends on the formula eventually completed. We introduce AlphaRJM, which addresses these difficulties through Reward-Jump Memory, an event-driven latent state that remains fixed during token construction and updates only at terminal evaluation events using the realized pool reward and evaluation outcome, and an action-conditioned SDE return critic that represents future discounted discovery returns with stochastic particles. The particles guide action selection through their mean and uncertainty and are learned using a distributional Bellman objective combining energy-distance matching, mean calibration, and jump regularization. Empirically, AlphaRJM delivers strong and stable gains across multiple equity universes, forecasting horizons, and random seeds, while ablations confirm the complementary roles of persistent evaluation history, stochastic return modeling, and distributional supervision.

Complexity vs Empirical Score

• Math Complexity: 8.0/10
• Empirical Rigor: 7.0/10
• Quadrant: Holy Grail — high math complexity, high empirical rigor

Why this score: This paper presents a highly mathematical approach to alpha discovery, incorporating SDEs and distributional Bellman objectives. It also demonstrates strong empirical results across multiple equity universes, suggesting a robust methodology. The novelty lies in the Reward-Jump Memory and SDE return critic for symbolic search.

Research Flowchart

  flowchart TD
    A[Research Goal: Formulaic Alpha Discovery] --> B{Challenges: Delayed Feedback & Uncertain Intermediate Value};
    B --> C[AlphaRJM Methodology: Reward-Jump Memory (RJM) & Action-Conditioned SDE Return Critic];
    C --> D[Data/Inputs: Equity Universes, Forecasting Horizons, Random Seeds];
    D --> E{Computational Process: Token Construction, Terminal Evaluation, Distributional Bellman Objective};
    E --> F[Key Findings: Strong & Stable Gains Across Universes/Horizons/Seeds];
    F --> G[Outcomes: RJM, Stochastic Return Modeling, Distributional Supervision are Complementary];

Cited References (19)

1. Generating synergistic formulaic alpha collections via reinforcement learning — → 2306.12964
2. Alphaqcm: Alpha discovery in finance with distributional reinforcement learning (2025)
3. Alphasage: Structure-aware alpha mining via gflownets for robust exploration (2026)
4. A distributional perspective on reinforcement learning (2017)
5. Implicit quantile networks for distributional reinforcement learning (2018)
6. Quantiled conditional variance, skewness, and kurtosis by Cornish–Fisher expansion (2026)
7. Flow network based generative models for non-iterative diverse candidate generation (2021)
8. Trajectory balance: Improved credit assignment in gflownets (2022)

1 of 19 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2518 linked papers.