Paper: arXiv 2509.01393

Authors: Qizhao Chen, Hiroaki Kawashima

Abstract

This paper proposes a reinforcement learning framework that employs Proximal Policy Optimization (PPO) to dynamically optimize the weights of multiple large language model (LLM)-generated formulaic alphas for stock trading strategies. Formulaic alphas are mathematically defined trading signals derived from price, volume, sentiment, and other data. Although recent studies have shown that LLMs can generate diverse and effective alphas, a critical challenge lies in how to adaptively integrate them under varying market conditions. To address this gap, we leverage the deepseek-r1-distill-llama-70b model to generate fifty alphas for five major stocks: Apple, HSBC, Pepsi, Toyota, and Tencent, and then use PPO to adjust their weights in real time. Experimental results demonstrate that the PPO-optimized strategy achieves strong returns and high Sharpe ratios across most stocks, outperforming both an equal-weighted alpha portfolio and traditional benchmarks such as the Nikkei 225, S&P 500, and Hang Seng Index. The findings highlight the importance of reinforcement learning in the allocation of alpha weights and show the potential of combining LLM-generated signals with adaptive optimization for robust financial forecasting and trading.

Complexity vs Empirical Score

  • Math Complexity: 7.5/10
  • Empirical Rigor: 8.0/10
  • Quadrant: Holy Grail — high math complexity, high empirical rigor

Why this score: The paper employs advanced reinforcement learning (PPO) and deals with mathematical formulation of alpha signals, indicating moderate to high math complexity, while the experimental setup with specific stock tickers, Sharpe ratios, and comparisons to benchmarks demonstrates strong empirical rigor.

Research Flowchart

  flowchart TD
  A["Research Goal: <br>Adaptive Alpha Weighting"] --> B["Data: 50 LLM-Generated Alphas<br>5 Stocks: Apple, HSBC, Pepsi, Toyota, Tencent"]
  B --> C["Core Methodology:<br>Proximal Policy Optimization PPO"]
  C --> D{"Compute Portfolio Weighted by<br>PPO-Optimized Alpha Weights"}
  D --> E["Execution: <br>Trading Strategy Backtest"]
  E --> F["Outcome: <br>Higher Returns & Sharpe Ratios"]
  F --> G["Comparison: <br>Outperforms Equal-Weight & Benchmarks S&P 500, Nikkei 225"]

Evidence Card

What the authors claim, on what data, and who has checked it. Read from the paper's own text by a language model: every quote is verbatim, nothing was re-run. How to read this card

Proposes

The paper proposes a reinforcement learning framework using Proximal Policy Optimization to dynamically weight multiple LLM‑generated formulaic alphas for stock trading.

Source sentence
This paper introduces a reinforcement learning framework that employs Proximal Policy Optimization (PPO) to dynamically optimize the weights of multiple large language model (LLM)-generated formulaic alphas for stock trading strategies.

Assumes 6

  • Trades are executed at the closing price of day t with no information beyond day t. execution
    Source sentence
    at the close price of day t, and no information beyond day t is used when determining trading positions.
  • Positions are held from day t to day t+1 and returns are realized using the next‑day return. execution
    Source sentence
    ... Trades are assumed to be executed immediately after the decision at day t, and the resulting position pt is held over the interval from day t to day t + 1.
  • Alpha signals are normalized to approximately a standard normal distribution. model
    Source sentence
    αi,t Alpha signals Normalized to ∼N(0, 1)
  • A soft penalty is applied when the position sign conflicts with a regime indicator defined by moving‑average crossover. market
    Source sentence
    The market regime indicator regimet ∈{0, 1} is defined based on the moving-average crossover, where regimet = 1 denotes a bullish regime (20-day MA above 100-day MA) and regimet = 0 indicates a bearish regime.
Show 2 more
  • Transaction costs are a fixed 0.1% per trade. execution
    Source sentence
    TCt Transaction cost per trade 0.001
  • Volatility targeting scales positions to keep annualized volatility near 15%. model
    Source sentence
    σtarget Annualized volatility target 0.15 (fixed)

Data

Backtest sample: US stocks · equities · daily

  • Ten selected stocks market-prices unknown
    ten stocks (e.g., Apple, HSBC, Pepsi, etc.)
    Source sentence
    we leverage a DeepSeek model to generate fifty alphas for ten stocks

Recognised sources: Yahoo Finance

Costs

Modeled Costs are part of the reported headline results.

Fixed cost of 0.001 (0.1%) per trade is included in the reward calculation.

Source sentence
TCt Transaction cost per trade 0.001

Finds 5

Reported performance (PPO-optimized LLM-generated alpha portfolio): Sharpe 2.41 · max drawdown 1.0%

  • The PPO‑adjusted strategy does not consistently achieve the highest cumulative returns but attains higher Sharpe ratios and lower maximum drawdowns in most cases. performance
    Source sentence
    Experimental results indicate that the PPO‑optimized strategy does not consistently deliver the highest cumulative returns across all stocks, but it achieves comparatively higher Sharpe ratios and smaller maximum drawdowns in most cases.
  • Sharpe ratio improvements over buy‑and‑hold are statistically significant for several stocks (e.g., Apple +1.77, HSBC +0.79). performance
    Source sentence
    Apple 1.7695* [0.1787, 3.4424] 0.0296 ... HSBC 0.7928* [0.4321, 2.2641] 0.0006
  • Diebold‑Mariano tests show negative DM statistics, indicating PPO outperforms baseline strategies for many stocks. statistical
    Source sentence
    A negative DM stat indicates PPO outperforms the benchmark.
  • Average daily turnover remains low for most stocks (e.g., Netflix 0.0090) suggesting limited impact of transaction costs. performance
    Source sentence
    Average daily turnover remains low for several stocks, including Netflix (0.0090) and HSBC (0.0348), and reaches its highest level for Airbus (0.2007).
  • Ablation analysis shows that reducing the number of alphas can slightly improve cumulative return and Sharpe for some stocks but may worsen performance for others. other
    Source sentence
    reducing the number of alphas leads to slight improvements in both cumulative return and Sharpe ratio for certain stocks. However, for others, their performance may worsen after reducing the number of alphas.

Checked by others

No other paper in the archive has been found to support or contradict these findings (compared with 57 related papers). Coverage is partial, so this is neither replication nor refutation.

Code

No repository by the authors found in the paper.

Extracted 2026-11-14 · same card as JSON · A language model chose these items: check the source sentence before relying on one.

Cited References (55)

  1. Technical Analysis Indicators in Stock Market Using Machine Learning: A Comparative Analysis (2021) — link
  2. Key technical indicators for stock market prediction (2025) — link
  3. Machine learning approaches in stock market prediction: A systematic literature review (2023) — link
  4. Interpretability of Machine Learning: Recent Advances and Future Prospects (2023) — link
  5. Automate Strategy Finding with LLM in Quant investment — → 2409.06289
  6. Stock Price Prediction Using LLM-Based Sentiment Analysis (2024) — link
  7. Common risk factors in the returns on stocks and bonds (1993) — link
  8. Short and long term stock trend prediction using decision tree (2017) — link
and 47 more
  1. Stocks market prediction using Support Vector Machine (2013) — link
  2. Stock Price Prediction Model Based on RBF-SVM Algorithm (2020) — link
  3. Research on stock forecasting based on random forest (2022) — link
  4. Stock Price Prediction Using CNN and LSTM-Based Deep Learning Models (2020) — link
  5. Stock Price Prediction using LSTM (2023) — link
  6. A CNN-LSTM-Based Model to Forecast Stock Prices (2020) — link
  7. Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting (2021) — link
  8. Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecasting (2022) — link
  9. TimeXer: Empowering Transformers for Time Series Forecasting with Exogenous Variables (2024) — link
  10. MASTER: Market-Guided Stock Transformer for Stock Price Forecasting (2023) — link
  11. FinGPT: Open-Source Financial Large Language Models — → 2306.06031
  12. PIXIU: A Large Language Model, Instruction Data and Evaluation Benchmark for Finance (2023) — link
  13. Automate Strategy Finding with LLM in Quant Investment — → 2409.06289
  14. GPT-Signal: Generative AI for Semi-automated Feature Engineering in the Alpha Research Process (2024) — link
  15. Proximal Policy Optimization Algorithms (2017) — link
  16. 101 Formulaic Alphas (2016) — link
  17. The unreasonable effectiveness of deep learning in artificial intelligence (2020) — link
  18. The alpha life cycle of quantitative strategy (2017) — link
  19. Stock Price Change Prediction Using Prompt-Based LLM s with RL -Enhanced Post-Hoc Adjustments (2025) — link
  20. A Deep Reinforcement Learning Framework for Dynamic Portfolio Optimization: Evidence from China’s Stock Market — → 2412.18563
  21. Advancing Investment Frontiers: Industry-grade Deep Reinforcement Learning for Portfolio Optimization (2024) — link
  22. Adaptive Investment Strategies: Deep Reinforcement Learning Approaches for Portfolio Optimization (2024) — link
  23. Distilling the Knowledge in a Neural Network (2015) — link
  24. Contrarian Investment Strategies: The Next Generation (1998)
  25. From local explanations to global understanding with explainable AI for trees (2020)
  26. A novel sentiment correlation-based method with dual transformer model for stock price prediction (2025) — link
  27. Sentiment-Aware Stock Price Prediction with Transformer and LLM-Generated Formulaic Alpha (2025) — link
  28. An end-to-end optimal trade execution framework based on proximal policy optimization (2021)
  29. Comparative Analysis of A3C and PPO Algorithms in Reinforcement Learning: A Survey on General Environments (2024) — link
  30. Proximal policy optimization via enhanced exploration efficiency (2022) — link
  31. Reinforcement Learning in a Virtual World: A Study of PPO and SAC within Unity ML-Agents (2025) — link
  32. Stock Price Prediction with LLM -Guided Market Movement Signals and Transformer Model (2025) — link
  33. Image-Driven Stock Price Prediction with LLaMA : A Prompt-Based Approach (2025) — link
  34. Explore Anomaly-Aware Transformers for Robust Financial Time Series Forecasting (2025) — link
  35. A Framework for Measuring How News Topics Drive Stock Movement (2025) — link
  36. Offline Reinforcement Learning for Automated Stock Trading (2023) — link
  37. Enhancing Deep Reinforcement Learning for Stock Trading: A Reward Shaping Approach via Expert Feedback (2025) — link
  38. Stock Market Trading via Actor–Critic Reinforcement Learning and Adaptable Data Structure (2025) — link
  39. A Novel RMS-Driven Deep Reinforcement Learning for Optimized Portfolio Management in Stock Trading (2025) — link
  40. Stock Closing Price Prediction using Machine Learning Techniques (2020) — link
  41. Multi-Feature Supervised Reinforcement Learning for Stock Trading (2023) — link
  42. Single stock trading with deep reinforcement learning: A comparative study (2022) — link
  43. Asynchronous Methods for Deep Reinforcement Learning (2016) — link
  44. Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor (2018) — link
  45. Addressing Function Approximation Error in Actor-Critic Methods (2018) — link
  46. Optimal ratio for data splitting (2022) — link
  47. Explore the Use of Prompt-Based LLM for Credit Risk Classification (2025) — link

4 of 55 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2596 linked papers.