Paper: arXiv 2509.01393
Authors: Qizhao Chen, Hiroaki Kawashima
Abstract
This paper proposes a reinforcement learning framework that employs Proximal Policy Optimization (PPO) to dynamically optimize the weights of multiple large language model (LLM)-generated formulaic alphas for stock trading strategies. Formulaic alphas are mathematically defined trading signals derived from price, volume, sentiment, and other data. Although recent studies have shown that LLMs can generate diverse and effective alphas, a critical challenge lies in how to adaptively integrate them under varying market conditions. To address this gap, we leverage the deepseek-r1-distill-llama-70b model to generate fifty alphas for five major stocks: Apple, HSBC, Pepsi, Toyota, and Tencent, and then use PPO to adjust their weights in real time. Experimental results demonstrate that the PPO-optimized strategy achieves strong returns and high Sharpe ratios across most stocks, outperforming both an equal-weighted alpha portfolio and traditional benchmarks such as the Nikkei 225, S&P 500, and Hang Seng Index. The findings highlight the importance of reinforcement learning in the allocation of alpha weights and show the potential of combining LLM-generated signals with adaptive optimization for robust financial forecasting and trading.
Complexity vs Empirical Score
- Math Complexity: 7.5/10
- Empirical Rigor: 8.0/10
- Quadrant: Holy Grail — high math complexity, high empirical rigor
Why this score: The paper employs advanced reinforcement learning (PPO) and deals with mathematical formulation of alpha signals, indicating moderate to high math complexity, while the experimental setup with specific stock tickers, Sharpe ratios, and comparisons to benchmarks demonstrates strong empirical rigor.
Research Flowchart
flowchart TD
A["Research Goal: <br>Adaptive Alpha Weighting"] --> B["Data: 50 LLM-Generated Alphas<br>5 Stocks: Apple, HSBC, Pepsi, Toyota, Tencent"]
B --> C["Core Methodology:<br>Proximal Policy Optimization PPO"]
C --> D{"Compute Portfolio Weighted by<br>PPO-Optimized Alpha Weights"}
D --> E["Execution: <br>Trading Strategy Backtest"]
E --> F["Outcome: <br>Higher Returns & Sharpe Ratios"]
F --> G["Comparison: <br>Outperforms Equal-Weight & Benchmarks S&P 500, Nikkei 225"]
Evidence Card
What the authors claim, on what data, and who has checked it. Read from the paper's own text by a language model: every quote is verbatim, nothing was re-run. How to read this card
Proposes
The paper proposes a reinforcement learning framework using Proximal Policy Optimization to dynamically weight multiple LLM‑generated formulaic alphas for stock trading.
Source sentence
This paper introduces a reinforcement learning framework that employs Proximal Policy Optimization (PPO) to dynamically optimize the weights of multiple large language model (LLM)-generated formulaic alphas for stock trading strategies.
Assumes 6
- Trades are executed at the closing price of day t with no information beyond day t. execution
Source sentence
at the close price of day t, and no information beyond day t is used when determining trading positions.
- Positions are held from day t to day t+1 and returns are realized using the next‑day return. execution
Source sentence
... Trades are assumed to be executed immediately after the decision at day t, and the resulting position pt is held over the interval from day t to day t + 1.
- Alpha signals are normalized to approximately a standard normal distribution. model
Source sentence
αi,t Alpha signals Normalized to ∼N(0, 1)
- A soft penalty is applied when the position sign conflicts with a regime indicator defined by moving‑average crossover. market
Source sentence
The market regime indicator regimet ∈{0, 1} is defined based on the moving-average crossover, where regimet = 1 denotes a bullish regime (20-day MA above 100-day MA) and regimet = 0 indicates a bearish regime.
Show 2 more
- Transaction costs are a fixed 0.1% per trade. execution
Source sentence
TCt Transaction cost per trade 0.001
- Volatility targeting scales positions to keep annualized volatility near 15%. model
Source sentence
σtarget Annualized volatility target 0.15 (fixed)
Data
US stocks · equities · daily
- Ten selected stocks market-prices unknown
Source sentence
we leverage a DeepSeek model to generate fifty alphas for ten stocks
Yahoo Finance
Costs
Modeled
Fixed cost of 0.001 (0.1%) per trade is included in the reward calculation.
Source sentence
TCt Transaction cost per trade 0.001
Finds 5
Sharpe 2.41 · max drawdown 1.0%
- The PPO‑adjusted strategy does not consistently achieve the highest cumulative returns but attains higher Sharpe ratios and lower maximum drawdowns in most cases. performance
Source sentence
Experimental results indicate that the PPO‑optimized strategy does not consistently deliver the highest cumulative returns across all stocks, but it achieves comparatively higher Sharpe ratios and smaller maximum drawdowns in most cases.
- Sharpe ratio improvements over buy‑and‑hold are statistically significant for several stocks (e.g., Apple +1.77, HSBC +0.79). performance
Source sentence
Apple 1.7695* [0.1787, 3.4424] 0.0296 ... HSBC 0.7928* [0.4321, 2.2641] 0.0006
- Diebold‑Mariano tests show negative DM statistics, indicating PPO outperforms baseline strategies for many stocks. statistical
Source sentence
A negative DM stat indicates PPO outperforms the benchmark.
- Average daily turnover remains low for most stocks (e.g., Netflix 0.0090) suggesting limited impact of transaction costs. performance
Source sentence
Average daily turnover remains low for several stocks, including Netflix (0.0090) and HSBC (0.0348), and reaches its highest level for Airbus (0.2007).
- Ablation analysis shows that reducing the number of alphas can slightly improve cumulative return and Sharpe for some stocks but may worsen performance for others. other
Source sentence
reducing the number of alphas leads to slight improvements in both cumulative return and Sharpe ratio for certain stocks. However, for others, their performance may worsen after reducing the number of alphas.
Checked by others
No other paper in the archive has been found to support or contradict these findings (compared with 57 related papers). Coverage is partial, so this is neither replication nor refutation.
Code
No repository by the authors found in the paper.
Extracted 2026-11-14 · same card as JSON · A language model chose these items: check the source sentence before relying on one.
Cited References (55)
- Technical Analysis Indicators in Stock Market Using Machine Learning: A Comparative Analysis (2021) — link
- Key technical indicators for stock market prediction (2025) — link
- Machine learning approaches in stock market prediction: A systematic literature review (2023) — link
- Interpretability of Machine Learning: Recent Advances and Future Prospects (2023) — link
- Automate Strategy Finding with LLM in Quant investment — → 2409.06289
- Stock Price Prediction Using LLM-Based Sentiment Analysis (2024) — link
- Common risk factors in the returns on stocks and bonds (1993) — link
- Short and long term stock trend prediction using decision tree (2017) — link
and 47 more
- Stocks market prediction using Support Vector Machine (2013) — link
- Stock Price Prediction Model Based on RBF-SVM Algorithm (2020) — link
- Research on stock forecasting based on random forest (2022) — link
- Stock Price Prediction Using CNN and LSTM-Based Deep Learning Models (2020) — link
- Stock Price Prediction using LSTM (2023) — link
- A CNN-LSTM-Based Model to Forecast Stock Prices (2020) — link
- Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting (2021) — link
- Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecasting (2022) — link
- TimeXer: Empowering Transformers for Time Series Forecasting with Exogenous Variables (2024) — link
- MASTER: Market-Guided Stock Transformer for Stock Price Forecasting (2023) — link
- FinGPT: Open-Source Financial Large Language Models — → 2306.06031
- PIXIU: A Large Language Model, Instruction Data and Evaluation Benchmark for Finance (2023) — link
- Automate Strategy Finding with LLM in Quant Investment — → 2409.06289
- GPT-Signal: Generative AI for Semi-automated Feature Engineering in the Alpha Research Process (2024) — link
- Proximal Policy Optimization Algorithms (2017) — link
- 101 Formulaic Alphas (2016) — link
- The unreasonable effectiveness of deep learning in artificial intelligence (2020) — link
- The alpha life cycle of quantitative strategy (2017) — link
- Stock Price Change Prediction Using Prompt-Based LLM s with RL -Enhanced Post-Hoc Adjustments (2025) — link
- A Deep Reinforcement Learning Framework for Dynamic Portfolio Optimization: Evidence from China’s Stock Market — → 2412.18563
- Advancing Investment Frontiers: Industry-grade Deep Reinforcement Learning for Portfolio Optimization (2024) — link
- Adaptive Investment Strategies: Deep Reinforcement Learning Approaches for Portfolio Optimization (2024) — link
- Distilling the Knowledge in a Neural Network (2015) — link
- Contrarian Investment Strategies: The Next Generation (1998)
- From local explanations to global understanding with explainable AI for trees (2020)
- A novel sentiment correlation-based method with dual transformer model for stock price prediction (2025) — link
- Sentiment-Aware Stock Price Prediction with Transformer and LLM-Generated Formulaic Alpha (2025) — link
- An end-to-end optimal trade execution framework based on proximal policy optimization (2021)
- Comparative Analysis of A3C and PPO Algorithms in Reinforcement Learning: A Survey on General Environments (2024) — link
- Proximal policy optimization via enhanced exploration efficiency (2022) — link
- Reinforcement Learning in a Virtual World: A Study of PPO and SAC within Unity ML-Agents (2025) — link
- Stock Price Prediction with LLM -Guided Market Movement Signals and Transformer Model (2025) — link
- Image-Driven Stock Price Prediction with LLaMA : A Prompt-Based Approach (2025) — link
- Explore Anomaly-Aware Transformers for Robust Financial Time Series Forecasting (2025) — link
- A Framework for Measuring How News Topics Drive Stock Movement (2025) — link
- Offline Reinforcement Learning for Automated Stock Trading (2023) — link
- Enhancing Deep Reinforcement Learning for Stock Trading: A Reward Shaping Approach via Expert Feedback (2025) — link
- Stock Market Trading via Actor–Critic Reinforcement Learning and Adaptable Data Structure (2025) — link
- A Novel RMS-Driven Deep Reinforcement Learning for Optimized Portfolio Management in Stock Trading (2025) — link
- Stock Closing Price Prediction using Machine Learning Techniques (2020) — link
- Multi-Feature Supervised Reinforcement Learning for Stock Trading (2023) — link
- Single stock trading with deep reinforcement learning: A comparative study (2022) — link
- Asynchronous Methods for Deep Reinforcement Learning (2016) — link
- Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor (2018) — link
- Addressing Function Approximation Error in Actor-Critic Methods (2018) — link
- Optimal ratio for data splitting (2022) — link
- Explore the Use of Prompt-Based LLM for Credit Risk Classification (2025) — link
4 of 55 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2596 linked papers.