Paper: arXiv 2502.11433
Authors: Guojun Xiong, Zhiyang Deng, Keyi Wang, Yupeng Cao, Haohang Li, Yangyang Yu, Xueqing Peng, Mingquan Lin, Kaleb E Smith, Xiao-Yang Liu, Jimin Huang, Sophia Ananiadou, Qianqian Xie
Abstract
Large language models (LLMs) fine-tuned on multimodal financial data have demonstrated impressive reasoning capabilities in various financial tasks. However, they often struggle with multi-step, goal-oriented scenarios in interactive financial markets, such as trading, where complex agentic approaches are required to improve decision-making. To address this, we propose \textsc{“FLAG-Trader”}, a unified architecture integrating linguistic processing (via LLMs) with gradient-driven reinforcement learning (RL) policy optimization, in which a partially fine-tuned LLM acts as the policy network, leveraging pre-trained knowledge while adapting to the financial domain through parameter-efficient fine-tuning. Through policy gradient optimization driven by trading rewards, our framework not only enhances LLM performance in trading but also improves results on other financial-domain tasks. We present extensive empirical evidence to validate these enhancements.
Complexity vs Empirical Score
- Math Complexity: 7.5/10
- Empirical Rigor: 8.0/10
- Quadrant: Holy Grail — high math complexity, high empirical rigor
Why this score: The paper employs advanced mathematical concepts such as Markov Decision Processes, policy gradients, and parameter-efficient fine-tuning, but also presents extensive empirical evidence, including backtested trading performance metrics and comparisons with baseline strategies.
Research Flowchart
flowchart TD A["Research Goal: Improve LLM Decision-Making for Multi-Step Financial Trading"] --> B["Data Input: Multimodal Financial Data"] B --> C["Methodology: FLAG-Trader Architecture"] C --> D["LLM as Policy Network<br>Parameter-Efficient Fine-Tuning"] C --> E["Gradient-Based RL<br>Policy Gradient Optimization"] D --> F["Computational Process:<br>Trading Reward Signal"] E --> F F --> G["Key Findings/Outcomes:<br>Enhanced Trading Performance<br>Improved General Financial Reasoning"]
Evidence Card
What the authors claim, on what data, and who has checked it. Read from the paper's own text by a language model: every quote is verbatim, nothing was re-run. How to read this card
Proposes
The paper proposes FLAG‑Trader, a unified architecture that integrates LLMs with gradient‑driven reinforcement learning policy optimization, using a partially fine‑tuned LLM as the policy network.
Source sentence
we propose \textsc{FLAG-Trader}, a unified architecture integrating linguistic processing (via LLMs) with gradient-driven reinforcement learning (RL) policy optimization
Assumes 4
- Only a subset of the LLM parameters is updated during fine‑tuning. model
Source sentence
Rather than fine-tuning the entire LLM, only a subset of its parameters is up‑dated
- Market data is represented as a textual state for the LLM. data
Source sentence
The model processes market data using a textual state representation
- Training uses the PPO reinforcement‑learning algorithm. model
Source sentence
FLAG‑TRADER is trained by using PPO algorithm
- Cumulative Return (CR) and Sharpe Ratio (SR) are the primary evaluation metrics. statistical
Source sentence
Accordingly, we treat CR and SR as our primary metrics for the final evaluation
Data
No dataset named in the text.
Costs
Unclear
Finds 3
- FLAG‑TRADER consistently outperforms both the buy‑and‑hold strategy and LLM‑agentic baselines, especially in cumulative return and Sharpe ratio. performance
Source sentence
Our results demonstrate that FLAG‑TRADER consistently outperforms both the buy‑and‑hold strategy and LLM‑agentic baselines, particularly in terms of cumulative return and Sharpe ratio
- A small‑scale (135M‑parameter) open‑source LLM fine‑tuned with RL surpasses much larger proprietary models. performance
Source sentence
Notably, our approach enables a small‑scale (135M parameter) open‑source LLM to surpass much larger proprietary models
- Extensive experiments across multiple stock trading scenarios show FLAG‑TRADER outperforms baseline methods. performance
Source sentence
Through extensive experiments across multiple stock trading scenarios, we demonstrated that FLAG‑TRADER consistently outperforms baseline methods
Checked by others
No other paper in the archive has been found to support or contradict these findings (compared with 44 related papers). Coverage is partial, so this is neither replication nor refutation.
7 papers in the archive : SBCA: Cross-Modal BERT-driven Actor-Critic for Multi-Asset …
Code
No repository by the authors found in the paper.
Extracted 2026-11-14 · same card as JSON · A language model chose these items: check the source sentence before relying on one.
Cited References (79)
- Proximal policy optimization algorithms (2017) — link
- Probability inequalities (2010)
- Human-level control through deep reinforcement learning (2015)
- On-line Q-learning using connectionist systems (1994)
- Learning from delayed rewards (1989)
- Reinforcement learning for trading (1998)
- Simple statistical gradient-following algorithms for connectionist reinforcement learning (1992)
- Trust Region Policy Optimization (2015) — link
and 71 more
- Asynchronous Methods for Deep Reinforcement Learning (2016) — link
- Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor (2018)
- Continuous control with deep reinforcement learning (2015) — link
- Deterministic policy gradient algorithms (2014)
- FinRL-Meta: Market environments and benchmarks for data-driven financial reinforcement learning (2022)
- FinRL: Deep reinforcement learning framework to automate trading in quantitative finance (2021)
- An algorithm for trading and portfolio management using q-learning and sharpe ratio maximization (2000)
- Adaptive stock trading with dynamic asset allocation using reinforcement learning (2006)
- A multi-agent Q-learning framework for optimizing stock trading systems (2002)
- Reinforcement learning for high-frequency market making (2018)
- Learning FX trading strategies with FQI and persistent actions (2021)
- Practical deep reinforcement learning approach for stock trading (2018) — link
- Deep reinforcement learning for financial trading using price trailing (2019)
- A tabular sarsa-based stock market agent (2020)
- Improving financial trading decisions using deep Q-learning: Predicting the number of shares, action strategies, and transfer learning (2019)
- MetaTrader: An reinforcement learning approach integrating diverse policies for portfolio optimization (2022)
- DeepScalper: A risk-aware reinforcement learning framework to capture fleeting intraday trading opportunities (2022)
- Using data augmentation based reinforcement learning for daily stock trading (2020)
- Fingpt: Open-source financial large language models — → 2306.06031
- Decision transformer: Reinforcement learning via sequence modeling (2021)
- Trading with the momentum transformer: An intelligent and interpretable architecture (2021) — link
- The finben: An holistic financial benchmark for large language models (2024) — link
- Open-finllms: Open multimodal large language models for financial applications — → 2408.11878
- Time-llm: Time series forecasting by reprogramming large language models (2023) — link
- Cryptotrade: A reflective llm-based agent to guide zero-shot cryptocurrency trading (2024)
- TradingGPT: Multi-agent system with layered memory and distinct characters for enhanced financial trading performance — → 2309.03736
- Fincon: A synthesized llm multi-agent system with conceptual verbal reinforcement for enhanced financial decision making (2025)
- What do llms know about financial markets? a case study on reddit market sentiment analysis (2023)
- FinMem: A performance-enhanced LLM trading agent with layered memory and character design — → 2311.13743
- Text mining for market prediction: A systematic review (2014)
- Deep Reinforcement Learning for Trading (2019) — link
- An automated FX trading system using adaptive reinforcement learning (2006)
- Deep direct reinforcement learning for financial signal representation and trading (2016)
- A multi-scale temporal feature aggregation convolutional neural network for portfolio management (2019)
- A multi-objective deep reinforcement learning approach for stock index future’s intraday trading (2017)
- Relation-aware transformer for portfolio policy learning (2021)
- Commission fee is not enough: A hierarchical reinforced framework for portfolio management (2021)
- Cost-sensitive portfolio selection via deep reinforcement learning (2020)
- DeepTrader: a deep reinforcement learning approach for risk-return balanced portfolio management with market conditions Embedding (2021)
- Reinforcement-learning based portfolio management with augmented asset movement prediction states (2020)
- Adversarial deep reinforcement learning in portfolio management (2018) — link
- An end-to-end optimal trade execution framework based on proximal policy optimization (2021)
- Deep reinforcement learning for automated stock trading: An ensemble strategy (2020)
- Quantitative day trading from natural language using reinforcement learning (2021)
- A novel deep reinforcement learning framework with BiLSTM-Attention networks for algorithmic trading (2024)
- Deep reinforcement learning for financial trading using multi-modal features (2024)
- Recent advances in reinforcement learning in finance (2023)
- Reinforcement learning for quantitative trading (2023)
- INVESTORBENCH: A Benchmark for Financial Decision-Making Tasks with LLM-based Agent — → 2412.18174
- Risk Management and Financial Institutions (2007)
- Large language models as generalizable policies for embodied tasks (2023)
- Fine-Tuning Large Vision-Language Models as Decision-Making Agents via Reinforcement Learning (2024) — link
- Convfinqa: Exploring the chain of numerical reasoning in conversational finance question answering (2022) — link
- Finbert: A pre-trained financial language representation model for financial text mining (2021)
- Finbert: A pretrained language model for financial communications (2020) — link
- Finbert: Financial sentiment analysis with pre-trained language models (2019) — link
- FinBERT: A large language model for extracting information from financial text (2023)
- FinBERT–MRC: financial named entity recognition using BERT under the machine reading comprehension paradigm (2023)
- When flue meets flang: Benchmarks and large pre-trained language model for financial domain (2022) — link
- Fingpt: Democratizing internet-scale data for financial large language models (2023) — link
- Pixiu: A large language model, instruction data and evaluation benchmark for finance (2023) — link
- Bloomberggpt: A large language model for finance (2023) — link
- Investlm: A large language model for investment using financial domain instruction tuning — → 2309.13064
- Learning to generate explainable stock predictions using self-reflective large language models — → 2402.03659
- Can large language models beat wall street? unveiling the potential of ai in stock selection — → 2401.03737
- FinAgent: A Multimodal Foundation Agent for Financial Trading: Tool-Augmented, Diversified, and Generalist — → 2402.18485
- FinRobot: An Open-Source AI Agent Platform for Financial Applications using Large Language Models — → 2405.14767
- When AI Meets Finance (StockAgent): Large Language Model-based Stock Trading in Simulated Real-world Environments — → 2407.18957
- TradeMaster: a holistic quantitative trading platform empowered by reinforcement learning (2024)
- Fincon: A synthesized llm multi-agent system with conceptual verbal reinforcement for enhanced financial decision making (2024) — link
- An overview of machine learning, deep learning, and reinforcement learning-based techniques in quantitative finance: recent progress and challenges (2023)
11 of 79 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2596 linked papers.