Paper: arXiv 2502.11433

Authors: Guojun Xiong, Zhiyang Deng, Keyi Wang, Yupeng Cao, Haohang Li, Yangyang Yu, Xueqing Peng, Mingquan Lin, Kaleb E Smith, Xiao-Yang Liu, Jimin Huang, Sophia Ananiadou, Qianqian Xie

Abstract

Large language models (LLMs) fine-tuned on multimodal financial data have demonstrated impressive reasoning capabilities in various financial tasks. However, they often struggle with multi-step, goal-oriented scenarios in interactive financial markets, such as trading, where complex agentic approaches are required to improve decision-making. To address this, we propose \textsc{“FLAG-Trader”}, a unified architecture integrating linguistic processing (via LLMs) with gradient-driven reinforcement learning (RL) policy optimization, in which a partially fine-tuned LLM acts as the policy network, leveraging pre-trained knowledge while adapting to the financial domain through parameter-efficient fine-tuning. Through policy gradient optimization driven by trading rewards, our framework not only enhances LLM performance in trading but also improves results on other financial-domain tasks. We present extensive empirical evidence to validate these enhancements.

Complexity vs Empirical Score

  • Math Complexity: 7.5/10
  • Empirical Rigor: 8.0/10
  • Quadrant: Holy Grail — high math complexity, high empirical rigor

Why this score: The paper employs advanced mathematical concepts such as Markov Decision Processes, policy gradients, and parameter-efficient fine-tuning, but also presents extensive empirical evidence, including backtested trading performance metrics and comparisons with baseline strategies.

Research Flowchart

  flowchart TD
  A["Research Goal: Improve LLM Decision-Making for Multi-Step Financial Trading"] --> B["Data Input: Multimodal Financial Data"]
  B --> C["Methodology: FLAG-Trader Architecture"]
  C --> D["LLM as Policy Network<br>Parameter-Efficient Fine-Tuning"]
  C --> E["Gradient-Based RL<br>Policy Gradient Optimization"]
  D --> F["Computational Process:<br>Trading Reward Signal"]
  E --> F
  F --> G["Key Findings/Outcomes:<br>Enhanced Trading Performance<br>Improved General Financial Reasoning"]

Evidence Card

What the authors claim, on what data, and who has checked it. Read from the paper's own text by a language model: every quote is verbatim, nothing was re-run. How to read this card

Proposes

The paper proposes FLAG‑Trader, a unified architecture that integrates LLMs with gradient‑driven reinforcement learning policy optimization, using a partially fine‑tuned LLM as the policy network.

Source sentence
we propose \textsc{FLAG-Trader}, a unified architecture integrating linguistic processing (via LLMs) with gradient-driven reinforcement learning (RL) policy optimization

Assumes 4

  • Only a subset of the LLM parameters is updated during fine‑tuning. model
    Source sentence
    Rather than fine-tuning the entire LLM, only a subset of its parameters is up‑dated
  • Market data is represented as a textual state for the LLM. data
    Source sentence
    The model processes market data using a textual state representation
  • Training uses the PPO reinforcement‑learning algorithm. model
    Source sentence
    FLAG‑TRADER is trained by using PPO algorithm
  • Cumulative Return (CR) and Sharpe Ratio (SR) are the primary evaluation metrics. statistical
    Source sentence
    Accordingly, we treat CR and SR as our primary metrics for the final evaluation

Data

No dataset named in the text.

Costs

Unclear The classifier was not confident.

Finds 3

  • FLAG‑TRADER consistently outperforms both the buy‑and‑hold strategy and LLM‑agentic baselines, especially in cumulative return and Sharpe ratio. performance
    Source sentence
    Our results demonstrate that FLAG‑TRADER consistently outperforms both the buy‑and‑hold strategy and LLM‑agentic baselines, particularly in terms of cumulative return and Sharpe ratio
  • A small‑scale (135M‑parameter) open‑source LLM fine‑tuned with RL surpasses much larger proprietary models. performance
    Source sentence
    Notably, our approach enables a small‑scale (135M parameter) open‑source LLM to surpass much larger proprietary models
  • Extensive experiments across multiple stock trading scenarios show FLAG‑TRADER outperforms baseline methods. performance
    Source sentence
    Through extensive experiments across multiple stock trading scenarios, we demonstrated that FLAG‑TRADER consistently outperforms baseline methods

Checked by others

No other paper in the archive has been found to support or contradict these findings (compared with 44 related papers). Coverage is partial, so this is neither replication nor refutation.

Cited by 7 papers in the archive (6 by independent authors): SBCA: Cross-Modal BERT-driven Actor-Critic for Multi-Asset … (uses its method or data)

Code

No repository by the authors found in the paper.

Extracted 2026-11-14 · same card as JSON · A language model chose these items: check the source sentence before relying on one.

Cited References (79)

  1. Proximal policy optimization algorithms (2017) — link
  2. Probability inequalities (2010)
  3. Human-level control through deep reinforcement learning (2015)
  4. On-line Q-learning using connectionist systems (1994)
  5. Learning from delayed rewards (1989)
  6. Reinforcement learning for trading (1998)
  7. Simple statistical gradient-following algorithms for connectionist reinforcement learning (1992)
  8. Trust Region Policy Optimization (2015) — link
and 71 more
  1. Asynchronous Methods for Deep Reinforcement Learning (2016) — link
  2. Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor (2018)
  3. Continuous control with deep reinforcement learning (2015) — link
  4. Deterministic policy gradient algorithms (2014)
  5. FinRL-Meta: Market environments and benchmarks for data-driven financial reinforcement learning (2022)
  6. FinRL: Deep reinforcement learning framework to automate trading in quantitative finance (2021)
  7. An algorithm for trading and portfolio management using q-learning and sharpe ratio maximization (2000)
  8. Adaptive stock trading with dynamic asset allocation using reinforcement learning (2006)
  9. A multi-agent Q-learning framework for optimizing stock trading systems (2002)
  10. Reinforcement learning for high-frequency market making (2018)
  11. Learning FX trading strategies with FQI and persistent actions (2021)
  12. Practical deep reinforcement learning approach for stock trading (2018) — link
  13. Deep reinforcement learning for financial trading using price trailing (2019)
  14. A tabular sarsa-based stock market agent (2020)
  15. Improving financial trading decisions using deep Q-learning: Predicting the number of shares, action strategies, and transfer learning (2019)
  16. MetaTrader: An reinforcement learning approach integrating diverse policies for portfolio optimization (2022)
  17. DeepScalper: A risk-aware reinforcement learning framework to capture fleeting intraday trading opportunities (2022)
  18. Using data augmentation based reinforcement learning for daily stock trading (2020)
  19. Fingpt: Open-source financial large language models — → 2306.06031
  20. Decision transformer: Reinforcement learning via sequence modeling (2021)
  21. Trading with the momentum transformer: An intelligent and interpretable architecture (2021) — link
  22. The finben: An holistic financial benchmark for large language models (2024) — link
  23. Open-finllms: Open multimodal large language models for financial applications — → 2408.11878
  24. Time-llm: Time series forecasting by reprogramming large language models (2023) — link
  25. Cryptotrade: A reflective llm-based agent to guide zero-shot cryptocurrency trading (2024)
  26. TradingGPT: Multi-agent system with layered memory and distinct characters for enhanced financial trading performance — → 2309.03736
  27. Fincon: A synthesized llm multi-agent system with conceptual verbal reinforcement for enhanced financial decision making (2025)
  28. What do llms know about financial markets? a case study on reddit market sentiment analysis (2023)
  29. FinMem: A performance-enhanced LLM trading agent with layered memory and character design — → 2311.13743
  30. Text mining for market prediction: A systematic review (2014)
  31. Deep Reinforcement Learning for Trading (2019) — link
  32. An automated FX trading system using adaptive reinforcement learning (2006)
  33. Deep direct reinforcement learning for financial signal representation and trading (2016)
  34. A multi-scale temporal feature aggregation convolutional neural network for portfolio management (2019)
  35. A multi-objective deep reinforcement learning approach for stock index future’s intraday trading (2017)
  36. Relation-aware transformer for portfolio policy learning (2021)
  37. Commission fee is not enough: A hierarchical reinforced framework for portfolio management (2021)
  38. Cost-sensitive portfolio selection via deep reinforcement learning (2020)
  39. DeepTrader: a deep reinforcement learning approach for risk-return balanced portfolio management with market conditions Embedding (2021)
  40. Reinforcement-learning based portfolio management with augmented asset movement prediction states (2020)
  41. Adversarial deep reinforcement learning in portfolio management (2018) — link
  42. An end-to-end optimal trade execution framework based on proximal policy optimization (2021)
  43. Deep reinforcement learning for automated stock trading: An ensemble strategy (2020)
  44. Quantitative day trading from natural language using reinforcement learning (2021)
  45. A novel deep reinforcement learning framework with BiLSTM-Attention networks for algorithmic trading (2024)
  46. Deep reinforcement learning for financial trading using multi-modal features (2024)
  47. Recent advances in reinforcement learning in finance (2023)
  48. Reinforcement learning for quantitative trading (2023)
  49. INVESTORBENCH: A Benchmark for Financial Decision-Making Tasks with LLM-based Agent — → 2412.18174
  50. Risk Management and Financial Institutions (2007)
  51. Large language models as generalizable policies for embodied tasks (2023)
  52. Fine-Tuning Large Vision-Language Models as Decision-Making Agents via Reinforcement Learning (2024) — link
  53. Convfinqa: Exploring the chain of numerical reasoning in conversational finance question answering (2022) — link
  54. Finbert: A pre-trained financial language representation model for financial text mining (2021)
  55. Finbert: A pretrained language model for financial communications (2020) — link
  56. Finbert: Financial sentiment analysis with pre-trained language models (2019) — link
  57. FinBERT: A large language model for extracting information from financial text (2023)
  58. FinBERT–MRC: financial named entity recognition using BERT under the machine reading comprehension paradigm (2023)
  59. When flue meets flang: Benchmarks and large pre-trained language model for financial domain (2022) — link
  60. Fingpt: Democratizing internet-scale data for financial large language models (2023) — link
  61. Pixiu: A large language model, instruction data and evaluation benchmark for finance (2023) — link
  62. Bloomberggpt: A large language model for finance (2023) — link
  63. Investlm: A large language model for investment using financial domain instruction tuning — → 2309.13064
  64. Learning to generate explainable stock predictions using self-reflective large language models — → 2402.03659
  65. Can large language models beat wall street? unveiling the potential of ai in stock selection — → 2401.03737
  66. FinAgent: A Multimodal Foundation Agent for Financial Trading: Tool-Augmented, Diversified, and Generalist — → 2402.18485
  67. FinRobot: An Open-Source AI Agent Platform for Financial Applications using Large Language Models — → 2405.14767
  68. When AI Meets Finance (StockAgent): Large Language Model-based Stock Trading in Simulated Real-world Environments — → 2407.18957
  69. TradeMaster: a holistic quantitative trading platform empowered by reinforcement learning (2024)
  70. Fincon: A synthesized llm multi-agent system with conceptual verbal reinforcement for enhanced financial decision making (2024) — link
  71. An overview of machine learning, deep learning, and reinforcement learning-based techniques in quantitative finance: recent progress and challenges (2023)

11 of 79 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2596 linked papers.