Paper: arXiv 2507.18560
Authors: Benjamin Coriat, Eric Benhamou
Abstract
This paper presents a novel hierarchical framework for portfolio optimization, integrating lightweight Large Language Models (LLMs) with Deep Reinforcement Learning (DRL) to combine sentiment signals from financial news with traditional market indicators. Our three-tier architecture employs base RL agents to process hybrid data, meta-agents to aggregate their decisions, and a super-agent to merge decisions based on market data and sentiment analysis. Evaluated on data from 2018 to 2024, after training on 2000-2017, the framework achieves a 26% annualized return and a Sharpe ratio of 1.2, outperforming equal-weighted and S&P 500 benchmarks. Key contributions include scalable cross-modal integration, a hierarchical RL structure for enhanced stability, and open-source reproducibility.
Complexity vs Empirical Score
- Math Complexity: 7.5/10
- Empirical Rigor: 6.0/10 (capped from 8.0: transaction costs not modelled; see penalty flags)
- Quadrant: Holy Grail — high math complexity, high empirical rigor (effective empirical rigor capped from 8.0)
Why this score: The paper employs advanced mathematical concepts in deep reinforcement learning, hierarchical architectures, and Markowitz portfolio theory, while also presenting a backtest-ready framework with specific performance metrics (26% annualized return, 1.2 Sharpe), detailed data pipelines, and open-source reproducibility.
Research Flowchart
flowchart TD A["Research Goal:<br>Integrate LLM Sentiment & DRL<br>for Portfolio Optimization"] --> B["Data Inputs:<br>Financial News & Market Data<br>(2000-2024)"] B --> C["HARLF Architecture"] C --> D["1. Base RL Agents:<br>Process hybrid data"] D --> E["2. Meta-Agents:<br>Aggregate decisions"] E --> F["3. Super-Agent:<br>Final sentiment-weighted action"] F --> G["Backtesting:<br>2018-2024 Period"] G --> H["Outcomes:<br>26% Annualized Return<br>Sharpe Ratio 1.2"]
Evidence Card
What the authors claim, on what data, and who has checked it. Read from the paper's own text by a language model: every quote is verbatim, nothing was re-run. How to read this card
Proposes
The paper proposes a hierarchical reinforcement learning framework that integrates lightweight large language models with deep RL to combine sentiment signals from financial news with traditional market indicators for portfolio optimization.
Source sentence
This paper presents a novel hierarchical framework for portfolio optimization, integrating lightweight Large Language Models (LLMs) with Deep Reinforcement Learning (DRL) to combine sentiment signals from financial news with traditional market indicators.
Assumes 2
- Data inputs are synchronously available. data
Source sentence
It assumes synchronously available data inputs, which may not align with real-world asynchronous market conditions.
- Transaction costs are excluded from the reported returns. execution
Source sentence
Transaction costs are excluded, potentially overestimating practical returns
Data
2018 – 2024 · US equities · equities · monthly
- Daily adjusted closing prices market-prices public
Source sentence
We gather daily adjusted closing prices for 14 financial asset from January 1, 2003, to December 31, 2024. This data is fetched using the yfinance Python library, which connects to Yahoo Finance.
- Financial news sentiment scores text-news public
Source sentence
Base agents, using Stable Baselines 3 algorithms, process monthly financial metrics from YahooFinance or sentiment scores from financial news via FinBERT
Costs
Not modeled
Source sentence
Transaction costs are excluded, potentially overestimating practical returns
Finds 2
Sharpe 1.20 (benchmark 0.57) · annual return 26.0%
- Both meta‑agents and the super‑agent outperform the equal‑weighted portfolio (7.5% ROI, Sharpe 0.57) and the S&P 500 (13.2% ROI, Sharpe 0.63). performance
Source sentence
Equal-Weights 7.5 0.57 13.3 ... S&P 500 13.2 0.63 19.7
- Compared to state‑of‑the‑art RL strategies, the super‑agent’s 26.0% ROI exceeds CNN‑RL’s 22.0% ROI and matches the best reported ROI. performance
Source sentence
Closely competing with CNN‑RL (22.0% ROI, 1.3 Sharpe), our super agent seems to have surpassed the current state of the art.
Checked by others
No other paper in the archive has been found to support or contradict these findings (compared with 7 related papers). Coverage is partial, so this is neither replication nor refutation.
Code
No repository by the authors found in the paper.
Extracted 2026-11-14 · same card as JSON · A language model chose these items: check the source sentence before relying on one.
Cited References (45)
- Structure and Interpretation of Computer Programs (1985)
- Visual Information Extraction with Lixto (2001)
- An overview of the KL-ONE knowledge representation system (1985)
- Complexity results for nonmonotonic logics (1992)
- Hypertree Decompositions and Tractable Queries (2002)
- Foundations of a functional approach to knowledge representation (1984)
- A logic of implicit and explicit belief (1984)
- On the compilability and expressive power of propositional planning formalisms (2000)
and 37 more
- IJCAI Camera Ready Submission
- FinBERT: Financial Sentiment Analysis with Pre-trained Language Models (2019) — link
- Deciphering the Liquidity and Credit Crunch 2007–2008 (2009) — link
- Generalized Mean-Variance Frameworks for Portfolio Optimization: A Comprehensive Literature Review (2009) — link
- Deep Direct Reinforcement Learning for Financial Signal Representation and Trading (2016) — link
- A Deep Reinforcement Learning Framework for the Financial Portfolio Management Problem (2017) — link
- Adversarial Deep Reinforcement Learning in Portfolio Management (2018) — link
- FinRL: A Deep Reinforcement Learning Library for Automated Stock Trading in Quantitative Finance (2020) — link
- Portfolio Selection (1952) — link
- Giving Content to Investor Sentiment: The Role of Media in the Stock Market (2007) — link
- BloombergGPT: A Large Language Model for Finance (2023) — link
- Proximal Policy Optimization Algorithms (2017) — link
- Continuous Control with Deep Reinforcement Learning (2015) — link
- Addressing Function Approximation Error in Actor-Critic Methods (2018) — link
- Financial News-Driven LLM Reinforcement Learning for Portfolio Management — → 2411.11059
- Stable-Baselines3: Reliable Reinforcement Learning Implementations (2021) — link
- Optimizing Performance: How Compact Models Match or Exceed GPT’s Classification Capabilities through Fine-Tuning — → 2409.11408
- Sentiment-Driven Reinforcement Learning for Portfolio Optimization (2023) — link
- A Systematic Approach to Portfolio Optimization: A Comparative Study of Reinforcement Learning Agents, Market Signals, and Investment Horizons (2024)
- News-Driven Reinforcement Learning for Algorithmic Trading (2018) — link
- Large Language Models in Finance: A Survey — → 2311.10723
- Finance Wizard at the FinLLM Challenge Task: Financial Text Summarization (2024) — link
- L3iTC at the FinLLM Challenge Task: Quantization for Financial Text Classification and Summarization (2024) — link
- CatMemo at the FinLLM Challenge Task: Fine-Tuning Large Language Models using Data Fusion in Financial Applications — → 2407.01953
- Twitter Mood Predicts the Stock Market (2011) — link
- OpenAI Gym (2016) — link
- Wisdom of Crowds: The Value of Stock Opinions Transmitted through Social Media (2019) — link
- Sentiment Analysis and its Impact on Financial Markets: A Comprehensive Review (2021) — link
- Human-level Control through Deep Reinforcement Learning (2015) — link
- Between MDPs and Semi-MDPs: A Framework for Temporal Abstraction in Reinforcement Learning (1999) — link
- Hierarchical Deep Reinforcement Learning: Integrating Temporal Abstraction and Intrinsic Motivation (2016)
- Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor (2018) — link
- Large Language Models for Contextual Financial Sentiment Analysis — → 2404.02053
- Deep Reinforcement Learning for Financial Trading (2019) — link
- Financial Sentiment Analysis on News and Reports Using Large Language Models (2024) — link
- Enhancing Financial Sentiment Analysis via Retrieval-Augmented Large Language Models — → 2310.04027
- ‘Finance Wizard’at the FinLLM Challenge Task: Financial Text Summarization (2024) — link
6 of 45 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2596 linked papers.