{"paper_id":"2507.18560","title":"HARLF: Hierarchical Reinforcement Learning and Lightweight LLM-Driven Sentiment Integration for Financial Portfolio Optimization","authors":["Benjamin Coriat","Eric Benhamou"],"paper_date":"2025-07-24","paper_type":"empirical","paper_url":"https://arxiv.org/pdf/2507.18560v1","page_url":"https://thequant.space/flowcharts/harlf-hierarchical-reinforcement-learning-and-lightweight-l/","scores":{"empirical_rigor":6,"math_complexity":7.5,"quadrant":"Holy Grail"},"proposes":{"status":"available","statement":"The paper proposes a hierarchical reinforcement learning framework that integrates lightweight large language models with deep RL to combine sentiment signals from financial news with traditional market indicators for portfolio optimization.","quote":"This paper presents a novel hierarchical framework for portfolio optimization, integrating lightweight Large Language Models (LLMs) with Deep Reinforcement Learning (DRL) to combine sentiment signals from financial news with traditional market indicators.","summary":"This paper presents a novel hierarchical framework for portfolio optimization, integrating lightweight Large Language Models (LLMs) with Deep Reinforcement Learning (DRL) to combine sentiment signals from financial news with traditional market indicators. Our three-tier architecture employs base RL…","strategy":"Hierarchical RL with LLM-driven sentiment (Super-Agent)"},"assumptions":{"status":"available","items":[{"assumption":"Data inputs are synchronously available.","type":"data","stated":"explicit","quote":"It assumes synchronously available data inputs, which may not align with real-world asynchronous market conditions."},{"assumption":"Transaction costs are excluded from the reported returns.","type":"execution","stated":"explicit","quote":"Transaction costs are excluded, potentially overestimating practical returns"}]},"datasets":{"status":"available","items":[{"name":"Daily adjusted closing prices","provider":"Yahoo Finance (yfinance library)","kind":"market-prices","access":"public","period":"2003-01-01 to 2024-12-31","universe":"14 financial assets (e.g., GSPC for S&P 500)","quote":"We gather daily adjusted closing prices for 14 financial asset from January 1, 2003, to December 31, 2024. This data is fetched using the yfinance Python library, which connects to Yahoo Finance.","known_id":"yahoo-finance"},{"name":"Financial news sentiment scores","provider":"FinBERT (domain‑specific LLM)","kind":"text-news","access":"public","period":null,"universe":null,"quote":"Base agents, using Stable Baselines 3 algorithms, process monthly financial metrics from YahooFinance or sentiment scores from financial news via FinBERT"}],"known":[],"sample":{"start":"2018","end":"2024","market":"US equities","asset_class":"equities","frequency":"monthly"},"data_resolution":"daily"},"transaction_costs":{"status":"available","modeled":"no","headline_net_of_costs":false,"detail":null,"quote":"Transaction costs are excluded, potentially overestimating practical returns","source":"front_matter","rule":"R3","candidates":{"front_matter":"no","evidence_card":"no","evidence_card_quoted":true},"card_detail":null,"card_quote":"Transaction costs are excluded, potentially overestimating practical returns","meaning":"a trading or execution result is reported without costs"},"results":{"status":"available","evaluation_type":"trading-pnl","findings":[{"finding":"Both meta‑agents and the super‑agent outperform the equal‑weighted portfolio (7.5% ROI, Sharpe 0.57) and the S&P 500 (13.2% ROI, Sharpe 0.63).","kind":"performance","quote":"Equal-Weights 7.5 0.57 13.3 ... S&P 500 13.2 0.63 19.7"},{"finding":"Compared to state‑of‑the‑art RL strategies, the super‑agent’s 26.0% ROI exceeds CNN‑RL’s 22.0% ROI and matches the best reported ROI.","kind":"performance","quote":"Closely competing with CNN‑RL (22.0% ROI, 1.3 Sharpe), our super agent seems to have surpassed the current state of the art."}],"reported_performance":{"strategy":"Hierarchical RL with LLM-driven sentiment (Super-Agent)","sharpe":1.2,"sharpe_benchmark":0.57,"annual_return_pct":26,"total_return_pct":null,"max_drawdown_pct":null},"out_of_sample":{"method":"holdout","headline_out_of_sample":true}},"independent_evidence":{"status":"none_found","supports":[],"contradicts":[],"qualifies":[],"compared_with":7,"citations":{"cited_by_in_corpus":0,"by_independent_authors":0,"how_cited":{"used_as_baseline":0,"uses_method_or_data":0,"extends":0,"discusses_findings":0,"mentions":0,"unclassified":0},"independent_papers":[],"scope":"Citations between papers in this corpus only; citations from outside it are not counted."}},"implementation":{"status":"available","public":false,"code_url":null,"note":"No repository presented by the authors as their own was found in the paper. Absence here is not proof that no code exists."},"replication":{"status":"none_found","card":null,"note":"No replication of this paper has been run by thequant.space yet."},"extraction":{"text_source":"fulltext","model":"openai/gpt-oss-120b","version":1,"extracted_at":"2026-11-14T17:31:12Z"},"provenance":{"claims":"Everything here is what the authors state in the paper. Nothing was re-run or reproduced by the platform.","quotes":"Assumptions, datasets, cost detail and findings were extracted by a language model. Every quote is verbatim paper text. Where an item has `support`, \"full\" means every figure in it appears in the quoted sentences and \"partial\" means a figure is stated elsewhere in the paper; items with a figure found nowhere in the paper were discarded.","datasets_known":"datasets.known lists well-known vendors and public datasets by a stable id, found by a dictionary over the full text where the sentence says the data is used; it is independent of the model and covers only names in the dictionary.","flags":"transaction_costs.modeled, results.evaluation_type and results.out_of_sample.method are platform classifications (structured-decision models); see https://thequant.space/score-guide/#penalty-flags","comparisons":"supports / contradicts / qualifies are a language model's reading of two papers' quoted findings, restricted to closely related papers that share no author. They are pointers to read, not verified replications, and coverage is partial.","implementation":"code_url is a repository the paper itself presents as its own (rule-scored, ambiguous cases judged by a model), checked for liveness when it was recorded."},"docs":"https://thequant.space/agents/"}