{"paper_id":"2410.14927","title":"Hierarchical Reinforced Trader (HRT): A Bi-Level Approach for Optimizing Stock Selection and Execution","authors":["Zijie Zhao","Roy E. Welsch"],"paper_date":"2024-10-19","paper_type":"empirical","paper_url":"https://arxiv.org/pdf/2410.14927v1","page_url":"https://thequant.space/flowcharts/hierarchical-reinforced-trader-hrt-a-bi-level-approach-fo/","scores":{"empirical_rigor":6,"math_complexity":7,"quadrant":"Holy Grail"},"proposes":{"status":"available","statement":"The paper proposes Hierarchical Reinforced Trader (HRT), a bi‑level reinforcement learning framework that separates sparse high‑level directional selection from risk‑aware low‑level execution for text‑aware portfolio management.","quote":"We propose Hierarchical Reinforced Trader (HRT), a bi-level reinforcement learning framework for text-aware portfolio management in multi-asset equity markets.","summary":"Leveraging Deep Reinforcement Learning (DRL) in automated stock trading has shown promising results, yet its application faces significant challenges, including the curse of dimensionality, inertia in trading actions, and insufficient portfolio diversification. Addressing these challenges, we…","strategy":"Hierarchical Reinforced Trader (HRT)"},"assumptions":{"status":"available","items":[{"assumption":"Trading decisions are based on a fixed 89‑stock Nasdaq universe.","type":"market","stated":"explicit","quote":"fixed 89-stock Nasdaq universe, using 2013–2018 for training, 2019 for validation, and 2020–2023 for final out-of-sample testing"},{"assumption":"Text signals are pre‑computed, timestamp‑aligned sentiment and risk scores.","type":"data","stated":"explicit","quote":"text is converted into timestamp-aligned structured sentiment and risk signals"},{"assumption":"Turnover, drawdown and text‑risk penalties constrain the low‑level controller.","type":"execution","stated":"explicit","quote":"risk-aware Low-Level Controller (LLC) converts these directions into feasible portfolio weight adjustments under turnover, drawdown, and text-risk penalties"},{"assumption":"Transaction costs are modeled as a linear turnover penalty and examined under stress‑test cost levels.","type":"model","stated":"explicit","quote":"reduces daily turnover from 0.112 to 0.090, and remains robust under transaction-cost stress"},{"assumption":"The benchmark protocol assumes no data leakage and uses a public timestamp‑clean protocol.","type":"other","stated":"explicit","quote":"public benchmark data availability under the same timestamp-clean text-aware protocol"}]},"datasets":{"status":"available","items":[{"name":"open stock‑news benchmark","provider":null,"kind":"benchmark","access":"public","period":"2013–2023","universe":"89‑stock Nasdaq universe","quote":"We evaluate HRT on an open stock-news benchmark with a fixed 89-stock Nasdaq universe, using 2013–2018 for training, 2019 for validation, and 2020–2023 for final out-of-sample testing"},{"name":"FN‑SPID financial news dataset","provider":null,"kind":"text-news","access":"public","period":null,"universe":null,"quote":"benchmark constructed from FN‑SPID financial news and daily market data with precomputed LLM‑generated sentiment and risk signals"}],"known":[{"id":"fnspid","name":"FNSPID","provider":"Dong et al.","kind":"text-news","access":"public","quote":"4.1 Benchmark and Experimental Setup We evaluate HRT on the open FinRL-DeepSeek stock-trading bench-mark, a FinRL-compatible benchmark constructed from FNSPID financial news and daily market data with precomputed LLM-generated sentiment and risk signals [1, 3, 12].","mentions":2,"used_in":1}],"sample":{"start":"2020","end":"2023","market":"US equities (89‑stock Nasdaq universe)","asset_class":"equities","frequency":"daily"},"data_resolution":"daily"},"transaction_costs":{"status":"available","modeled":"yes","headline_net_of_costs":true,"detail":"Costs are incorporated via a turnover penalty and evaluated across 0–50 bps per trade in sensitivity analysis.","quote":"All strategies deteriorate as transaction costs increase, but HRT achieves the highest Sharpe ratio among the reported strategies across the evaluated 1–50 bps cost settings","source":"front_matter","rule":"R3","candidates":{"front_matter":"yes","evidence_card":"yes","evidence_card_quoted":true},"card_detail":"Costs are incorporated via a turnover penalty and evaluated across 0–50 bps per trade in sensitivity analysis.","card_quote":"All strategies deteriorate as transaction costs increase, but HRT achieves the highest Sharpe ratio among the reported strategies across the evaluated 1–50 bps cost settings","meaning":"costs are part of the reported headline results"},"results":{"status":"available","evaluation_type":"trading-pnl","findings":[{"finding":"HRT improves Sharpe ratio from 1.06 (HRT‑Base) to 1.24 and reduces daily turnover from 0.112 to 0.090 in out‑of‑sample testing (2020‑2023).","kind":"performance","quote":"The full model improves Sharpe from 1.06 for HRT-Base to 1.24, reduces daily turnover from 0.112 to 0.090"},{"finding":"Maximum drawdown improves from –0.305 to –0.245 when moving from HRT‑Base to full HRT.","kind":"performance","quote":"improving maximum drawdown from -0.305 to -0.245"},{"finding":"Ablation shows that adding sparse high‑level activation, text‑risk signal, and risk‑aware LLC sequentially improves Sharpe and reduces turnover and drawdown.","kind":"performance","quote":"Sparse high-level activation sacrifices some raw return but tends to improve Sharpe, drawdown, and turnover. Adding text-derived risk information reduces ex-post text-risk exposure and improves path-level downside control. The full risk-aware LLC further improves the return–risk–cost balance"},{"finding":"Under transaction‑cost stress tests (0–50 bps), HRT maintains the highest Sharpe ratio among all evaluated strategies.","kind":"performance","quote":"HRT achieves the highest Sharpe ratio among the reported strategies across the evaluated 1–50 bps cost settings"},{"finding":"The hierarchical design yields a stronger overall return‑risk‑cost trade‑off compared to flat RL, alpha‑only selection, and non‑RL baselines.","kind":"performance","quote":"HRT delivers the strongest learning-based return–risk–cost trade-off"}],"reported_performance":{"strategy":"Hierarchical Reinforced Trader (HRT)","sharpe":1.24,"sharpe_benchmark":0.8,"annual_return_pct":23.4,"total_return_pct":132.1,"max_drawdown_pct":24.5},"out_of_sample":{"method":"holdout","headline_out_of_sample":true}},"independent_evidence":{"status":"none_found","supports":[],"contradicts":[],"qualifies":[],"compared_with":23,"citations":{"cited_by_in_corpus":1,"by_independent_authors":1,"how_cited":{"used_as_baseline":0,"uses_method_or_data":0,"extends":0,"discusses_findings":0,"mentions":1,"unclassified":0},"independent_papers":[],"scope":"Citations between papers in this corpus only; citations from outside it are not counted."}},"implementation":{"status":"available","public":false,"code_url":null,"note":"No repository presented by the authors as their own was found in the paper. Absence here is not proof that no code exists."},"replication":{"status":"none_found","card":null,"note":"No replication of this paper has been run by thequant.space yet."},"extraction":{"text_source":"fulltext","model":"openai/gpt-oss-120b","version":1,"extracted_at":"2026-11-14T17:58:22Z"},"provenance":{"claims":"Everything here is what the authors state in the paper. Nothing was re-run or reproduced by the platform.","quotes":"Assumptions, datasets, cost detail and findings were extracted by a language model. Every quote is verbatim paper text. Where an item has `support`, \"full\" means every figure in it appears in the quoted sentences and \"partial\" means a figure is stated elsewhere in the paper; items with a figure found nowhere in the paper were discarded.","datasets_known":"datasets.known lists well-known vendors and public datasets by a stable id, found by a dictionary over the full text where the sentence says the data is used; it is independent of the model and covers only names in the dictionary.","flags":"transaction_costs.modeled, results.evaluation_type and results.out_of_sample.method are platform classifications (structured-decision models); see https://thequant.space/score-guide/#penalty-flags","comparisons":"supports / contradicts / qualifies are a language model's reading of two papers' quoted findings, restricted to closely related papers that share no author. They are pointers to read, not verified replications, and coverage is partial.","implementation":"code_url is a repository the paper itself presents as its own (rule-scored, ambiguous cases judged by a model), checked for liveness when it was recorded."},"docs":"https://thequant.space/agents/"}