{"paper_id":"2402.16609","title":"Combining Transformer based Deep Reinforcement Learning with Black-Litterman Model for Portfolio Optimization","authors":["Ruoyu Sun","Angelos Stefanidis","Zhengyong Jiang","Jionglong Su"],"paper_date":"2024-02-23","paper_type":"empirical","paper_url":"https://arxiv.org/pdf/2402.16609v1","page_url":"https://thequant.space/flowcharts/combining-transformer-based-deep-reinforcement-learning-with/","scores":{"empirical_rigor":8,"math_complexity":7.5,"quadrant":"Holy Grail"},"proposes":{"status":"available","statement":"The paper proposes a hybrid portfolio optimization model that combines a deep reinforcement learning (DRL) agent with the Black‑Litterman (BL) model to learn dynamic asset correlations and implement a long/short strategy.","quote":"In this research, we propose a hybrid portfolio optimization model combining the DRL agent and the Black-Litterman (BL) model to enable the DRL agent to learn the dynamic correlation between the portfolio asset returns and implement an efficacious long/short strategy.","summary":"As a model-free algorithm, deep reinforcement learning (DRL) agent learns and makes decisions by interacting with the environment in an unsupervised way. In recent years, DRL algorithms have been widely applied by scholars for portfolio optimization in consecutive trading periods, since the DRL…","strategy":"DRL agent with Black-Litterman model"},"assumptions":{"status":"available","items":[{"assumption":"Zero Market Impact: trading actions do not affect asset price fluctuations.","type":"execution","stated":"explicit","quote":"Zero Market Impact: The trading actions made by the agent do not affect the asset price fluctuation."},{"assumption":"Zero Slippage: selected assets have abundant liquidity and orders are executed immediately.","type":"execution","stated":"explicit","quote":"Zero Slippage: The selected assets in the portfolio have notably abundant liquidity in the market. Hence, whether the long position trade orders or short selling orders are issued, they can be immediately executed after they are issued."},{"assumption":"Risk‑free rate is assumed to be zero.","type":"model","stated":"explicit","quote":"The risk‑free rate is assumed to be zero in our research."},{"assumption":"One share cannot be divided; target quantities are integer via floor function.","type":"other","stated":"explicit","quote":"In Equation (4), we apply the floor function so that the target quantity … contains integers since we assume that one share of stock cannot be divided."}]},"datasets":{"status":"available","items":[{"name":"Dow Jones Industrial Average constituent stocks","provider":"Yahoo Finance","kind":"market-prices","access":"public","period":null,"universe":"30 prominent U.S. companies (29 after removing missing data)","quote":"We use DJIA constituent stocks to construct our portfolio. After removing stocks with missing data, 29 constituent stocks remain.","known_id":"yahoo-finance"}],"known":[{"id":"yahoo-finance","name":"Yahoo Finance","provider":"Yahoo","kind":"market-prices","access":"public","quote":"Data availability This research uses only publicly available American stock data from Yahoo Finance platform.","mentions":2,"used_in":1}],"sample":{"start":null,"end":null,"market":"US stock market (Dow Jones Industrial Average constituents)","asset_class":"equities","frequency":null},"data_resolution":"daily"},"transaction_costs":{"status":"available","modeled":"yes","headline_net_of_costs":true,"detail":"Commission fee rate, lending rate of cash, lending rate of stock, trading cost and short‑position cost are included in the cash value update equation.","quote":"represent the commission fee rate, lending rate of cash, and lending rate of stock.","source":"front_matter","rule":"R3","candidates":{"front_matter":"yes","evidence_card":"yes","evidence_card_quoted":true},"card_detail":"Commission fee rate, lending rate of cash, lending rate of stock, trading cost and short‑position cost are included in the cash value update equation.","card_quote":"represent the commission fee rate, lending rate of cash, and lending rate of stock.","meaning":"costs are part of the reported headline results"},"results":{"status":"available","evaluation_type":"trading-pnl","findings":[{"finding":"The DRL agent outperforms various comparison portfolio strategies and alternative DRL frameworks by at least 42% in accumulated return on US stock data.","kind":"performance","quote":"demonstrate that our DRL agent significantly outperforms various comparison portfolio choice strategies and alternative DRL frameworks by at least 42% in terms of accumulated return."},{"finding":"The proposed BDA outperforms its two variants by at least 247% in accumulated return and by at least 24% in Sharpe ratio and Sortino‑Tail ratio.","kind":"performance","quote":"The empirical results show that our BDA outperform the variants by at least 247% in terms of AR. In terms of SR and STR, our BDA outperforms the variants by at least 24% in all experiments."},{"finding":"Using the BL model for weight determination yields better performance than a softmax‑based variant, improving accumulated return by at least 246% and return‑per‑unit‑of‑risk by at least 7%.","kind":"performance","quote":"BDA exhibits better performance in accumulated return and return per unit of risk. Specifically, our BDA outperforms the variant of BDA (BDA-V4) by at least 246% in terms of AR. In terms of SR and STR, our BDA outperforms the variant of BDA (BDA-V4) by at least 7%."},{"finding":"Extending the trading period to five days improves out‑sample accumulated return and risk‑adjusted performance, with the BDA outperforming a one‑day variant by at least 12% in SR and STR.","kind":"performance","quote":"the empirical results indicate that extending the trading period to five days is necessary."}],"reported_performance":{"strategy":"DRL agent with Black-Litterman model","sharpe":null,"sharpe_benchmark":null,"annual_return_pct":null,"total_return_pct":null,"max_drawdown_pct":null},"out_of_sample":{"method":"walk-forward","headline_out_of_sample":true}},"independent_evidence":{"status":"none_found","supports":[],"contradicts":[],"qualifies":[],"compared_with":29,"citations":{"cited_by_in_corpus":3,"by_independent_authors":3,"how_cited":{"used_as_baseline":0,"uses_method_or_data":0,"extends":0,"discusses_findings":0,"mentions":2,"unclassified":1},"independent_papers":[],"scope":"Citations between papers in this corpus only; citations from outside it are not counted."}},"implementation":{"status":"available","public":false,"code_url":null,"note":"No repository presented by the authors as their own was found in the paper. Absence here is not proof that no code exists."},"replication":{"status":"none_found","card":null,"note":"No replication of this paper has been run by thequant.space yet."},"extraction":{"text_source":"fulltext","model":"openai/gpt-oss-120b","version":1,"extracted_at":"2026-11-14T18:14:14Z"},"provenance":{"claims":"Everything here is what the authors state in the paper. Nothing was re-run or reproduced by the platform.","quotes":"Assumptions, datasets, cost detail and findings were extracted by a language model. Every quote is verbatim paper text. Where an item has `support`, \"full\" means every figure in it appears in the quoted sentences and \"partial\" means a figure is stated elsewhere in the paper; items with a figure found nowhere in the paper were discarded.","datasets_known":"datasets.known lists well-known vendors and public datasets by a stable id, found by a dictionary over the full text where the sentence says the data is used; it is independent of the model and covers only names in the dictionary.","flags":"transaction_costs.modeled, results.evaluation_type and results.out_of_sample.method are platform classifications (structured-decision models); see https://thequant.space/score-guide/#penalty-flags","comparisons":"supports / contradicts / qualifies are a language model's reading of two papers' quoted findings, restricted to closely related papers that share no author. They are pointers to read, not verified replications, and coverage is partial.","implementation":"code_url is a repository the paper itself presents as its own (rule-scored, ambiguous cases judged by a model), checked for liveness when it was recorded."},"docs":"https://thequant.space/agents/"}