{"paper_id":"2306.10950","title":"Benchmarking Robustness of Deep Reinforcement Learning approaches to Online Portfolio Management","authors":["Marc Velay","Bich-Liên Doan","Arpad Rimmel","Fabrice Popineau","Fabrice Daniel"],"paper_date":"2023-06-19","paper_type":"dataset-benchmark","paper_url":"https://arxiv.org/pdf/2306.10950v1","page_url":"https://thequant.space/flowcharts/benchmarking-robustness-of-deep-reinforcement-learning-appro/","scores":{"empirical_rigor":6,"math_complexity":7.5,"quadrant":"Holy Grail"},"proposes":{"status":"available","statement":"The paper proposes a training and evaluation process to assess the performance of classical deep reinforcement learning algorithms for portfolio management.","quote":"We propose a training and evaluation process to assess the performance of classical DRL algorithms for portfolio management.","summary":"Deep Reinforcement Learning approaches to Online Portfolio Selection have grown in popularity in recent years. The sensitive nature of training Reinforcement Learning agents implies a need for extensive efforts in market representation, behavior objectives, and training processes, which have often…","strategy":null},"assumptions":{"status":"available","items":[{"assumption":"The financial market environment is highly uncertain and non‑stationary.","type":"market","stated":"implied","quote":"OLPS is unlike many other RL problems due to its high uncertainty"},{"assumption":"Agents can keep a portion of wealth in cash (liquidity) to reduce portfolio volatility.","type":"model","stated":"implied","quote":"We add the possibility for the agent to keep liquidity to reduce portfolio volatility."},{"assumption":"Training uses a fixed starting portfolio value of 100,000 USD.","type":"execution","stated":"implied","quote":"The starting value of the portfolio was 100000USD."},{"assumption":"During training, 20 stocks are randomly subsampled from the S&P500 constituents.","type":"data","stated":"implied","quote":"During training, 20 stocks are randomly subsampled."},{"assumption":"Backtesting evaluates a manually selected set of 20 assets from diverse industries.","type":"execution","stated":"implied","quote":"For backtesting, we manually selected 20 assets from diverse industries with moderate to high returns for the period."}]},"datasets":{"status":"available","items":[{"name":"S&P500 constituent stocks","provider":"public market data (S&P500 constituents as of Jan 2023)","kind":"market-prices","access":"public","period":"2010-01 to 2022-12","universe":"500 S&P500 constituent stocks (20‑asset portfolio sampled)","quote":"We focused on S&P500 constituent stocks at the date of January 2023, for the period from January 2010 to December 2022."}],"known":[],"sample":{"start":null,"end":null,"market":null,"asset_class":null,"frequency":null},"data_resolution":"daily"},"transaction_costs":{"status":"available","modeled":"no","headline_net_of_costs":null,"detail":null,"quote":null,"source":"front_matter","rule":"R1","candidates":{"front_matter":"no","evidence_card":"unclear","evidence_card_quoted":false},"meaning":"a trading or execution result is reported without costs"},"results":{"status":"available","evaluation_type":"trading-pnl","findings":[{"finding":"Most deep reinforcement learning algorithms were not robust, with strategies generalizing poorly and degrading quickly during backtesting.","kind":"negative","quote":"We found that most Deep Reinforcement Learning algorithms were not robust, with strategies generalizing poorly and degrading quickly during backtesting."},{"finding":"Overfitting agents achieved good validation performance but quickly became obsolete when deployed.","kind":"negative","quote":"we have found that overfitting agents predicting the same allocation weights regardless of selected assets and market changes had good performance measures in validation but would quickly become obsolete if deployed."}],"reported_performance":null,"out_of_sample":{"method":"holdout","headline_out_of_sample":null}},"independent_evidence":{"status":"available","supports":[{"paper":{"paper_id":"2509.12764","title":"Myopic Optimality: why reinforcement learning portfolio management strategies lose money","paper_date":"2025-09-16","authors":["Yuming Ma"],"empirical_rigor":4,"page_url":"https://thequant.space/flowcharts/myopic-optimality-why-reinforcement-learning-portfolio-mana/"},"question":"Do reinforcement learning portfolio management strategies provide robust and profitable performance?","this_paper_finding":"Most deep reinforcement learning algorithms were not robust, with strategies generalizing poorly and degrading quickly during backtesting.","other_paper_finding":"Myopic optimization outperforms reinforcement learning: RL yields lower or negative returns, higher variance, larger costs, heavier CVaR, lower profitability, and greater model risk.","other_paper_quote":"Myopic optimization (MO) outperforms reinforcement learning (RL) in portfolio management: RL yields lower or negative returns, higher variance, larger costs, heavier CVaR, lower profitability, and greater model risk.","note":"Both findings agree that RL methods perform poorly: A reports lack of robustness and quick degradation, while B shows lower/negative returns, higher variance, costs, CVaR, and model risk.","confidence":0.912}],"contradicts":[],"qualifies":[],"compared_with":10,"citations":{"cited_by_in_corpus":1,"by_independent_authors":1,"how_cited":{"used_as_baseline":0,"uses_method_or_data":0,"extends":0,"discusses_findings":0,"mentions":1,"unclassified":0},"independent_papers":[],"scope":"Citations between papers in this corpus only; citations from outside it are not counted."}},"implementation":{"status":"available","public":false,"code_url":null,"note":"No repository presented by the authors as their own was found in the paper. Absence here is not proof that no code exists."},"replication":{"status":"none_found","card":null,"note":"No replication of this paper has been run by thequant.space yet."},"extraction":{"text_source":"fulltext","model":"openai/gpt-oss-120b","version":1,"extracted_at":"2026-11-14T18:25:28Z"},"provenance":{"claims":"Everything here is what the authors state in the paper. Nothing was re-run or reproduced by the platform.","quotes":"Assumptions, datasets, cost detail and findings were extracted by a language model. Every quote is verbatim paper text. Where an item has `support`, \"full\" means every figure in it appears in the quoted sentences and \"partial\" means a figure is stated elsewhere in the paper; items with a figure found nowhere in the paper were discarded.","datasets_known":"datasets.known lists well-known vendors and public datasets by a stable id, found by a dictionary over the full text where the sentence says the data is used; it is independent of the model and covers only names in the dictionary.","flags":"transaction_costs.modeled, results.evaluation_type and results.out_of_sample.method are platform classifications (structured-decision models); see https://thequant.space/score-guide/#penalty-flags","comparisons":"supports / contradicts / qualifies are a language model's reading of two papers' quoted findings, restricted to closely related papers that share no author. They are pointers to read, not verified replications, and coverage is partial.","implementation":"code_url is a repository the paper itself presents as its own (rule-scored, ambiguous cases judged by a model), checked for liveness when it was recorded."},"docs":"https://thequant.space/agents/"}